Data Modification Kit (DMK)
Update: While there remain some potential uses of the DMK, after 10 years of success in enterprise solutions worldwide, the DMK is nearing its end of life. Please see OSR’s File Encryption Solution Framework (FESF) to learn about OSR’s next generation file encryption solution for Windows.
Products that Windows file system filter drivers to encrypt, compress, or otherwise modify file-based data require some of the most complex kernel software developed for Windows. Even for seasoned kernel-mode programmers, the complexity of delivering per-file data modification solutions on Windows is significant. When a development team has significant Windows file system expertise, the task can take years. For a development team that is not fortunate enough to include multiple Windows file system and internals experts, the task of actually getting such a product to work can be daunting.
OSR has years of experience working with clients and other members of the development community to deliver such solutions. OSR has gone a step further with the Data Modification Kit (DMK) . Read on, to find out how the OSR DMK can facilitate the development of per-file data modification solutions for Windows that encompass a broad range of features while maintaining correctness, flexibility, and ease of use.
A common use for a Windows file system filter driver is to extend the base OS to provide support for encryption, compression, or other custom modifications on a per-file basis. We call this “per-file data modification.” Often, supporting per-file data modification requires changing the size and format of the data as well as storing additional meta-data with the file. Unfortunately, implementing this level of functionality on Windows is a task riddled with numerous difficulties. Some of the most common difficulties are:
- Support for executable images – Because the executable image is memory mapped, file system filter drivers are bypassed when they interact with the Windows Virtual Memory system regarding the size of the file-backed section. This can lead to incorrect behavior of the application.
- Support for memory mapped data files – Any application that uses the Windows file mapping APIs can cause problems. Such applications are common, including Microsoft Word and Notepad.
- Support for arbitrary data modification – To simplify the task, existing solutions typically restrict their meta-data additions to affixing a header or trailer to the data. This does not provide them with the ability to perform arbitrary data modifications (such as compression, for example).
- Support for all standard Windows file systems – Each Windows file system comes with its own set of idiosyncratic behaviors that must be accommodated. And of course, this results in an increased testing burden because each unique scenario has to be scripted, tested, and have its results validated.
- A changing file system filter model – With Microsoft slowly moving to a newer filter model, most development teams are now faced with the task of having to support the older legacy platforms and yet coexist and work well with the new platforms of the future.
- Subtle, yet crucial, differences among Windows versions – Each version of Windows has introduced changes that affect Windows file systems. Just discovering these changes can be difficult enough; properly accommodating them in a product can be maddening.
There are a number of partial solutions to these issues. However, such solutions can compromise the capabilities of the product and still involve considerable specialized development.
The OSR Data Modification Kit (DMK) addresses these complex issues by providing a comprehensive solution for supporting per-file data modification on Windows.
The DMK’s primary value is that it provides a ready-made infrastructure that properly deals with Windows’ complexities. As a result, regardless of the specifics of your data modification, you need only provide the actual routines that perform the data modification. This simplifies the development process, increases reliability, and decreases the time required to deliver your product to market. Additionally, because the DMK includes source code, it is suitable for use even in highly sensitive projects – everything is available for your inspection, modification, and customization.
The following key features are supported by OSR’s DMK:
- Customizable data modification algorithms. You decide the minimum block size for your algorithm. The DMK calls your data modification function, providing the data requiring modification. In response, your data modification function returns an optional header block plus the transformed data block. Note that the transformed data block need not be the same size as the input data block- you can make it smaller or larger.Benefit – Complete flexibility in choice of algorithm for your solution, regardless of minimum block size.
- Support for multiple, sequential, transformations. Because the OSR DMK allows the output data returned by your library to be any size relative to the input data, you can easily implement multiple transformations.Benefit – Implement compression and encryption in your solution (for example), gaining both disk space and performance, and giving your product another advantage for your customer.
- Support for all file systems. The OSR DMK supports FAT, NTFS, network redirectors, and other Windows file systems.Benefit – Your product works across all common Windows client and server releases, with no source-code changes to your data transformation library.
- Data safety. Even if the data modification Solution Kit is removed from a system that contains transformed files, each transformed file appears as a single file to the base file system, so it can be safely backed up, copied to removable storage, and later re-transformed if required.Benefit – Data integrity, including added meta-data, is maintained.
- Case-by-case file access control. Each time the file is opened, the DMK calls an authorization function that you provide, to perform custom access checks (if desired) and indicate whether file data should be transformed as it is accessed. Alternatively, your transformation library can indicate to the DMK that for a given open file instance the data should be provided as is – without performing any transformation. If your data modification driver is present on both a server and client system, you can (if you wish) shift the work required to transform (or reverse transform) to the client system, distributing workload, potentially decreasing network overhead (e.g., compressed data) and increasing security (e.g., encrypted data).Benefit – Implement your own access control scheme to augment the existing Windows implementation as well as control which version of the data will be seen by individual callers.
- Policy Change. The DMK allows you to change the specific data modification policy for a given file in a reliable and transparent fashion. Thus, the handling of existing files can be changed simply by indicating to the DMK that this should be done.Benefit – Provides tremendous flexibility in reconfiguring the product dynamically based upon policy changes.
Using the DMK provides a number of business advantages as well:
- Time to Market. Because you no longer need to have your team learn the esoteric details of Windows file system filter drivers, you can have them focus on the value-add provided by your application.
- Quality. The OSR team has been developing Windows file system filter drivers for over ten years. This is our core competency. Be assured that this is no hollow claim: We’ve developed the core code behind the most comprehensive, successful, products in the industry.
- World-class support. When your team has problems or questions, your developers interact with our developers. We enjoy success when our clients enjoy success – so we’re here to support your development efforts every step of the way.
- Broadest feature set. The OSR team has made it possible to provide the best possible set of features. By using the DMK, you can support the broadest range of features within your product – giving your product all the advantages of a solid base solution plus the unique features only your team can provide.
The DMK is ideally suited for the following types of data modification:
- Encryption. The DMK allows your product to decide if a particular user may access a given file’s data; your product also can control whether the user sees the clear or encrypted version of the data. If your product is not installed, the file remains a single intact unit, but the stored data is still in its encrypted form.
- Data Compression. The DMK allows arbitrary resizing of the data, allowing your product to trivially incorporate data compression. Further, the DMK provides the infrastructure necessary for handling the issues that can arise with data compression.
The DMK provides support for the following 32-bit and 64-bit versions of Windows:
- Windows XP
- Windows Server 2003
- Windows Vista
- Windows Server 2008
- Windows 7
Quality Assurance and Support
Your investment in the DMK is sure to be a valuable one. The DMK is tested exhaustively by OSR prior to each release to ensure it is of the highest quality. Of course, you are protected against both bugs and obsolescence in the DMK with one full year of maintenance and support from OSR’s world-class technical staff. Following that initial year, optional support services are available to protect your investment into the future.