Python
Aim:
To design and implement DEDUCT, a secure and efficient data deduplication method for textual data. The goal is to reduce storage demands while ensuring data confidentiality and supporting resource-constrained devices.
Abstract:
The rapid increase in textual data in navigation tasks for devices like GPS or smart assistants creates challenges for managing and storing data in large-scale systems. Data deduplication, which reduces storage needs by eliminating duplicate data, offers a solution but raises security concerns. This paper introduces DEDUCT, a new method that combines cloud-side and client-side deduplication to achieve high data compression while protecting data privacy.
Designed for devices with limited resources, such as IoT devices, DEDUCT includes lightweight preprocessing and safeguards against security risks like side-channel attacks. Testing on a navigation dataset shows that DEDUCT can compress data by up to 66%, significantly cutting storage costs while keeping data secure, making it an efficient choice for managing large-scale data systems.
Existing System:
The existing system for managing encrypted data focuses on efficient storage and transmission by utilizing two main components: the Cloud Service Provider (CSP) and authorized clients. The CSP stores encrypted data and employs a pointer-based approach to avoid storing duplicate data, saving valuable storage space. Authorized clients, who have limited storage resources, follow a five-step process before uploading data. This process begins with tokenization, breaking the data into smaller parts, followed by transformation into base and deviation pairs using the Wagner-Fischer algorithm. Clients then calculate unique CRC values for each base, which are stored locally to detect and avoid duplicates.
After encrypting the data using a predefined encryption key, the client uploads the encrypted base, CRC value, and deviation to the CSP. If the CRC value already exists locally, only the CRC and deviation are sent, further optimizing bandwidth and storage usage. This approach helps ensure data security, reduces redundancy, and minimizes resource consumption in cloud storage systems. However, it lacks advanced integrity checks, which could enhance data reliability and security in the future.
Disadvantage in Existing System:
No File Downloading – Users are unable to download files, limiting access to their stored data.
No File Viewing – Users cannot view details of the chunks of data or the contents of their Encrypted files directly.
No File Sharing – The system does not currently support file sharing, limiting users’ ability to collaborate or distribute their files to others.
Lack of Real-Time Access – Without the ability to download or view files, users cannot access their data in real-time when needed.
information about the individual chunks of their uploaded files.
Proposed System:
In the proposed system, users can easily register, log in, and access their personal accounts with a secure authentication process. After logging in, users are provided with the ability to upload a wide range of file types, including images, PDFs, Word documents, and text files, to the cloud for secure storage. Once a file is uploaded, the system processes the data securely and efficiently. The first step involves tokenizing the file, where the data is divided into smaller, manageable components or tokens.
The Wagner-Fischer algorithm is then applied to transform these tokens into base and deviation pairs, which helps to handle the data more accurately, enabling efficient comparison and deduplication. Following this, CRC values are calculated for each base to detect duplicate data, which minimizes storage redundancy and optimizes space. After processing the data, the system encrypts the file to ensure that it remains secure and confidential during storage in the cloud. Users can view their files securely within the system, with the data remaining encrypted to maintain privacy.
Furthermore, the system provides users with the ability to download their files at any time. When downloading a file, users must decrypt the file using a unique decryption key. This key, which is securely associated with the user’s account, ensures that only authorized users can access and decrypt the file. The decryption process restores the original content of the file, allowing users to view or use the data in its original, unencrypted form.
The decryption key is carefully managed and protected within the system to maintain data confidentiality and prevent unauthorized access. This secure download process ensures that even during transmission, the data remains encrypted and protected, preserving privacy. Overall, the system integrates strong encryption techniques, tokenization for efficient storage, and secure user access, providing a seamless, secure, and efficient file management experience, all while minimizing redundancy and maximizing storage efficiency.
Advantages in Proposed System:
Support for Multiple File Types – The system allows users to upload a variety of file types, including images, PDFs, Word documents, and text files, offering flexibility in file management.
Secure File Viewing The Encrypted Data – Users can view their files securely, with data remaining encrypted to maintain privacy while being accessed.
File Download with Decryption – When downloading, users must decrypt files using a unique decryption key, ensuring that only authorized users can access their data.
File Sharing – The system enables users to easily share their files with other users, facilitating collaboration and data exchange. By securely sharing files
View Chunk Details – Users have the ability to view detailed information about the individual chunks of their uploaded files.
Reviews
There are no reviews yet.