I am going to try to write something to give me equivalent features to WHS drive extender, to be used in linux (maybe later, even windows?)
But to start, I need to think things through:
Function
The program will be based around creating symbolic links from files on the pool of specified drives, and amalgamate them to be accessible from one central location.
There will be certain limitations to using symbolic links. Renaming files will not be possible(update: renaming may be possible with a bit of work) Other limitations are unknown right now.
The way the system will work:
There will be a cache folder on the main system.
The cache folder is where all the data appears to be coming from. This is also where all the data is written to.
What the software will do (at user defined intervals, maybe through cron jobs?) during a re-sync operation is scan the pool of drives/folders and first create a mirror of all the folders/subfolders and then create links to all the files.
The re-sync will attempt to resynchronize the contents of the pool of drives to the contents of the cache folder.
The software will also look at what files have been written to the cache folder and depending on what subfolder it was written to, it will move that file to drive/filder location that has the most space available. Some drives/folders will be allowed to be designated as locations where no new material should be added.
When the new material is added to a virgin drive, the entire folder structure that the file requires will be built to accommodate the file’s location.
This takes care of reading and writing of files.
Re-writing over an old file, will just re-write the final file location.
Moving files requires a re-sync.
Deleting files will have 2 modes (which can be preset in the config file). Upon deleting a link, only the link is deleted (the actual file remains) or upon deleting a file, upon re-sync, the actual file is also deleted.
Hopefully this is enough of a description for me to start programing and use as a guideline.
I would prefer to not depend on any external databases to keep track of files in both locations.
Some commands that seem useful:
Following obtained from here:
ls -l | grep “^l” ### lists all the links only
ls -l | grep -v “^l” ### lists only the files and dir only (opposite of above)