Category Archives: Computer geek stuff

UnRAID hidden files problem

For the longest time (ever since I had set up unRaid) I had an issue with the files saved on the unRaid system showing up as hidden files when viewed in a microsoft windows environment.

Never really liked that. My solution had been to enable viewing of hidden and system files, but it was a workaround. Never really knew why this was happening, until now.

I came across this post on the Unraid Forums. Just need to add a couple of lines to discourage samba to make the files hidden and system files.

Xenserver: moving data to remote file server

Leave a reply

I am considering moving the Xenserver machines to a network location, removing them from the box in which the hypervisor is residing.

I have a gigabit network which theoretically can transport just over 100MB/s worth of data (about 125MB/s I think is the calculation) but I expect less then that in the real world.

A 100Mb network can do 12.5MB/s, but realistically, only about 9MB/s is capable after all the networking overhead is introduced, so assuming 90MB/s for a gigabit connection may not be unreasonable.

HD normally put out 40-60MB/s, so a gigabit network should not be a bottleneck (assuming only one connection).

Going to run some tests to see what throughput my LAN can sustain.

Running bonnie++ on /mnt/tower/disk1 I get

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
apps-server05 2072M 22582  11 23699   1 15228   0 53534  35 62733   1 165.6   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   832   2  1852   1  1079   1   812   1  2153   0   840   1

Locally on the workstation running Bonnie++ on the SSD drive with a 8gig file (4gigs ram in machine) I get:

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
workstation01 7416M 63910  62 49520  20 24627  11 96748  92 233131  38  2300   9
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++

On the same workstation, using a 7200rpm 160 gig drive, I get:

To be done

I did some file copies with SCP (since it reports the transfer rate).
Between machines on the xen hypervisor with a local file I am getting about 75MB/s
Between same machines, with a file stored on the UnRAID server I am getting about 30MB/s in read speed. This is obviously limited by both the UnRAID machine or the network connection.

Did another SCP between my workstation machine and a virtual machine and I topped out at 9.9MB/s
Will have to investigate this further since I’m obviously only getting 100Mbit speeds, when I have a 1Gb connection? Maybe not…

Copy recursively files of one type

Leave a reply

I needed to copy a bunch of JPGs from one folder structure to a mirror structure, but didn’t want to copy all the RAW files. Came across two examples. I tried the second (mainly because I understand it), and it worked nicely.

Example 1

Code:

#!/bin/bash
cd /home/images
find . -depth -type f -name '*.jpg' | cpio --pass-through \
 --preserve-modification-time \
 --make-directories --verbose /home/new_dir

Example 2

Code:

#!/bin/bash
cd /home/images
find . -type d -name '*' -exec mkdir -p /home/new_dir/{} \;
find . -type f -name '*.jpg' -exec cp {} /home/new_dir/{} \;

Access to folders outside chroot

Leave a reply

I’m setting up an ftp server on my home machine, and wanted to give access to a user to files/folders outside their chrooted account.

This describes an elegant way to do it.

http://www.ducea.com/2006/07/27/allowing-ftp-access-to-files-outside-the-home-directory-chroot/

New Album Releases

Leave a reply

I am happy with the way the Discogs Album Release plugin is coming together.

Version 1.0 does what I need it for now.

Version 2.x has new features I want to add. The main feature is adding an audio preview of each album so that you can listen to the tracks directly from my site. This will only work if Amazon has the album on their database, thought the majority of albums I’ve come across are on Amazon.com.

Using code from this site and this site, I’ve made a php script that searches the Amazon.com database for a keyword (artist + album) and returns the ASIN # (This is the unique number given to each amazon item).

If I don’t get a number back, then the album doesn’t exist. If I get a number back then I can add a music player that pops up so the viewer can listen to the Amazon previews of the tracks and maybe buy them.
This check needs to happen every time the page is loaded, so that if in the future when the album is added to amazon.com it will show up. On the other hand, is the complexity of doing this worth the work, considering I doubt many people will be looking at month old releases…

Another thing for 2.x is to build some sort of widget that shows a list of the latest albums. Maybe have it update every time the page is loaded and display the latest releases. This brings up the question of the necessity of the daily digest version of the album releases…

I also signed up for a merchant account so if a user buys the album, I get a cut of the actions. Wohoo!