You know, just once, I would like to go to sleep and wake up, read the newspaper and exclaim: "Wow, that's a really good idea the Government just had!".
Instead, we have the content of the previous blog and today, I wake up to this:
'Mandatory' green projects for householders building conservatories
and
Bank holidays 'cost economy £19bn'
Addressing the former article: Superficially seems like a reasonable idea, except:
The Law of Unintended Consequences will kick in, like it did with the Part P (electrical works brought under building regulations). Regarding Part P - a question in the House of Commons led to the revelation that around 8 deaths per year in England and Wales would be prevented by the introduction of Part P
(see http://www.publications.parliament.uk/pa/cm200405/cmhansrd/vo050113/text/50113w02.htm )
Woopy doo. That was worth the expense then. A fraction of the effort expended and money spent could have reduced the 3000 deaths per year on the road by that amount. Also, I suspect a great many people have adapted to this sort of interfering state nannying by simply not bothering to inform the local council if they are undertaking internal renovation work. Which is a shame as Building Inspectors are very helpful in my personal experience when it comes to matters of the structural integrity of a house and how building works may affect that.
In this case I predict a rise in unregistered gas fitters (including a large proportion of "cowboys") and a further reluctance to inform Building Control departments about anything.
In the second case, it would be funny if someone hadn't wasted money on it. Just read this for the low down: http://www.salon.com/2012/03/14/bring_back_the_40_hour_work_week/
Therefore, I rename "Think Tank" to "Oxygen Thieves" and "MP"
Sigh...
Monday, 9 April 2012
Wednesday, 4 April 2012
Stalinist UK?
Wiretapping without warrant: Expansion of GCHQ internet monitoring proposed
Wow.
Now let's be clear - GCHQ, the UK communications monitoring (aka spying) agency will not allegedly be able to read your email or listen to your phone calls without a warrant.
But they will be able to see who you called and who called you at what time and who emailed you and who you have emailed and which websites you accessed.
That's a lot of information to be able pick up on a whim. Are you comfortable with the idea that some anonymous spook can correlate all the people you communicate with and where you browse on the Internet?
This is akin to steaming open your post and recording the sender's details (but promising to not read the actual contents of your letter) and having to check out a book in the library even for browsing whilst presenting the librarian with ID so your name and book title can be noted in a register.
Your IP is anonymous? Is it? Well, this plan will be enacted with your ISP complicit, so at any given time your IP will be able to be correlated with your account. About the only way to be truly anonymous will be to use Tor or to buy a PAYG data SIM for your phone for cash and hope your face didn't get on any CCTV recordings with face recognition technology.
I suppose you feel rest assured that if you've done nothing wrong, you've nothing to hide? So what happens when you are a lecturer who emails a student whose uncle who is on a watchlist of "people who may be sympthetic with terrorism", even though the student knows nothing of his uncle's affiliations? Or your son is best friends with the kid from down the road and is phoning the house owned by a man who is a member of No2ID or runs a website like this one. Happy now?
Or you are part of a perfectly legal group that is critical of government policy. I don't know, perhaps Liberty, or an anti nuclear group or a group that believes the country should become a republic or a group that believes the current form of democracy is ineffective or a group that investigates politicians with corrupt dealings with big business?
Perhaps you don't care, because you think all such groups are whackpots? Fine. When you are used to this then what about when the government moves the goalposts a little. Then a little more. Until they are watching you because your child has joined a student protest in London. Or because you believe personal use of cannabis should be legalised. Or you are a member of an opposition party trying to displace your local MP in the next election and the MP happens to be the Home Secretary?
Don't think it could happen here? Well, if you are old enough, or know someone who was alive in the 1970's or 1960's - compare society then with now. Make a checklist of information government agencies could trivially hold on you - then and now. You may be surprised.
And now, advances in computing technology make it easy to correlate. Gone are the days of the cold war when, in order to build up a detailed dossier on someone, you would need a Michael Caine character personally on your tail backed by a small team - which made the possibility of mass surveillance fundamentally self limiting.
Still don't care? Well, don't complain if one day, you or someone you care about ends up in a secret court.
And for those of you who are up on the news enough to see this piece: this is not the first time such a policy has been pushed. How many more times will it happen? Even on this occasion, despite the media outrage, will some of the measures still be enacted? Then a few more. Then a few more?
Wow.
Now let's be clear - GCHQ, the UK communications monitoring (aka spying) agency will not allegedly be able to read your email or listen to your phone calls without a warrant.
But they will be able to see who you called and who called you at what time and who emailed you and who you have emailed and which websites you accessed.
That's a lot of information to be able pick up on a whim. Are you comfortable with the idea that some anonymous spook can correlate all the people you communicate with and where you browse on the Internet?
This is akin to steaming open your post and recording the sender's details (but promising to not read the actual contents of your letter) and having to check out a book in the library even for browsing whilst presenting the librarian with ID so your name and book title can be noted in a register.
Your IP is anonymous? Is it? Well, this plan will be enacted with your ISP complicit, so at any given time your IP will be able to be correlated with your account. About the only way to be truly anonymous will be to use Tor or to buy a PAYG data SIM for your phone for cash and hope your face didn't get on any CCTV recordings with face recognition technology.
I suppose you feel rest assured that if you've done nothing wrong, you've nothing to hide? So what happens when you are a lecturer who emails a student whose uncle who is on a watchlist of "people who may be sympthetic with terrorism", even though the student knows nothing of his uncle's affiliations? Or your son is best friends with the kid from down the road and is phoning the house owned by a man who is a member of No2ID or runs a website like this one. Happy now?
Or you are part of a perfectly legal group that is critical of government policy. I don't know, perhaps Liberty, or an anti nuclear group or a group that believes the country should become a republic or a group that believes the current form of democracy is ineffective or a group that investigates politicians with corrupt dealings with big business?
Perhaps you don't care, because you think all such groups are whackpots? Fine. When you are used to this then what about when the government moves the goalposts a little. Then a little more. Until they are watching you because your child has joined a student protest in London. Or because you believe personal use of cannabis should be legalised. Or you are a member of an opposition party trying to displace your local MP in the next election and the MP happens to be the Home Secretary?
Don't think it could happen here? Well, if you are old enough, or know someone who was alive in the 1970's or 1960's - compare society then with now. Make a checklist of information government agencies could trivially hold on you - then and now. You may be surprised.
And now, advances in computing technology make it easy to correlate. Gone are the days of the cold war when, in order to build up a detailed dossier on someone, you would need a Michael Caine character personally on your tail backed by a small team - which made the possibility of mass surveillance fundamentally self limiting.
Still don't care? Well, don't complain if one day, you or someone you care about ends up in a secret court.
And for those of you who are up on the news enough to see this piece: this is not the first time such a policy has been pushed. How many more times will it happen? Even on this occasion, despite the media outrage, will some of the measures still be enacted? Then a few more. Then a few more?
Orwell may have been a little off on his timing and symbolism, but the day when everything you do outside your home and a fair bit of what you do inside is monitored is most certainly coming and a good deal of it is already here.
Unless you, and I mean you, take a stand. I'm not suggesting you wear a funny beret and join the Tooting Popular Front. All it takes is a letter to your MP, a blog post giving your opinions, talking to friends - anything to keep awareness of the issue alive. Such policies are not successful when everyone is watching out for them - they are successfully enacted little by little when you are distracted, bored or don't care about anything except who gets evicted from Big Brother.
Unless you, and I mean you, take a stand. I'm not suggesting you wear a funny beret and join the Tooting Popular Front. All it takes is a letter to your MP, a blog post giving your opinions, talking to friends - anything to keep awareness of the issue alive. Such policies are not successful when everyone is watching out for them - they are successfully enacted little by little when you are distracted, bored or don't care about anything except who gets evicted from Big Brother.
Friday, 6 January 2012
New OpenBox Theme for Lubuntu/LXDE/OpenBox
Well, finally - I got around to making a window decoration theme for OpenBox.
Big Bright Green
It's simple, very bold and extremely clear (well, to me anyway). The "Green" is a matter of personal preference as I wanted the window titlebars to stand out from the rest of the junk on the screen, but you could easily edit those (either edit the themerc file or use "obtheme").
Here are the screenshots:
Big Bright Green
It's simple, very bold and extremely clear (well, to me anyway). The "Green" is a matter of personal preference as I wanted the window titlebars to stand out from the rest of the junk on the screen, but you could easily edit those (either edit the themerc file or use "obtheme").
Here are the screenshots:
Friday, 2 December 2011
Huawei E585 Review
I've been using a Huawei E585 for several months now, mostly on the train from Robertsbridge to London. This is a cellular 3G to WIFI adaptor (often called a MIFI device). It sits on its own cellular connection and acts as a WIFI base station, meaning it can be used with any WIFI capabale device, eg laptop, phone, internet radio.
Not bad when it works - with a "3" SIM, I can pull 4.3Mbit/s downstream and over 1Mbit/s upstream at good positions.
The problem comes with all the tunnels on the Hastings line - there are two a mile long either side of Sevenoaks and over half a dozen others. Obviously, I don't expect the device to work when there is no signal.
However, I do expect it to re-acquire cleanly when the signal comes back.
The E585 doesn't. Not reliably anyway. On the odd day, I can travel from London Charing Cross to Roberstbridge without a problem. On most days though, the device will lose signal in a tunnel, then get the signal back at the other end - indicated by lots of bars on the display. What it then does is lose its WAN IP - ie loses the IP layer. At this point, it should be making active efforts to get it back. It doesn't. It sits there like a lemon until manually rebooted, whereupon it will come straight back with a strong signal and connection.
This wouldn't be so bad if:
1) I could programmatically reboot the device. So far, all attempts to remote script the web interface have failed due to the particularly weird session handling they employ.
2) They actually issued some firmware updates occasionally;
3) They actually had technical support. "3" don't count, like most cell phone companies, you'll get a bloke in India who asks for a load of irrelevant details then tells you Linux isn't supported - despite the fact it's a WIFI device!
This is typical of so much of the consumer electronics industry where the motto is "first to market, then make the next one".
Not bad when it works - with a "3" SIM, I can pull 4.3Mbit/s downstream and over 1Mbit/s upstream at good positions.
The problem comes with all the tunnels on the Hastings line - there are two a mile long either side of Sevenoaks and over half a dozen others. Obviously, I don't expect the device to work when there is no signal.
However, I do expect it to re-acquire cleanly when the signal comes back.
The E585 doesn't. Not reliably anyway. On the odd day, I can travel from London Charing Cross to Roberstbridge without a problem. On most days though, the device will lose signal in a tunnel, then get the signal back at the other end - indicated by lots of bars on the display. What it then does is lose its WAN IP - ie loses the IP layer. At this point, it should be making active efforts to get it back. It doesn't. It sits there like a lemon until manually rebooted, whereupon it will come straight back with a strong signal and connection.
This wouldn't be so bad if:
1) I could programmatically reboot the device. So far, all attempts to remote script the web interface have failed due to the particularly weird session handling they employ.
2) They actually issued some firmware updates occasionally;
3) They actually had technical support. "3" don't count, like most cell phone companies, you'll get a bloke in India who asks for a load of irrelevant details then tells you Linux isn't supported - despite the fact it's a WIFI device!
This is typical of so much of the consumer electronics industry where the motto is "first to market, then make the next one".
Thursday, 13 October 2011
Dell EqualLogic PS6500E 1TB SATA RAID capacities
RAID layouts and capacities.
Disk capacity is 931.51GiB and there are 48 such drives
RAID 5
37.83TiB and there are 2 hotspares. Overhead = 90GB per disk.
RAID 6
35.12TiB and 1 hotspare
RAID 10
20.71TiB with 2 hotspares
RAID 50
35.12TiB with 1 hotspares
Disk capacity is 931.51GiB and there are 48 such drives
RAID 5
37.83TiB and there are 2 hotspares. Overhead = 90GB per disk.
RAID 6
35.12TiB and 1 hotspare
RAID 10
20.71TiB with 2 hotspares
RAID 50
35.12TiB with 1 hotspares
Wednesday, 12 October 2011
EqualLogic PS6500E speed test with linux hosts
Equipment
2 linux hosts, each comprising:
echo 4194304 > /proc/sys/net/core/rmem_default
echo 4194304 > /proc/sys/net/core/rmem_max
echo 4194304 > /proc/sys/net/core/wmem_default
echo 4194304 > /proc/sys/net/core/wmem_max
echo 0 > /proc/sys/net/ipv4/tcp_timestamps
echo 1 > /proc/sys/net/ipv4/tcp_sack
echo 1 > /proc/sys/net/ipv4/tcp_window_scaling
echo "4194304 4194304 4194304" > /proc/sys/net/ipv4/tcp_rmem
echo "4194304 4194304 4194304" > /proc/sys/net/ipv4/tcp_wmem
ifconfig eth8 txqueuelen 1000
ifconfig eth9 txqueuelen 1000
ifconfig eth10 txqueuelen 1000
ifconfig eth11 txqueuelen 1000
PS6500E has 2 x 10TB volumes ona RAID50, each volume is mounted to one of the hosts using round robin multipathing, formatted to XFS and mounted:
iscsiadm -m node --logoutall=all
iscsiadm -m discovery -t st -p 10.223.10.5
iscsiadm -m node --login -T 'iqn.2001-05.com.equallogic:0-8a0906-388f3f80a-e907d15ce794e948-test3'
multipath
mkfs.xfs -L TEST3 /dev/dm-0
mount -onoatime,logbsize=262144,logbufs=8 /dev/dm-0 /mnt/
We used streamput and streamget which are simple homegrown C programs to read and write single large buffers of random data to files (few large files), run for a reasonable time to ensure the host RAM is saturated vis a vis caches (streamput can also do O_DIRECT writes to bypass caching). Buffer for single IO operation is 1MB of random data, generated once at program start. Buffer is page aligned for optimal DMA and O_DIRECT.
Our basic streamput test comprised:
./streamput -v -t 1200 -l 4096 -w /mnt/<uniquesubdir>
(Write 4GB files with random name into test dir and repeat for a total of 1200 seconds using plain ordinary C file writes with no special flags)
Our basic streamread test comprised
./streamread -v -t 1200 -w /mnt/<uniquesubdir>
(Read 4GB files with random name from test dir and repeat for a total of 1200 seconds using plain ordinary C file writes with no special flags)
Test 1 simple streamed writing
2 x streamput tests on 2 hosts in parallel
Results
streamput,PID,14225,Megabytes/sec,45.8
streamput,PID,14227,Megabytes/sec,45.0
streamput,PID,13640,Megabytes/sec,44.1
streamput,PID,13639,Megabytes/sec,44.1
Total 179MB/sec
Test 2 reading
2 x streamread tests on 2 hosts in parallel
raid5: (groupid=0, jobs=64): err= 0: pid=21295
read : io=40636KB, bw=654080B/s, iops=159, runt= 63618msec
clat (usec): min=527, max=6767K, avg=374501.76, stdev=152589.35
bw (KB/s) : min= 0, max= 318, per=2.21%, avg=14.08, stdev= 1.64
cpu : usr=0.09%, sys=0.01%, ctx=10210, majf=0, minf=1651
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w: total=10159/0, short=0/0
lat (usec): 750=0.11%, 1000=0.15%
lat (msec): 2=0.97%, 4=9.81%, 10=55.35%, 20=17.33%, 50=1.62%
lat (msec): 100=0.83%, 250=2.15%, 500=1.53%, 750=0.88%, 1000=0.64%
lat (msec): 2000=1.39%, >=2000=7.24%
Run status group 0 (all jobs):
READ: io=40636KB, aggrb=638KB/s, minb=654KB/s, maxb=654KB/s, mint=63618msec, maxt=63618msec
Disk stats (read/write):
dm-0: ios=10138/0, merge=0/0, ticks=3905828/0, in_queue=3905812, util=99.62%, aggrios=0/0, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
IOPS=3068
IOPS=8(Seems wrong but repeatible!)
2 linux hosts, each comprising:
- Dell R610, 96GB RAM, 2x Intel Xeon Processor X5680 (12M Cache, 3.33 GHz, 6.40 GT/s) (6 cores, 12 hyperthreads), 2 x 4 port Intel Pro1000 NICs (Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01))
- 2 x Dell PC6224 stacked gigabit switches
- 1 x EQL PS6500E with 48 x 1TB SATA 7200RPM drives ST31000524NS SATA 3Gb/s 1TB 32MB cache
- Wiring: 4 x gig from each host to PC6224s split across switches; 8 x gig from both PS6500E controllers to swiches, split across switches.
- Switches and hosts confirgured for MTU of 9000 to match PS6500
echo 4194304 > /proc/sys/net/core/rmem_default
echo 4194304 > /proc/sys/net/core/rmem_max
echo 4194304 > /proc/sys/net/core/wmem_default
echo 4194304 > /proc/sys/net/core/wmem_max
echo 0 > /proc/sys/net/ipv4/tcp_timestamps
echo 1 > /proc/sys/net/ipv4/tcp_sack
echo 1 > /proc/sys/net/ipv4/tcp_window_scaling
echo "4194304 4194304 4194304" > /proc/sys/net/ipv4/tcp_rmem
echo "4194304 4194304 4194304" > /proc/sys/net/ipv4/tcp_wmem
ifconfig eth8 txqueuelen 1000
ifconfig eth9 txqueuelen 1000
ifconfig eth10 txqueuelen 1000
ifconfig eth11 txqueuelen 1000
PS6500E has 2 x 10TB volumes ona RAID50, each volume is mounted to one of the hosts using round robin multipathing, formatted to XFS and mounted:
iscsiadm -m node --logoutall=all
iscsiadm -m discovery -t st -p 10.223.10.5
iscsiadm -m node --login -T 'iqn.2001-05.com.equallogic:0-8a0906-388f3f80a-e907d15ce794e948-test3'
multipath
mkfs.xfs -L TEST3 /dev/dm-0
mount -onoatime,logbsize=262144,logbufs=8 /dev/dm-0 /mnt/
We used streamput and streamget which are simple homegrown C programs to read and write single large buffers of random data to files (few large files), run for a reasonable time to ensure the host RAM is saturated vis a vis caches (streamput can also do O_DIRECT writes to bypass caching). Buffer for single IO operation is 1MB of random data, generated once at program start. Buffer is page aligned for optimal DMA and O_DIRECT.
Our basic streamput test comprised:
./streamput -v -t 1200 -l 4096 -w /mnt/<uniquesubdir>
(Write 4GB files with random name into test dir and repeat for a total of 1200 seconds using plain ordinary C file writes with no special flags)
Our basic streamread test comprised
./streamread -v -t 1200 -w /mnt/<uniquesubdir>
(Read 4GB files with random name from test dir and repeat for a total of 1200 seconds using plain ordinary C file writes with no special flags)
Test 1 simple streamed writing
2 x streamput tests on 2 hosts in parallel
Results
streamput,PID,14225,Megabytes/sec,45.8
streamput,PID,14227,Megabytes/sec,45.0
streamput,PID,13640,Megabytes/sec,44.1
streamput,PID,13639,Megabytes/sec,44.1
Total 179MB/sec
Test 2 reading
2 x streamread tests on 2 hosts in parallel
Results
streamread,PID,14364,Megabytes/sec,54.5
streamread,PID,14363,Megabytes/sec,50.9
streamread,PID,13767,Megabytes/sec,54.6
streamread,PID,13766,Megabytes/sec,64.5
Total 224.5MB/sec
Reconfigure to RAID10, setup with XFS s above.
Test 3
2 x streamput tests on 2 hosts in parallel
Results
streamput,PID,15107,Megabytes/sec,46.1
streamput,PID,15106,Megabytes/sec,48.6
streamput,PID,14471,Megabytes/sec,44.2
streamput,PID,14470,Megabytes/sec,50.8
Total 189.7MB/sec
Test 4
echo 3 > /proc/sys/vm/drop_caches
2 x streamread tests on 2 hosts in parallel
Results
streamread,PID,15197,Megabytes/sec,19.9
streamread,PID,15198,Megabytes/sec,20.0
streamread,PID,14558,Megabytes/sec,23.4
streamread,PID,14557,Megabytes/sec,23.1
Total 86.4 MB/sec
Reconfigure to RAID10, setup with XFS s above.
Test 5
2 x streamread tests on 2 hosts in parallel
Results
streamput,PID,15845,Megabytes/sec,40.8
streamput,PID,15843,Megabytes/sec,49.7
streamput,PID,15063,Megabytes/sec,43.6
streamput,PID,15062,Megabytes/sec,45.9
Total 180MB/sec
Test 6
echo 3 > /proc/sys/vm/drop_caches
2 x streamread tests on 2 hosts in parallel
Reconfigure to RAID10, setup with XFS s above.
Test 3
2 x streamput tests on 2 hosts in parallel
Results
streamput,PID,15107,Megabytes/sec,46.1
streamput,PID,15106,Megabytes/sec,48.6
streamput,PID,14471,Megabytes/sec,44.2
streamput,PID,14470,Megabytes/sec,50.8
Total 189.7MB/sec
Test 4
echo 3 > /proc/sys/vm/drop_caches
2 x streamread tests on 2 hosts in parallel
Results
streamread,PID,15197,Megabytes/sec,19.9
streamread,PID,15198,Megabytes/sec,20.0
streamread,PID,14558,Megabytes/sec,23.4
streamread,PID,14557,Megabytes/sec,23.1
Total 86.4 MB/sec
Reconfigure to RAID10, setup with XFS s above.
Test 5
2 x streamread tests on 2 hosts in parallel
Results
streamput,PID,15845,Megabytes/sec,40.8
streamput,PID,15843,Megabytes/sec,49.7
streamput,PID,15063,Megabytes/sec,43.6
streamput,PID,15062,Megabytes/sec,45.9
Total 180MB/sec
Test 6
echo 3 > /proc/sys/vm/drop_caches
2 x streamread tests on 2 hosts in parallel
streamread,PID,15088,Megabytes/sec,72.0
streamread,PID,15087,Megabytes/sec,45.6
streamread,PID,15878,Megabytes/sec,49.4
streamread,PID,15877,Megabytes/sec,35.9
Total 202.9MB/sec
Reconfigure to RAID10, setup with XFS s above.
Test 7 during RAID5 reconstruction
echo 3 > /proc/sys/vm/drop_caches
2 x streamread tests on 2 hosts in parallel
Results
streamput,PID,15379,Megabytes/sec,43.4
streamput,PID,15378,Megabytes/sec,48.8
streamput,PID,16235,Megabytes/sec,44.7
streamput,PID,16234,Megabytes/sec,47.9
Total 184.8MB/sec
Test 8 Array reconstructing
echo 3 > /proc/sys/vm/drop_caches
2 x streamread tests on 2 hosts in parallel
Results
streamread,PID,15398,Megabytes/sec,45.6
streamread,PID,15397,Megabytes/sec,44.4
streamread,PID,16253,Megabytes/sec,36.5
streamread,PID,16252,Megabytes/sec,36.8
Total 163.3MB/sec
Array (re)construction time:
Time to initialise RAID5: 58% completion in approx 14 hours. At 13:05, 62%, 15:29 73%. Estimated full time 21 hours 49 minutes
fio IOPS tests
Test 9 RAID5 (Raid rebuild complete)
fio --filename=/dev/dm-0 --direct=1 --rw=randwrite --bs=4k --numjobs=64 --runtime=300 --group_reporting --name=raid5
on 1 host
IOPS=2862
raid5: (groupid=0, jobs=64): err= 0: pid=21151
write: io=3407MB, bw=11452KB/s, iops=2862, runt=304630msec
clat (usec): min=220, max=7629K, avg=21083.39, stdev=47412.87
bw (KB/s) : min= 0, max= 5261, per=2.36%, avg=269.97, stdev=69.20
cpu : usr=0.04%, sys=0.10%, ctx=874983, majf=0, minf=1523
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w: total=0/872151, short=0/0
lat (usec): 250=2.28%, 500=90.23%, 750=3.62%, 1000=0.47%
lat (msec): 2=2.27%, 4=0.53%, 10=0.09%, 20=0.06%, 50=0.05%
lat (msec): 100=0.02%, 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.33%
Run status group 0 (all jobs):
WRITE: io=3407MB, aggrb=11451KB/s, minb=11726KB/s, maxb=11726KB/s, mint=304630msec, maxt=304630msec
Disk stats (read/write):
dm-0: ios=0/871635, merge=0/0, ticks=0/19261868, in_queue=19261440, util=99.97%, aggrios=0/0, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
Test 10 RAID5 write (Raid rebuild complete) shorter test time
fio --filename=/dev/dm-0 --direct=1 --rw=randwrite --bs=4k --numjobs=64 --runtime=60 --group_reporting --name=raid5
IOPS=3261
raid5: (groupid=0, jobs=64): err= 0: pid=21222
write: io=839936KB, bw=13048KB/s, iops=3261, runt= 64375msec
clat (usec): min=222, max=7480K, avg=18589.17, stdev=43118.68
bw (KB/s) : min= 0, max= 7520, per=1.55%, avg=202.70, stdev=13.74
cpu : usr=0.11%, sys=0.10%, ctx=211492, majf=0, minf=1523
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w: total=0/209984, short=0/0
lat (usec): 250=11.03%, 500=83.24%, 750=2.35%, 1000=0.52%
lat (msec): 2=1.74%, 4=0.54%, 10=0.09%, 20=0.06%, 50=0.05%
lat (msec): 100=0.02%, 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.03%, >=2000=0.29%
Run status group 0 (all jobs):
WRITE: io=839936KB, aggrb=13047KB/s, minb=13360KB/s, maxb=13360KB/s, mint=64375msec, maxt=64375msec
Disk stats (read/write):
dm-0: ios=0/209453, merge=0/0, ticks=0/3930736, in_queue=3930732, util=99.61%, aggrios=0/0, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
Test 10 RAID5 read
fio --filename=/dev/dm-0 --direct=1 --rw=randread --bs=4k --numjobs=64 --runtime=60 --group_reporting --name=raid5
IOPS=159
Reconfigure to RAID10, setup with XFS s above.
Test 7 during RAID5 reconstruction
echo 3 > /proc/sys/vm/drop_caches
2 x streamread tests on 2 hosts in parallel
Results
streamput,PID,15379,Megabytes/sec,43.4
streamput,PID,15378,Megabytes/sec,48.8
streamput,PID,16235,Megabytes/sec,44.7
streamput,PID,16234,Megabytes/sec,47.9
Total 184.8MB/sec
Test 8 Array reconstructing
echo 3 > /proc/sys/vm/drop_caches
2 x streamread tests on 2 hosts in parallel
Results
streamread,PID,15398,Megabytes/sec,45.6
streamread,PID,15397,Megabytes/sec,44.4
streamread,PID,16253,Megabytes/sec,36.5
streamread,PID,16252,Megabytes/sec,36.8
Total 163.3MB/sec
Array (re)construction time:
Time to initialise RAID5: 58% completion in approx 14 hours. At 13:05, 62%, 15:29 73%. Estimated full time 21 hours 49 minutes
fio IOPS tests
Test 9 RAID5 (Raid rebuild complete)
fio --filename=/dev/dm-0 --direct=1 --rw=randwrite --bs=4k --numjobs=64 --runtime=300 --group_reporting --name=raid5
on 1 host
IOPS=2862
raid5: (groupid=0, jobs=64): err= 0: pid=21151
write: io=3407MB, bw=11452KB/s, iops=2862, runt=304630msec
clat (usec): min=220, max=7629K, avg=21083.39, stdev=47412.87
bw (KB/s) : min= 0, max= 5261, per=2.36%, avg=269.97, stdev=69.20
cpu : usr=0.04%, sys=0.10%, ctx=874983, majf=0, minf=1523
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w: total=0/872151, short=0/0
lat (usec): 250=2.28%, 500=90.23%, 750=3.62%, 1000=0.47%
lat (msec): 2=2.27%, 4=0.53%, 10=0.09%, 20=0.06%, 50=0.05%
lat (msec): 100=0.02%, 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.01%, >=2000=0.33%
Run status group 0 (all jobs):
WRITE: io=3407MB, aggrb=11451KB/s, minb=11726KB/s, maxb=11726KB/s, mint=304630msec, maxt=304630msec
Disk stats (read/write):
dm-0: ios=0/871635, merge=0/0, ticks=0/19261868, in_queue=19261440, util=99.97%, aggrios=0/0, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
Test 10 RAID5 write (Raid rebuild complete) shorter test time
fio --filename=/dev/dm-0 --direct=1 --rw=randwrite --bs=4k --numjobs=64 --runtime=60 --group_reporting --name=raid5
IOPS=3261
raid5: (groupid=0, jobs=64): err= 0: pid=21222
write: io=839936KB, bw=13048KB/s, iops=3261, runt= 64375msec
clat (usec): min=222, max=7480K, avg=18589.17, stdev=43118.68
bw (KB/s) : min= 0, max= 7520, per=1.55%, avg=202.70, stdev=13.74
cpu : usr=0.11%, sys=0.10%, ctx=211492, majf=0, minf=1523
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w: total=0/209984, short=0/0
lat (usec): 250=11.03%, 500=83.24%, 750=2.35%, 1000=0.52%
lat (msec): 2=1.74%, 4=0.54%, 10=0.09%, 20=0.06%, 50=0.05%
lat (msec): 100=0.02%, 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec): 2000=0.03%, >=2000=0.29%
Run status group 0 (all jobs):
WRITE: io=839936KB, aggrb=13047KB/s, minb=13360KB/s, maxb=13360KB/s, mint=64375msec, maxt=64375msec
Disk stats (read/write):
dm-0: ios=0/209453, merge=0/0, ticks=0/3930736, in_queue=3930732, util=99.61%, aggrios=0/0, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
Test 10 RAID5 read
fio --filename=/dev/dm-0 --direct=1 --rw=randread --bs=4k --numjobs=64 --runtime=60 --group_reporting --name=raid5
IOPS=159
raid5: (groupid=0, jobs=64): err= 0: pid=21295
read : io=40636KB, bw=654080B/s, iops=159, runt= 63618msec
clat (usec): min=527, max=6767K, avg=374501.76, stdev=152589.35
bw (KB/s) : min= 0, max= 318, per=2.21%, avg=14.08, stdev= 1.64
cpu : usr=0.09%, sys=0.01%, ctx=10210, majf=0, minf=1651
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued r/w: total=10159/0, short=0/0
lat (usec): 750=0.11%, 1000=0.15%
lat (msec): 2=0.97%, 4=9.81%, 10=55.35%, 20=17.33%, 50=1.62%
lat (msec): 100=0.83%, 250=2.15%, 500=1.53%, 750=0.88%, 1000=0.64%
lat (msec): 2000=1.39%, >=2000=7.24%
Run status group 0 (all jobs):
READ: io=40636KB, aggrb=638KB/s, minb=654KB/s, maxb=654KB/s, mint=63618msec, maxt=63618msec
Disk stats (read/write):
dm-0: ios=10138/0, merge=0/0, ticks=3905828/0, in_queue=3905812, util=99.62%, aggrios=0/0, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
Test 10 RAID5 read on 2 hosts in parallel
IOPS=138 and 123, total 261
fio --filename=/dev/dm-0 --direct=1 --rw=randread --bs=4k --numjobs=64 --runtime=60 --group_reporting --name=raid5
Test 11 RAID 10 Write (Reconstructing) 1 Host
fio --filename=/dev/dm-0 --direct=1 --rw=randwrite --bs=4k --numjobs=64 --runtime=60 --group_reporting --name=raid10 Test 12 RAID 10 Read (Reconstructing) 1 Host
fio --filename=/dev/dm-0 --direct=1 --rw=randread --bs=4k --numjobs=64 --runtime=60 --group_reporting --name=raid10Tuesday, 23 August 2011
debirf - Building custom debian USB keys/CDROMs and run in RAM
I needed a way to make a simple custom bootable USB key to run linux on some test servers.
Furthermore I needed a strong linux environment that was easy to install additional tools into - such as an iSCSI initiator, bonnie++ and other tools to benchmark a new SAN.
I was also looking for a simple way to run the system from RAM (tmpfs) so I was not dependent on leaving the media in.
debirf is just such a tool. It builds an ISO image of a debian 6 system which boots into RAM and behaves like a normal system (ie you can trivially apt-get extra packages into the running system).
Turns out it's not too hard to tell it to add more packages to the ISO. Of course, when you have an ISOLINUX bootable ISO, it's not too hard to transfer to a USB key, though there is no automated way to do this yet.
It exists as a Debian package and also in Ubuntu, though I found problems running it from Ubuntu 10.10, so I installed a minimal Debian 6 system in a VirtualBox virtual machine.
The rough steps are, as root on the Debian 6 machine:
Furthermore I needed a strong linux environment that was easy to install additional tools into - such as an iSCSI initiator, bonnie++ and other tools to benchmark a new SAN.
I was also looking for a simple way to run the system from RAM (tmpfs) so I was not dependent on leaving the media in.
debirf is just such a tool. It builds an ISO image of a debian 6 system which boots into RAM and behaves like a normal system (ie you can trivially apt-get extra packages into the running system).
Turns out it's not too hard to tell it to add more packages to the ISO. Of course, when you have an ISOLINUX bootable ISO, it's not too hard to transfer to a USB key, though there is no automated way to do this yet.
It exists as a Debian package and also in Ubuntu, though I found problems running it from Ubuntu 10.10, so I installed a minimal Debian 6 system in a VirtualBox virtual machine.
The rough steps are, as root on the Debian 6 machine:
- apt-get install debirf syslinux
- mkdir /root/debirf; cd /root/debirf # You need lots of space here, use a suitable location
- tar -xzf /usr/share/doc/debirf/example-profiles/rescue.tgz
- Now edit rescue/debirf.conf and add the following line:
export DEBIRF_ISO_BOOTLOADER=isolinux
That just makes life slightly easier as you will get a usable isolinux config for using syslinux on a USB key.
Now some more steps:
- as an example of how to add more packages to the image, add a file rescue/modules/benchmark:
#!/bin/bash -e
# debirf module: benchmark
# remove/install extra packages
#
# install packages
debirf_exec apt-get --no-install-recommends --assume-yes install bonnie++ iftop fio iperf
- debirf make rescue
- debirf makeiso rescue
Hopefully now, you will find an ISO file in rescue/
Burn this to a CDWR, or prep a USB key in the usual way:
- fdisk /dev/sdX # Add one primary partition, type 0xc, boot flag set
- syslinux -i /dev/sdX1
- mount /dev/sdX /mnt/usb
- mount -oloop rescue/<isofilefromdebirf>.iso /mnt/iso
- cp /mnt/iso/* /mnt/usb/
- mv /mnt/usb/isolinux.cfg /mnt/usb/syslinux.cfg
- umount /mnt/iso /mnt/usb
Insert USB key (or CDWR) into computer, boot from it and you should see a syslinux menu offering console over "video" or "serial" (ttyS0 at 115000 baud).
Remove key and use again if required.
UPDATE
You can also just run the "isohybrid" command on the original iso, the dd the result to a USB key:
UPDATE
You can also just run the "isohybrid" command on the original iso, the dd the result to a USB key:
- isohybrid rescue/<isofilefromdebirf>.iso
- dd if=rescue/<isofilefromdebirf>.iso of=/dev/sdX
Where /dev/sdX is the device node of your USB stick. Make sure you umount the sick first as many OSes will auto mount it upon insertion!
Subscribe to:
Posts (Atom)