AIs will wipe the floor with human programmers

Sometimes when I'm writing particularly inefficient code, intentionally, because I know it will be easier for me to maintain or the inefficiency doesn't seem like it will matter, I wonder what it would be like if there were AI's around today which were 10 or 100 or 1000 times better at programming than humans. Say, instead of being able to quickly make sense of a function with 7 variables like the average human, they could quickly make sense of a function with 7000 variables. What could an intelligence like that do with today's computers?

It's obscene how little performance we get out of our computing systems these days. Python code is almost always awful, awful, awful in terms of runtime efficiency, and yet it's one of the best tools out there for solving big problems with small investments of human time. As our hardware continues to get faster we'll end up wasting more and more of it, smartly avoiding optimizations that aren't worth the effort. But the inefficiencies will always remain, just waiting for some super-intelligent optimizing process to come along and eradicate them.

It's hard to even begin to imagine the enormity of amazing things that could be built with today's existing hardware if only we were better at managing the complexity.

The Future Is Predictably Weird

Reading old science fiction is entertainment of the grandest sort, the kind that lets you feel smugly superior. "Hahaha, author," you get to jeer, "if only you'd known you'd be wrong in every way."

Space travel, flying cars, effective world governments... all par for the course by 2000, if some sci-fi were to be believed.

Looking back, one clear trend in the last few decades is that technology's been completely driven by the market economy. Sure, space travel might have taken off if governments had kept funding it, but there's just not much economic incentive to leave the planet right now. Given the US government's looming budgetary crises and eagerness to pander to voters, I think it's safe to say this trend will continue.

Another trend is that while completely unprecedented things do show up from time to time (eg, the Internet), it's much more common to see incremental improvements on existing technologies. So we don't have flying cars, but we do have much more efficient, stylish, and refined road cars using a variety of propulsion techniques.

If we extrapolate along these lines, what could we predict about 2025? Hmm...

We'll have UHDTV that can fill an entire wall with high-res video for reasonable prices, but no mass-market contact lenses that can overlay video onto your field of view.

The US will finally have broken free of its dependence on fossil fuels, but most of us will still be driving relatively inefficient cars on old-fashioned road networks (instead of mass transit).

We'll have multi-petabyte hard disks (or similar capacity from some competing technology), enough to store every word ever written thousands of times over, but people will still lose data because they haven't made any backups.

Cell phones will be ridiculously fast, small, easy-to-use, and cheap, but mega-corporations will still control the spectrum and there will be no cheap/free, open, high-bandwidth mesh network.

Video game graphics will be ridiculously realistic and AAA titles will be produced by studios 3 times as big as today's biggest, but we still won't see much procedural content generation and advances in AI will be pathetic.

Most homes will have multi-terabit Internet connections that can download 12 DVD's per second with ease, but they'll still go down entirely every time some moron with a backhoe doesn't call Miss Utility.

I hope I'm wrong about most of these.

Software RAID 5 in Windows 7

A bit of Googling around reveals that MS has removed the old software RAID 5 feature from Windows 7, although it's still present in Server 2008. My experience with the stuff in Windows 2003 was that it was very reliable, but for some reason much slower than mdadm's implementation on identical hardware. I know there is some complexity around RAID 5 with write holes and potential data corruption, so I don't know if mdadm's implementation is better or just more dangerous. I certainly don't have battery backup for writing the data from cache to the disk in the event of a total power failure, and I hope mdadm doesn't think I do.

But if you are really desperate for software RAID 5 (or you happen to have an old mdadm array lying around, like me), there is one very free, very hacky solution that will let you run software RAID 5 in Windows 7: virtualize linux.

That's right: you can download the free VirtualBox, install linux, map your SATA drives, and then run mdadm just as if linux were running natively. In fact, I created the array with the machine booted into to linux, then switched to Windows and mounted the array with a virtual machine. Samba is probably the best tool for interacting with the ext3 fs from Windows.

Scrub throughput is only about 22MB/s with this setup on my 4x500GB array, but then speed probably wasn't your top concern if you really wanted software RAID 5.

For some reason the Virtualbox documentation is only available as PDF, so I can't link you to the section on mapping physical drives which unfortunately is not as easy as it is with the for-pay VMWare Workstation. Mdadm is pretty easy to use, so getting the physical drives mapped correctly is the hardest part of the process, and may involve painful things like forcing VirtualBox to run as administrator.

---

Update: Initially I planned for this to be a stop-gap solution while I looked for a different medium to migrate the array to, but I continue to be surprised by how well it works. With a network drive mapped to the Samba share I can even hibernate Windows while the VM is running and everything just picks up right where it left off on resume. Even media files that were playing from the array! The array still works fine from within a native linux boot as well.

---

Update 2: This post is ranking pretty highly in Google for "Raid 5 Windows 7" and a lot of you are undoubtedly looking for solutions. David wrote in to ask for clairification:

> So you installed VirtualBox (which I assume works like VMWare,
> roughly speaking), mounted all the SATA drives in Linux, used
> mdadm to create a RAID array. I understand that. But the next
> part you say you "switched to Windows and mounted the array
> with a virtual machine". What do you mean by this?

Yes, it is similar to VMWare Workstation.

To clarify: I originally created the mdadm array while running a linux operating system on the host. Later, I wanted to access the array while running Windows 7 as the host operating system.

The fact that I was using linux as the original host OS should be an incidental detail: I was merely trying to illustrate that the solution is not dependent on the host OS. You can switch between linux and Windows and still create, access, or destroy mdadm arrays based on physical devices from within either host OS.

So, you should be able to create an array just fine with Windows as the host OS once the drives are mapped correctly.

> Then you mention Samba -- did you share the drive via Samba in
> the Linux VM and then essentially use Windows' "Map Network
> Drive" to map the share?

Precisely.

> If there are any other specifics you think might be helpful,
> that would be fantastic as well.

VirtualBox doesn't offer a nice interface for setting up physical drives like VMWare does; you have to muck about with XML files and might need to run the VirtualBox process with elevated privileges. If you have a VMWare license I'd definitely try that first. But VirtualBox is free, so...

> Is there any way (VMWare or VirtualBox) to easily make the RAID
> array visible as an actual device in Windows?

Probably not unless Windows 7 has some native support for mounting devices over the network. I don't know of such features but they may exist. Samba is probably also not the ideal choice for a protocol in terms of performance, but it has the advantage that it's easy to setup and the mapped drive behaves almost like a regular drive in the windows explorer.