The weather is great today and we're spending a lot of time on the deck.

After we came back from a walk to the playground Nora had her lunch outside. We still have to get used to everything -- we all got startled when a squirrel ran along the fence of the garden. There are a lot of birds outside.

Last night we finished unpacking the kitchen, did the first laundry and cooked for the first time on the new stove.

We moved into our new house! It was a very exciting week.

The painters finished up the house on Tuesday evening. That night we were packing until 2.00 am and at 6.00 am the alarm went. I drove Nora to Falls Church where she was staying with Lela Snezha for the day. The movers arrived around 9.30 am, a huge van and 3 strong guys. We had packed everything well using the boxes from our previous moves and the loading went smoothly. It was only a 10 minute drive to the new house.

From 12:00 pm through 3.00 pm the movers placed everything in the house. Ever since our first move from Arlington to Durham I am a little wary about movers, but this team from ACE was pleasant and very careful. It's amazing how much space we have! We've lived in apartments for the past 15 years and it's incredible to see how much room we have now. The majority of our boxes are in our "mudroom" in the basement so nicely out of sight.

In the afternoon the electrician came and put our new light fixture in the dining area, and our contractor stopped by for an inspection as well. When the movers were gone we had lunch together and then went back to the apartment. We made three trips by car, for a few loose items and our plants.

We picked up Nora in the evening. She had a great day with Lela Snezha's two grandchildren and the rest of the family. Once home, I put together her crib and it was time for her to sleep. The next two days we brought her to Lela Snezha as well since the house was not quite ready yet, but now the living room is Nora-proof.

Moving the plants.

Moving in.

We spent the whole weekend packing and preparing our move, but on Sunday afternoon we went to Sophie's birthday party. She turned one year old and John and Juliana had organized a big party for her, with a puppeteer, music, a bubble machine and lots of balloons.

Nora was very impressed -- overwhelmed even -- and it took her a little bit to warm up to it but after 10 minutes or so she started enjoying things. Of course, balloons make everything great and she loved being around other kids.

The movers will be here on Wednesday morning, so we're packing and preparing this weekend. Things are nicely on track. We still had all the boxes from our previous move and by now we're pretty experienced.

In the evening we went to several stores, including Home Depot -- probably the 10th visit there in the past few weeks.

Tonight I pulled the coax cables from our garden into the basement, where the cable modem will be. There was already a hole in the basement wall which was used for 4 coax cables to the roof where an old satellite dish stands. We won't use that so I disconnected them and reused the opening for a new coax cable, connected to the Comcast cable outside.

I grounded the cable where it enters the house using a copper water pipe and then sealed the hole with caulk. It was the first time I've used caulk and it was fun.

Once everything is done there will be an ethernet connection from the cable modem in the basement to a wireless router in the living room, providing coverage for the whole house. I ordered the cable modem from Amazon so we don't have to pay a monthly rent for the modem. After the cable work was done the activation of the actual connection was easy. We signed up for 50Mbps/10Mbps, which is faster than what we have now.

Slowly but surely we're getting ready to move into our new house.

In the afternoon a technician from Comcast, the cable company, came by. The cable ends at our outside wall -- I need to bring it inside myself -- but he tested the connection and we should be ready for cable internet.

Tonight we visited the house together. Sasha and I were talking about the paint color, Balboa, and Nora liked the word and started repeating "Balboa, Balboa".

We started packing too. We can't move anything in the main rooms yet, but most of the things that will be in storage in the basement are already moved. This will save us quite a lot of time during moving day, which is going to be Monday or Tuesday.

Yesterday I attended the Cloudera Sessions, an event on using Hadoop, HBase and other "big data" tools organized by Cloudera.

Big Data is an interesting field and I enjoyed this well-organized day. Cloudera is a provider of commercial solutions around the open-source Hadoop stack. There were speakers from Cloudera and several of their commercial partners, talking about the practical experiences so far and plans for the future.

An event like this is meant to convince people to use Cloudera's stuff -- but it is also a good way to find out how people are actually using Hadoop in commercial applications. This is the part I liked best. There were several speakers who talked about their (very) recent experience with commercial roll-outs and I spoke to people at my table and over lunch about what they are doing with this technology.

Many companies are still experimenting, but there are several early adopters who have real production deployments. Unsurprisingly, the latter includes many starts-ups.

Two years ago I learned more about the technology behind Hadoop in a great book, Data-Intensive Text Processing using MapReduce.

Speakers

Mike Olson, the CEO of Cloudera gave a nice overview of the market which he called "Next Gen Data Management". Instead of talking about gigabytes of data, we're moving towards petabytes of data, much of which is machine-generated. By using the map reduce paradigm, where every node in the cluster combines data storage with computing power, an enormous leap in performance and cost-effectiveness can be achieved.

"Pushing the the analysis to the data", as he described it, is somewhat similar in approach to the -fanciful- agent-approach that was popular at the VU when I graduated there.

Another interesting point that was made was that the use of tools like Hadoop, which is dramatically cheaper than previous big Data Warehouse solutions, leads to a different approach of data gathering. It used to be that people were afraid to "over-use my data", in that there was a hard limit how much the current Data Warehouse could hold, data-wise or at least under the current license. With Hadoop the approach is much more: let's collect as much as we can, go after everything.

It is clear that Cloudera has encountered some skepticism from 'business' towards Hadoop. In their presentation they emphasized strongly that they will take care of things like upgrades etc, to free up the employees to do actual analysis. Obviously Hadoop is not as mature as existing data warehouse tools, which is a very well developed field. I got the feeling that the typical data warehouse organization has pretty high expectations on the maturity of tools -- a challenge and opportunity for Cloudera at the same time.

The only partner that was a bit out of place, for me, was HP. Big Data is very much about software, and hardware is really just an afterthought. The speaker wasn't very inspired (or inspiring) either.

Josh Wills, data scientist

I particularly enjoyed the presentation by Josh Wills, a data scientist at Cloudera who has worked on the Google ad engine. He described several techniques for creating models and touched on how they would work in the Hadoop eco-system.

"My nightmare scenario is that a business person comes to me a says: 'Go find me some insights'. It doesn't really work that way."

He emphasized the need for monitoring and experimenting -- it is impossible to things right from the get-go. He illustrated this with an example of a team in the 1960's that was able to design a human-powered airplane. That team, unlike the competition, focused very strongly on quick turn-around: they were able to rebuild their crash airplanes quickly, which allowed them to experiment much more than the others. Interestingly, a member in the audience spoke up and mentioned that he had been one of the grad students working at CalTech at the time, which was cool. The lesson is one that holds true in my field of software engineering as well: don't try to build a complex system from scratch, it won't work. Instead, start with a very simple but working system, and make it better over time.

Follow-ups

I'd like to read more on the ETL tools that SyncSort offers -- perhaps this might be of interest to some other teams at my work. Most of all I'd like to learn more about Mahoud, for example by starting at Cloudera's 'Building Recommender Systems'. This is fun stuff!