Posts

Showing posts from April, 2007

links for 2007-04-30

links for 2007-04-28

Pig - Analysis of Very Large Data Sets
We are creating infrastructure to support ad-hoc analysis of very large data sets.
(tags: database)

links for 2007-04-23

Better Gmail
A collection of scripts to enhance gmail.
(tags: emailgmail)

links for 2007-04-22

NetBeans Ruby IDE
A stand alone ruby and ruby on rails IDE on Netbeans
(tags: ruby)

links for 2007-04-20

Welcome to the JDBM Project
Java object persistence engine, similar to Berkeley DB
(tags: javadatabase)

Calculating and Compressing OLAP Cubes

Over the weekend, I've been hard at work at writing an implementation of the Dwarf algorithm for compressing and calculating (aggregating) OLAP cubes. The initial results look good, and I've learned a great deal.

To begin, a little background. OLAP cubes typically perform numerous calculations and aggregations on a dimensional model in order to speed up query performance. Data warehouses are all about storing extremely large sets of data, and presenting it to the user for analysis. Users being users, they don't want to wait while you perform a SQL GROUP BY and SUM over 10 million rows. So an effective OLAP cube calculates all of the GROUP BY combinations ahead of time, dramatically speeding up users' queries against the cube.

The trouble is that the number of GROUP BY combinations increases exponentially as the numbers of dimensions (and dimension attributes) increases. Plus, data warehouses are meant to store data over multiple years, and at a very fine grain. Th…

links for 2007-04-15

Washing Machine Smell
Tips and tricks to get rid of the nasty smell in my washing machine.
(tags: house)

links for 2007-04-14

CURE for Cubes: Cubing Using a ROLAP Engine
Notes on building and storing an OLAP cube inside a relational database.
(tags: olapdatabase)

Another Ruby ETL Project from Google’s Summer of Code

Google's Summer of Code is sponsoring a Framework for ETL and Data mining operations in Ruby.

Hmm... sounds a lot like ActiveWarehouse ETL, which is a Ruby library for ETL. ActiveWarehouse-ETL is already in progress and well on its way.

Here's the whole list of Ruby projects sponsored by Summer of Code.

Dabble DB Brings the Web of Data to Life

Dabble DB has completely blown me away. Dabble DB is like Club Med for your data. You want your data to get a massage while sipping a Mai Tai on the beach, you got it. Your data will get the five star treatment at Dabble DB.

So everyone is talking about Web 3.0, AKA the Web of Data, AKA the Semantic Web. Those visions are all well and good, and I do believe we'll see a Data Centric Web soon. But if there's a Web of Data, that must mean you've got Data on the Web.

What? Your data is buried in some SQL Server database on the company LAN? That doesn't sound very webby to me. And you're building all these custom, one-off, Visual Basic apps or Excel macros manage your data? Tisk, tisk. So not webby.

This is where Dabble DB comes in. Not only does it provide a very slick, dripping with AJAX interface for you to import and manage your data, it's a *very* smart interface. Normal muggles (Haven't read Harry Potter? Whaaaa?) can easily use Dabble DB to clas…

links for 2007-04-12

Yahoo! UI Library: Grids CSS
CSS framework, gets me out of the business of micromanaging my web page layouts.
(tags: css)


Yahoo Grids Cheat Sheet
Cheat sheet PDF for Yahoo Grids CSS layout framework.
(tags: css)


Ext JS - JavaScript Library
Javascript framework and library that contains widgets.
(tags: cssjavascript)


Mims's "This Is Why I'm Hot": A Graphical Dissertation on America's Number One Song by Rob Harvilla
Mims is hot like Mims. Does it get hotter than that?
(tags: music)

Excellent REST Tutorial Series

Softies on Rails is posting an excellent REST tutorial series. They are doing a great job in describing how a web application is just a collection of resources (and not necessarily web pages), and how you can think of your web application as the gateway to manipulate those resources. If you want to learn REST, and good web architecture, this tutorial series is the way to start.

This reminds me of the great "A resource is a web page about your car, not your car itself" debate.

Norm Walshsums this debate up nicely:


Suffice it to say that some folks think http://example.org/some/path must identify “a document” (what the AWWW calls an “information resource”) and some folks think it can identify anything at all.


In any case, thinking of your web application as a collection of resources is the way to go. You don't usually need to get all carried away with "what is a resource?" unless you are trying to model knowledge, but that's another story...

Ruby on Rails Mangles Decimal Type to Integer

So when is a DECIMAL not a DECIMAL? Go ahead, take your time. I'll wait.

A decimal type is converted to an integer by Ruby on Rails through its infinite wisdom. As seen in active\_record\connection\_adapters\abstract\schema\_definitions.rb, line 180 (in Rails 1.2.3):

when /decimal|numeric|number/i
extract_scale(field_type) == 0 ? :integer : :decimal

So, apparently, Rails thinks, "Well, I know you specified a decimal type, but hey, you didn't specify a scale, so you must have made a typo! Here, let me just change that to integer for you."

Or, in other words:

You: I'd like a hamburger with nothing on it.

Rails: OK, here's your hot dog.

You: WTF??!?

Apparently, this issue has been reported before. However, as of Version 1.2.3 of Rails, this issue still exists.

I've just opened a new issue on the Rails Trac site. You can see if any of the core developers will see why converting a decimal to an integer is completely counter intuitive.

links for 2007-04-07

Apple - QuickTime - Tutorials - Creating Amazing Video Content with H.264
Specific instructions for encoding H.264 video using Quicktime Pro. Useful for encoding to YouTube.
(tags: videoyoutube)

links for 2007-04-06

Introducing BDD
Nice introduction to behavior driven development (BDD).
(tags: programming)


How To Make YouTube Videos Look Great on Squidoo
compression suggestions and techniques for youtube
(tags: youtubevideo)

links for 2007-04-03

Caboose Sample rails application
A blank shell for up to date, best practices, Ruby on Rails applications.
(tags: rubyonrails)