Names

06/08/2009 - [ names name online research database analysis ] - posted by fansipans

Two cool, and unexpected, online applications for researching and comparing given names and surnames:

• WhitePages.com: names.whitepages.com - Shows popularity, given/surname combinations, and geographical distribution.


• WolframAlpha: woflramalpha.com - Shows a wide variety of information about (most) names, and allows for name comparisons, for example: Michael vs. Bruno

Voronoi Diagrams

03/10/2008 - [ math software modeling diagram graph algorithm design interface data analysis ] - posted by fansipans

I ran across an interesting article today, about a simple solution to a seemingly complex problem.

The question is: how do you subdivide a plane containing multiple points into cells containing each point, enclosing all the points in the plane closest to that point?

One of the ways of constructing these 'Voronoi Diagrams', as they're called, is Fortune's algorithm, which is based around intersecting parabolas.

See the article here: Voronoi Diagrams and a Day at the Beach

And a java simulator of Fortune's algorithm here: Fortune's Algorithm Simulator

What I find fascinating about these types of diagrams are the ways in which they can be used to aid data analysis through visualization and geometric representation (see also this blog post.)

There are so many secondary characteristics of these diagrams: cell area, adjacent cells, border length, distance to border, border segment count... it makes me want to throw all sorts of different data sets in there and see what unobvious connections and patterns become plain as day.

Statistics, Number Ranges, and Data Analysis

01/08/2008 - [ analysis statistics postgresql database data sql math ] - posted by fansipans

A gripe I've always had with numerical and statistical analysis using spreadsheets and traditional RDBMSes is the lack of native support for ranges.

So say I have a table 'num' with the numbers:

(10,11,14,15,11,10,6,9,10,8)

Getting the average is easy,

SELECT avg(n) FROM num; => 10.4

But how could I easily get the numbers in 'num' that are within 20% of the average, without coding a bunch of explicit math?

Using PostgreSQL's Geometric types and operators, we can represent each number as a circle on the x-axis, with its radius representing the allowed tolerance, so a circle at (12,0) with radius 6 would represent the number 12, with accuracy of +/- 50%. Once you have all your numbers with their tolerances, simple use the overlap operator '&&' to tell you which circles overlap another circle. For the average, I'll use a circle at (10.4,0) with radius 0.

So for my simple table of integers, this tells me which ones are within 20% of 10.4.

SELECT n FROM num WHERE
circle(point(n,0),n*0.2) && circle(point(10.4,0),0);

=> (10,11,11,10,9,10)

It would be just as easy for each row in the table, to have a 'tolerance' or 'accuracy' column, or to derive that tolerance value based on previous query.

If I wanted to further measure the difference , I could use the '<->' distance operator to see how far apart two circles are.

This lets you get away with relatively simple SQL for complex statements like "For the ordered set X, show me the largest contiguous subset Y such that each number is within 3% of its neighbor, and within 10% of the series min and max values."

The native support for geometric operations in PostgreSQL seems broad and deep. Types include point, line, box, path (open and closed), polygon, and circle. Here are a few of the many operators:

## - Closest point to first object on second object
<-> - Distance between objects
&& - Overlap?
<< - Stricly left of?
>> - Stricly right of?
?# - Intersects?
@> - Contains?

What's really cool about this is that there are many ways where a geometric context can help associate, analyze, and relate data, where the underlying mathematical formulas would be hideously complex and non-intuitive. Imagine datapoints across one or two dimensions, circles of differing radii, paths, bounding boxes, etc.