Logistic regression

The next data-analysis method I'm playing with is logistic regression (LR).

It's a method often said to be included in a field of statistical computing (called machine learning, statistical learning, artificial intelligence, data mining, unsupervised learning, etc). LR is mostly used in cases of yes/no classifications.

I'm trying to use data from my previous blogs where possible. So the dataset is not really super suitable for LR (I don't believe I could really use the model result to predict the probability of future undisturbed nights of sleep) - but I'll use it anyway as a data-exploration tool. The data are taken from camera imagery of our baby/toddler's sleep each night (see earlier blog posts).

It is probably possible to estimate, by eye from the red crosses, that there have been more undisturbed nights of sleep in the later months. But the logistic regression makes it clear that there is a positive relationship - i.e. our baby/toddler is sleeping through more and more nights (apparently sleeping through most nights by the end of the year).

k-means clustering

I want to play with data. In this case I want to find regimes/clusters in a dataset I saw in book "The Spirit Level" from the Equality Trust. I took the data (year 2010) from GapMinder (Hans Rosling is an idol of mine!).

It's a method in a field of statistical computing (called machine learning, statistical learning, artificial intelligence, data mining, unsupervised learning, etc). The method is called k-means clustering. 'k' simply refers to the number of groups (clusters) in the dataset.

In this case the idea is that once a country reaches a certain level of wealth (about 5000-10000 USD/capita), no longer does life expectancy increase much (or at all) any more with still further increases in wealth.

k=3. When I was at school we were taught mostly about three worlds: under-developed, emerging and developed economies. So the first group includes countries like Mozambique, Malawi, Afghanistan. The second group Russia, India, Chile, China. The final group most western countries (USA, UK, Germany, etc).

k=9. Interestingly with more clusters small features start to emerge, like a group for South Africa and Botswana where increases in wealth have not translated into increases in life expectancy.

With k-means typically equal-size clusters are typical, so small clusters don't emerge, e.g. Bermuda and Luxembourg as the two most-rich per-capita countries.

k-nearest neighbour

I previously made some observations of lake depth in central Finland and plotted them in octave. I let octave use its own smoothing and interpolation between my datapoints. Now I'm curious to look a little at a simple algorithm of my own.

It's a simple method in a field of statistical computing (called machine learning, statistical learning, artificial intelligence, data mining, supervised learning, etc). The method is called k-nearest neighbours. 'k' simply refers to the number of nearest neighbours one takes into account.

Moisture indoors

In Finland in winter, the indoor air is normally so dry that it causes problems with cracking/itchy skin. Ideal indoor moisture should be 30-50% relative humidity (NB. too high RH causes damp/mould), but easily indoor RH can get down to 10% in cold spells in Finland.

I have bought a sonic humidifier that has a 1 litre capacity and takes about a day to emit this moisture into the air. I want to know roughly how much effect this might have.

I'm going to ignore effects of sources/sinks/storage like showers/cooking/washing/ventilation.

If we imagine a starting point of 20C and 25% RH, then this is about 4 g/m^3.

Now let's imagine instantaneously and evenly adding all that 1 litre (1000 g) of water into the air in my flat (72 m^2 and about 3m tall, so volume of about 216m^3). This would be adding 1000/216 = 4.6 g/m^3.

So we would be roughly doubling the moisture content, and the RH would increase to about 50%. This would be a great result.

Future blog update...estimate the effect of time on this scenario by looking at the rate at which the humidifier works and the rate of ventilation in/out of the flat...

Winter wonderland

All winters are cold and snowy in Finland, but some much more than others.

Weather station at Helsinki Airport in Vantaa.

"Cold days" are with maximum temperature below 0C.

"Snow days" are simply days with at least some snow on the ground.

Winters are shown such that e.g. "2010" on the graph means the winter of 2009/10 (e.g. November 2009 --  April 2010).

Source: Finnish Meteorological Institute.

Summer days

I had the impression that this year had lovely summery weather. it did, and I enjoyed it lots. But apparently it was only about the same as the long-term (30-yr) average (of about 50 days per year), and even this year was cooler than some of the recent warmer summers that Helsinki's had in the last decade or so.

Weather station at Helsinki Airport in Vantaa.
"BBQ days" are with maximum temperature above 18C, at least 4 hours of sunshine and no rain.
"Hot days" are with maximum temperature above 25C.

Source: Finnish Meteorological Institute.

It doesn't rain as often as we think.

It doesn't rain on me very often going to work. I wanted to know how often it happens...

It's dry a lot of the time: for 88% of all 10-min periods and 81% of all 1-hour periods. Although the statistics for 24-hours looks less good - only 37% of days are completely dry.

My bike-ride to work is 5 minutes each way. So there's an 88% chance of being dry each day when I go to work - i.e. it'll rain on me on about only 2-3 days per month. 

I also looked at Helsinki (Vantaa) airport 1985-2015 for a few more things PER YEAR: 

• 52 BBQ days (Tmax>18C, sun>4hrs, rain=0) 
• 17 hot days (Tmax>25C) 
• 85 gloomy days (sun=0, rain>0) 
• 70 cold days (Tmax<0) 

Data from Finnish Meteorological Institute

Kävelytunnelit Helsingin keskustassa

When weather is against us, it can be nice to use some of Helsinki's tunnels to walk from place to place. But it can be hard to navigate underground. So I made a map of some key underground and covered walkways in Helsinki city centre.

Naturally there are simplifications to keep the map readable. But I'd be happy to hear about any key things to add to or remove from the map. I assume I'll make another version...