Thursday, September 25, 2014

Is the number of female Royal Society Fellowships a red herring?

Personally I'm in no doubt that conscious and unconscious, personal and systematic biases hinder women's progress in science on many levels. However, the recent outcry over the number of women selected by the Royal Society for university fellowships is (probably) a red herring.

Lets break it down statistically. We'll be looking for significant statistical deviation, at the 0.05 level, and doing 1-tailed tests (only looking at whether women are under-represented, I doubt if anyone including myself much cares about the possibility of men being under-represented)

Here is the table of application and acceptance rates since 2010 from the +The Royal Society



The key ingredients here, in assessing whether the Royal Society shows any bias, are the number of women accepted, and the percentage of female applicants.

Doing a 1-tailed binomial significance test for the number of women accepted each year, based on the number expected give the percentage of applicants, we get the following (I've used the numbers above, though other sources give 43 for the number of awards in 2014 - I don't know why that discrepancy exists):

P_2014 = 0.0164
P_2013 = 0.437
P_2012 = 0.533
P_2011 = 0.440
P_2010 = 0.972

(p values calculated using http://graphpad.com/quickcalcs/binomial1.cfm)

Clearly 2014 is very different from the other years, where the number of awards to female scientists was pretty much as expected (in fact we almost saw a significant over-representation in 2010, but as I said before, who cares? I think we can live with that!)

So it looks like 2014 saw a sudden emergence of bias in the Royal Society's selections. But we need to consider the fact that we're performing this significance test 5 times. If you look at anything for long enough it's going to do something 'odd', and no doubt generate Twitter outrage. The standard way to account for this in statistics is a correction for multiple hypothesis testing. The best standard method is probably the Holm-Bonferroni method, which says that to find any significant results here we need to take the p-value we deem to be significant (0.05) and divide it by the number of tests (5). So we're left with a new significance level of 0.01, just below what we see in 2014. I'd say the best interpretation of this years acceptance rate then is that it's suspicious, but not sufficiently low to be definite evidence of bias.

Of course, we should be asking questions about why only 20% of applicants were female in the first place. The Royal Society, like almost every scientific body, could probably be doing more to encourage female applicants. But given the applications we saw, there is no firm statistical evidence of biased selection. Of course, if we saw the same sort of results next year that would change everything - but given the outcry I think we can rest assured that that won't be allowed to happen. I'm also a little concerned about that 40/43 discrepancy which might make this result just significant - thats the absurd binary nature of significance tests for you. If you want to do and compare a Bayesian analysis then please do and I'll happily post the results.

Update 1: Luca Borger points out that accepting so few women applicants this time is unlikely to mean more apply next year. And Ben Sheldon points to the fact that fewer women were selected at every stage of the process. The problem with assessing bias against women in science is that the overall effect, which is undoubtably large, comes from many cumulative and feedback effects like this, so the outcry against the Royal Society may be useful in combating this, even if in isolation their selection does not appear statistically biased.

Update 2: David Sumpter shows how the cumulative effect of many individually non-significant biases can lead to huge system-wide imbalances.

Sunday, May 18, 2014

This is a blog post I wrote about our seminar speaker at IFFS on Friday May 16, mainly for David Sumpter's blog and the IFFS website, but it won't do any harm to post it here as well. I and the speaker, Michael Osborne, did our PhDs together, and now he's one of Oxford's foremost experts on Machine Learning. In this presentation he described how Machine Learning will change everyone's employment in the coming century...

The future of automation


Depending on your perspective, technological development has been saving us from drudgery, or destroying our livelihoods, for centuries. From the very first domestication of animals we’ve been finding ways to perform tasks with less human action since civilisation began.


Last week Dr. Michael Osborne from the University of Oxford gave a presentation at the Institute for Futures Studies showing his predictions about which of us will be losing our jobs in the century to come. Michael, as an expert in Machine Learning, is interested in which jobs will be automated as a result of increasing artificial intelligence in the Big Data era. He and his colleagues have been impressed at the rapid pace with which tasks that were seen as impossible for computers to perform, such as driving a car or translating accurately between different languages have become almost routine.


Machine Learning itself can be used to predict which tasks are ripe for automation. First they gathered data on the skills necessary to perform over 700 different jobs, such as social sensitivity, manual dexterity and creativity. A panel of experts was then asked to predict which of 70 specific jobs would be automatable in the near future. Using Gaussian process regression, Michael and his colleagues learned a relationship between the skills a job requires and the probability that a computer will be able to perform, and extrapolated this relationship to the 700 jobs the panel had not evaluated. Their results give us a view on which sectors of the economy will be most affected by the continued rise of artificial intelligence. The graph below shows, by sector, what proportion of jobs are at low, medium or high risk of being automated. In general, those jobs requiring the most necessary social interactions and/or high level creativity appear to be safest from the coming tide of job losses, but none of us can rest too easy!


Inline images 1


However, we shouldn’t be too distressed at this imminent redundancy. As Michael pointed out for example, while technological progress has reduced the workforce in agriculture from almost 40% of employment in 1900 to around 2% today, the total unemployment rate has barely changed. Technology has allowed society to move human labour to more productive areas. The results of Michael’s analysis also show that it is generally lower paid, lower skilled jobs that will be destroyed, giving hope that people will be able to move into better employment, if society provides them with the necessary skills.

Inline images 2


Nonetheless, Michael also showed examples of resistance to change, such as the guilds of Tudor England blocking the development of machines for making textiles in fear of their members livelihoods. The ever increasing rate of automation, and the subsequent need for people to continually adapt to new careers and find new skills presents society with a powerful challenge, that may require new social contracts, such as a guaranteed citizen’s income and much more investment in public education to solve. It will be exciting to see where this process takes us!