Gender Bias in Scientific Citations
Data and code are on the Canvas site in midterm.zip.This includes midtermStart.R.
We will study the effect of gender on scientific citations.To this end,we will analyze a dataset consisting of all publications in top 4 statistics journals (AOS,JASA,Biometrika and JRSSB)from 2003 to the first half of 2012.The data includes 3607 authors and 3248 research articles.Gender of each author has been predicted based on their first name using several databases containing information from different geographical regions (see R package gender for more information).We incorporate topics of each paper and decompose the title into words. The data consists of four tables:
(1)AUTHORFINAL.CSV
contains 3607 rows and five columns describing names (first and second)and predicted gender(female,male and unknown)and a confidence score outputted by gender predic- tion algorithms (ranging from 0 to 1 where large values provide more confidence that the name is in fact a male name).
(2)PAPERCOVARIATES.CSV
is a matrix of paper characteristics.There are 3248 rows and 6 columns
·Citation (the number of times the paper has been cited)
·Year(the year of publication)
·Journal(where it was published,either AOS,JASA,JRSSB or Biometrika)
·References(how many of the other 3247 papers has each paper referenced)
·Seniority(logarithm of the total number of papers wrote by all coauthors)
·Nauthors(the number of coauthors on the paper)
·Female coauthors(the number of female coauthors)
(3)WORDDOCCITATION.CSV
is a simple triplet matrix of 20303 article/word pairings.The table consists of 3 columns
·Paper index(a row index of PaperList.cSv)
·Word used (a row index of CitationWords.CSV)
·Times word:how many times the word occurred in the title
(4)CITATIONWORDS.CSV
contains 686 alphabetically ordered words that occur in the titles of the research articles.
1 Marginal Significance Screening and False Discovery
To begin,we would like to find words in paper titles that associate with citations.We will look at one-at-a-time (marginal)regressions (regressing outcome on each word separately)to see if there are certain key title words that can predict citations.We willlook at the following outcome Y;=log(citationi+1)for the i-th research article.
That is,we fit
Yi=α+β;[xij>0]+Eij
for each word j with count xij in the title of i-th article,and return the p-values associated with a test of βj≠0.
1.1
Run the marginal screening for every title word.Plot the p-values and comment on their distri- butions.Is there enough signal to predict citations based on the topic of the article?
1.2
What is the alpha value associated with 20%False Discovery Rate?How many words are significant at this level?What are the advantages and disadvantages of FDR for word selection?
1.3
If I expect no more than 3 false discoveries,what is the maximum number of discoveries you can make?Are the p-values independent?Discuss.
2 LASSO Variable Selection
We want to build a LASSO predictor of citations using title words and characteristics of the paper as well as coauthors(such as their seniority and gender).
2.1
Use the LASSO method to come up with a set of words that predict Y from Q1.Pick a lambda and comment on the in-sample R².Is there enough evidence to conclude that titles predict citations?
2.2
Repeat the analysis from 2.1 but add extra covariates(characteristics of the paper and coau- thors).Report your selected lambda and what is the in-sample R²now?Describe the LASSO path and pick the top 10 strongest coefficients.What is the interpretation of the coefficient of the word lasso?
2.3
Note that around 50%of the papers got no citation whatsoever.We now try to predict whether
a paper will get at least one citation by running a LASSO with a binary outcome Y;=Ⅱ(citationi>0).
Repeat the analysis from 2.2.Report your selected lambda what is the in-sample R²?What is the interpretation of the coefficients of words seniority and female-coauthors.
2.4
Construct a95%confidence interval for the LASSO coefficient of the word female-coauthors. Does this interval cover your estimate from 2.3?
3 High-dimensional Controls and Double LASSO
We want to isolate the effect of gender(the number of female coauthors)on the number of citations,controlling for relevant words (paper topics)and other characteristics such as seniority (experience)of the authors.Our treatment variable will be
d=number of female coauthors
in the i-th article and we use binary outcome Y;=Ⅱ(citation;>0).
3.1
Explore the association between citations and gender (both graphically and using a marginal regression or other statistical tests)to see if there is any association.Interpret the coefficient from your marginal regression(Y on d).Predict d from x(title words and other predictors) using Poisson regression(see below for guidance on Poisson regression),and comment on the degree of confounding we can expect.Is there any information in d independent of x?
Poisson regression is used for modeling count response,say if you have y∈{0,1,2,3,} instead of a binary response y∈{0,1}.To run a lasso regression for d on x with Poisson link, you specify the family to be “poisson”,as
gamlr(x =X,Y =d,family ="poisson").
Remember to use type ="response"for prediction.
3.2
Isolate the effect of d by running the causal(double)LASSO.Interpret this effect and compare it to the effect obtained from the naive LASSO.
3.3
Consider the estimated treatment effect for d.We want to know how variable this estimate is (i.e.compute the standard error)and construct confidence intervals for inference.
-What is the standard error for the treatment effect d?
-Find the 95%CI for d?
Can we safely claim that the effect is casual?