Creating a dataset from an XML file in R statistics
===

I am trying to download an XML file of journal article records and create a dataset for further interrogation in R. I'm completely new to XML and quite novice at R. I cobbled together some code using bits of code from 2 sources:
[GoogleScholarXScraper][1]
and
[Extracting records from pubMed][2]

    library(RCurl)
    library(XML)
    library(stringr)

    #Search terms
    SearchString<-"cancer+small+cell+non+lung+survival+plastic"
    mySearch<-str_c("http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=",SearchString,"&usehistory=y",sep="",collapse=NULL)

    #Seach
    pub.esearch<-getURL(mySearch)

    #Extract QueryKey and WebEnv
    pub.esearch<-xmlTreeParse(pub.esearch,asText=TRUE)
    key<-as.numeric(xmlValue(pub.esearch[["doc"]][["eSearchResult"]][["QueryKey"]]))
    env<-xmlValue(pub.esearch[["doc"]][["eSearchResult"]][["WebEnv"]])

    #Fetch Records
    myFetch<-str_c("http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&WebEnv=",env,"&retmode=xml&query_key=",key)
    pub.efetch<-getURL(myFetch)
    myxml<-xmlTreeParse(pub.efetch,asText=TRUE,useInternalNodes=TRUE)

    #Create dataset of article characteristics #This doesn't work
    pub.data<-NULL
    pub.data<-data.frame(
      journal <- xpathSApply(myxml,"//PubmedArticle/MedlineCitation/MedlineJournalInfo/MedlineTA", xmlValue),
      abstract<- xpathSApply(myxml,"//PubmedArticle/MedlineCitation/Article/Abstract/AbstractText",xmlValue),
      affiliation<-xpathSApply(myxml,"//PubmedArticle/MedlineCitation/Article/Affiliation", xmlValue),
      year<-xpathSApply(myxml,"//PubmedArticle/MedlineCitation/Article/Journal/JournalIssue/PubDate/Year", xmlValue)
      ,stringsAsFactors=FALSE)


The main problem I seem to have is that my returned XML file is not completely uniformly structured. For example, some references have a node structure like this:

    - <Abstract>
    <AbstractText>The Wilms' tumor gene... </AbstractText>

Whilst some have labels and are like this

    - <Abstract>
    <AbstractText Label="BACKGROUND &#38; AIMS" NlmCategory="OBJECTIVE">Some background text.</AbstractText>
    <AbstractText Label="METHODS" NlmCategory="METHODS"> Some text on methods.</AbstractText>

When I extract the 'AbstactText' I am hoping to get 24 rows of data back (there are 24 records when I run this made up search today), but xpathSApply returns all labels within 'AbstactText' as individual elements of my dataframe. Is there a way to collapse the XML structure in this instance/Ignore the labels? Is there a way to make xpathSApply return 'NA' when nothing is found at end of a path? I am aware of xmlToDataFrame, which sounds like it should fit the bill, but whenever I try to use this it doesn't seem to give me anything sensible. 

Thanks for your help


  [1]: https://github.com/tonybreyal/Blog-Reference-Functions/blob/master/R/googleScholarXScraper/googleScholarXScraper.R
  [2]: http://rpsychologist.com/an-r-script-to-automatically-look-at-pubmed-citation-counts-by-year-of-publication/