Question

I imported data from tab-delimited text files into a data.frame. Now I’d like to adjust and edit the contents of a column called C1. Regular expressions were recommended to me for this purpose. At first I used:

for (rn in 1:length(C1))
C1s <- strsplit(as.character(C1[rn]), "; ", fixed = TRUE)[[1]]

to separate individual entries on examples such as these:

• [Zhang, Junling] China Agr Univ, Coll Resources & Environm Sci, Beijing 100094, Peoples R China; [Zhang, Junling] Univ Hohenheim, Inst Plant Nutr, D-7000 Stuttgart, Germany; [George, Eckhard] Humboldt Univ, Inst Crop Sci, Dept Plant Nutr, D-14979 Grossbeeren, Germany; [George, Eckhard] Leibniz Inst Vegetable & Ornamental Crops Theodor, D-14979 Grossbeeren, Germany
• UNIV DORTMUND,INST PHYS,D-44221 DORTMUND,GERMANY; HUMBOLDT UNIV,INST PHYS,D-15738 ZEUTHEN,GERMANY
• UNIV KEELE,DEPT CHEM,KEELE ST5 5BG,STAFFS,ENGLAND; MT SINAI HOSP,MT SINAI SCH MED,DR A A FISHBERG CTR NEUROBIOL,NEW YORK,NY 10029; HUMBOLDT UNIV BERLIN,CHARITE HOSP,DEPT PATHOL & CLIN BIOCHEM,BERLIN,GERMANY
• NIFAD,MOLEC MICROBIOL LAB,BETHESDA,MD

Then I wanted to

• set everything to upper case
• replace the strings “Humboldt…Germany” with “HUMBOLDT” replace the strings “England”, “Scotland”, “Wales”, and “North Ireland” with “UNITED KINGDOM”
• replace combination of U.S. state abbreviations and ZIP codes with “USA”
• replace U.S. state abbreviations with “USA”

In addition I wanted to delete everything but semicolons (including the spaces after them), the words directly in front of them, and the very last word from the examples above.
I used

gsub('[.*\\] ', ''(toupper(C1s))  

and

gsub(',\\s*', ','(toupper(C1s))  

, e.g., but couldn’t get it to work correctly.
I’d like to get the following output:

• PEOPLES R CHINA; GERMANY; HUMBOLDT; GERMANY
• GERMANY; HUMBOLDT
• UNITED KINGDOM; USA; HUMBOLDT
• USA

So my question is: How can I achieve the results I desire?
Thank you very much in advance for your consideration!

Update and additional problem

Thanks to mrdwab’s helpful reply and comments I made a lot of headway.

Unfortunately, only now I realized that there also are addresses such as these with more than one author in square brackets. The algorithm proposed by mrdwab doesn’t work correctly with these, unfortunately.

> test = c("[Bocquet, F. C.; Giovanelli, L.; Abel, M.; Porte, L.; Themlin, J. -M.] Aix Marseille Univ, Inst Mat Microelect & Nanosci Prov IM2NP, F-13397 Marseille 20, France; [Bocquet, F. C.; Giovanelli, L.; Abel, M.; Porte, L.; Themlin, J. -M.] CNRS, Inst Mat Microelect & Nanosci Prov IM2NP, UMR 6242, Marseille, France; [Amsalem, P.; Koch, N.] Humboldt Univ, Inst Phys, D-12489 Berlin, Germany; [Petaccia, L.; Topwal, D.; Gorovikov, S.; Goldoni, A.] Sincrotrone Trieste, I-34149 Trieste, Italy")

This is the result I got:

> test
[1] "[BOCQUET,  F. C." "GIOVANELLI,  L."  "ABEL,  M."        "PORTE,  L."      
[5] "FRANCE"           "[BOCQUET,  F. C." "GIOVANELLI,  L."  "ABEL,  M."       
[9] "PORTE,  L."       "FRANCE"           "[AMSALEM,  P."    "HUMBOLDT"        
[13] "[PETACCIA,  L."   "TOPWAL,  D."      "GOROVIKOV,  S."   "ITALY" 

This is the result I’d like to get instead:

[1] "FRANCE"; "FRANCE"; "HUMBOLDT"; "ITALY"

I tried using this in order to delete each square bracket and its contents individually:

C1s = gsub("(.*)[(.*)]", "\\2", C1s)

But instead of that everything between the first opening bracket and the last closing bracket was deleted …
Maybe it’d work if I replaced all semicolons within square brackets by commas first? I tried

C1s = gsub("[(.*);(.*)]", "[(.*),(.*)]", C1s)

to accomplish that, but it didn’t work.
So I’d appreciate your help in that regard!

In addition to that I’m still stuck at yet another obstacle which I just cannot seem to resolve on my own, unfortunately …
This is my current output:

> C1s
[1] "PEOPLES R CHINA" "GERMANY"         "HUMBOLDT"         "GERMANY"
[5] "GERMANY"         "HUMBOLDT"        "UNITED KINGDOM"   "USA"
[9] "HUMBOLDT"        "USA"
> dims
[1] 4 2 3 1
> is.list(C1)
[1] FALSE
> is.vector(C1)
[1] TRUE

But how exactly can I use the information in dims to create this desired output?:

[1] "PEOPLES R CHINA"; "GERMANY"; "HUMBOLDT"; "GERMANY"
[2] "GERMANY"; "HUMBOLDT"
[3] "UNITED KINGDOM"; "USA"; "HUMBOLDT"
[4] "USA"

Thank you very much in advance for your support!

Answer 1

Assuming your data looks something like this:

test = c("[Zhang, Junling] China Agr Univ, Coll Resources & Environm Sci, Beijing 100094, Peoples R China", 
         " [Zhang, Junling] Univ Hohenheim, Inst Plant Nutr, D-7000 Stuttgart, Germany", 
         " [George, Eckhard] Humboldt Univ, Inst Crop Sci, Dept Plant Nutr, D-14979 Grossbeeren, Germany", 
         " [George, Eckhard] Leibniz Inst Vegetable & Ornamental Crops Theodor, D-14979 Grossbeeren, Germany"
)

First, let’s make it all upper case:

test = toupper(test)

Then, let’s change HUMBODT…GERMANY to HUMBOLDT

test = gsub("(.*)(HUMBOLDT)(.*)", "\\2", test)

Then, let’s just extract the last part of the string.

test = gsub("(.*), +([A-Z ]+$)", "\\2", test)
> test
[1] "PEOPLES R CHINA" "GERMANY"         "HUMBOLDT"        "GERMANY"    

If I had a more complete example from you, I might be able to help you with the other parts too.

Update

Here’s a mostly worked solution that should help you figure out what to do.

Using “scan” to copy your data from your post, we can get data that looks like this:

test = c("[Zhang, Junling] China Agr Univ, Coll Resources & Environm Sci, Beijing 100094, Peoples R China; [Zhang, Junling] Univ Hohenheim, Inst Plant Nutr, D-7000 Stuttgart, Germany; [George, Eckhard] Humboldt Univ, Inst Crop Sci, Dept Plant Nutr, D-14979 Grossbeeren, Germany; [George, Eckhard] Leibniz Inst Vegetable & Ornamental Crops Theodor, D-14979 Grossbeeren, Germany", 
         "UNIV DORTMUND,INST PHYS,D-44221 DORTMUND,GERMANY; HUMBOLDT UNIV,INST PHYS,D-15738 ZEUTHEN,GERMANY", 
         "UNIV KEELE,DEPT CHEM,KEELE ST5 5BG,STAFFS,ENGLAND; MT SINAI HOSP,MT SINAI SCH MED,DR A A FISHBERG CTR NEUROBIOL,NEW YORK,NY 10029; HUMBOLDT UNIV BERLIN,CHARITE HOSP,DEPT PATHOL & CLIN BIOCHEM,BERLIN,GERMANY", 
         "NIFAD,MOLEC MICROBIOL LAB,BETHESDA,MD")

You’ll need to go step-by-step to be able to make the transformations you want. Here’s how I did it.

test.orig = test # A backup, just in case
test = toupper(test) # To uppercase
test = strsplit(test, ";") # Split everything up into a list
dims = sapply(test, length) # We might need this later
test = unlist(test) # Now, a single vector
test = gsub(",", " , ", test) # Pad those commas with some space
# Replace any 'HUMBOLDT'...'GERMANY' stuff with 'HUMBOLDT'
test = gsub("(.*)(HUMBOLDT)(.*)(GERMANY$)", "\\2", test)
# Replace a bunch of stuff with 'UNITED KINGDOM'
test = gsub("ENGLAND|SCOTLAND|WALES|NORTH IRELAND", "UNITED KINGDOM", test)
# Search for a pair of letters followed by a space followed by five digits
# and replace it with USA
test = gsub(" [A-Z]{2} [0-9]{5}", " USA", test)
# Find the item following the last comma
test = gsub("(.*), ([A-Z ]+)$", "\\2", test)
# Remove any whitespace
test = gsub("^ | $", "", test)
> test
[1] "PEOPLES R CHINA" "GERMANY"         "HUMBOLDT"        "GERMANY"        
[5] "GERMANY"         "HUMBOLDT"        "UNITED KINGDOM"  "USA"            
[9] "HUMBOLDT"        "MD"                  
# Unlist it if you need to
> split(test, rep(dims, dims))
$`1`
[1] "MD"

$`2`
[1] "GERMANY"  "HUMBOLDT"

$`3`
[1] "UNITED KINGDOM" "USA"            "HUMBOLDT"      

$`4`
[1] "PEOPLES R CHINA" "GERMANY"         "HUMBOLDT"        "GERMANY"   

But, I’ll leave you the task of figuring out how to replace state abbreviations with USA, and how to put it back in the same line order.