{"id":116,"date":"2011-02-13T20:08:00","date_gmt":"2011-02-13T07:08:00","guid":{"rendered":"http:\/\/deborahfitchett.com\/blog\/?p=116"},"modified":"2011-02-13T20:08:00","modified_gmt":"2011-02-13T07:08:00","slug":"converting-a-plaintext-bibliography-to-endnoteris-format-with-help-from-linuxterminal","status":"publish","type":"post","link":"https:\/\/deborahfitchett.com\/blog\/2011\/02\/converting-a-plaintext-bibliography-to-endnoteris-format-with-help-from-linuxterminal\/","title":{"rendered":"Converting a plaintext bibliography to Endnote\/RIS format with help from Linux\/Terminal"},"content":{"rendered":"

[Update 16\/7\/2011:<\/strong> See my more recent post on the topic, Launching Ref2RIS – convert your typed bibliography to Endnote format<\/a>, which makes things even easier.]<\/p>\n

You won’t want to do this unless you’ve got literally hundreds of references. Any less, and these suggestions<\/a> are way easier.<\/p>\n

1. Format references so they’re each on their own line – no blank lines.<\/p>\n

2. Use Word’s “Find Special” capabilities to replace a phrase in italics<\/i> with {it}a phrase in italics{endit} and a phrase in bold<\/b> with {b}a phrase in bold{endb}.\u00a0\u00a0(Similarly if the citations contain underlines.)<\/p>\n

3. Save as plaintext – say, source.txt.\u00a0\u00a0Now the fun begins…\u00a0\u00a0My own source text contains 600-odd lines in ACS style, like this: <\/p>\n

Bamford, C. H.; Tipper, C. F. H. {it}Comprehensive Chemical Kinetics{endit}; Elsevier: New York, {b}1977{endb}. 
House, D. A.{it}Chem. Rev.{endit} {b}1962{endb}, {it}62{endit}, 185 <\/pre>\n
4. Open up Terminal or some other Linux command line.<\/p>\n
5. Endnote records are separated by a line <\/p>\n
ER\u00a0\u00a0- <\/pre>\n
 – that’s two spaces before the hyphen and one after.\u00a0\u00a0(All these details come from Endnote’s help pages<\/a>.) This is the easy part: type in <\/p>\n
sed -e 's\/^\\(.*\\)\/\\1ER\u00a0\u00a0- \/' source.txt > source1.txt<\/pre>\n
6. The start of each Endnote record tells you what kind of citation it is – eg a book, journal etc.\u00a0\u00a0To find every line that includes a colon (ie separating the publisher from the city published in) type in <\/p>\n
sed -e 's\/^\\(.*:\\)\/TY\u00a0\u00a0- BOOK@@\\1\/' source1.txt > source2.txt<\/pre>\n
 Note 1: The “@@” is in there as a sign that you’ll need to replace this with a new line later; but we want to keep everything on one line for now.
Note 2: This is a good example of why this whole method is highly suspect, because it’ll also catch citations which have a colon in the article title or in a typo or whatever.\u00a0\u00a0So if you can think of a better sign that a citation is a book then use that instead of the colon.<\/p>\n
Alternatively, you could type in <\/p>\n
sed -e 's\/^\\(.*{it}[0-9]*{endit}\\)\/TY\u00a0\u00a0- JOUR@@\\1\/' source1.txt > source2.txt<\/pre>\n
 to find every line that contains {it}[some number]{endit} which, in my source, is the best indicator that I’m dealing with a journal.\u00a0\u00a0The same caveats apply – you’ll get both false positives and false negatives.<\/p>\n
Anyway, keep doing what seems best given your source, and fix up the inevitable mistakes by hand until each line starts with TY\u00a0\u00a0– something.\u00a0\u00a0If you want to give up and just assume that everything that isn’t already assigned as something must be a journal then try <\/p>\n
sed -e 's\/^\\([^(TY\u00a0\u00a0- )].*$\\)\/TY\u00a0\u00a0- JOUR@@\\1\/' source2.txt > source3.txt<\/pre>\n
I now have source looking like:<\/p>\n
TY\u00a0\u00a0- BOOK@@Bamford, C. H.; Tipper, C. F. H. {it}Comprehensive Chemical Kinetics{endit}; Elsevier: New York, {b}1977{endb}. 
ER\u00a0\u00a0- 
TY\u00a0\u00a0- JOUR@@House, D. A.{it}Chem. Rev.{endit} {b}1962{endb}, {it}62{endit}, 185 
ER\u00a0\u00a0- <\/pre>\n
7. Now we keep playing with patterns.\u00a0\u00a0(You may be able to do large chunks of this with regular find\/replace, but for illustrative purposes I’ll keep using Terminal.)<\/p>\n
For example, in my source the authors are nicely set off: they come after “@@” and before the first “{it}” (or “in {it}”), and if there’s more than one of them they’re separated by “;”.\u00a0\u00a0So a few commands: <\/p>\n
sed -e 's\/@@\\(.* in {it}\\)\/@@A1\u00a0\u00a0- \\1\/' source3.txt > source4.txt
sed -e 's\/@@\\(.* {it}\\)\/@@A1\u00a0\u00a0- \\1\/' source3.txt > source4.txt
sed -e 's\/;\\(.*;\\)\/@@A1\u00a0\u00a0- \\1\/' source5.txt > source6.txt (This one I had to repeat a few times depending how many authors could be cited in one reference; there's supposed to be a way to do it globally but my unix fu is not strong.)
sed -e 's\/;\\(.*{it}\\)\/@@A1\u00a0\u00a0- \\1\/' source8.txt > source9.txt<\/pre>\n
Journal titles: <\/p>\n
sed -e 's\/^\\(TY\u00a0\u00a0- JOUR.*\\)\\({it}.*{endit} {b}\\)\/\\1@@JO\u00a0\u00a0- \\2\/' source9.txt > source10.txt<\/pre>\n
Years: <\/p>\n
sed -e 's\/\\({b}[0-9]*{endb}\\)\/@@Y1\u00a0\u00a0- \\1\/' source10.txt > source11.txt<\/pre>\n
And so forth.\u00a0\u00a0You pretty soon start to see why the first suggestion on most lists of ways to convert plaintext citations into RIS format is always “Just type it in \/ search for it again by hand”.\u00a0\u00a0The method above is really only suitable if you’ve got literally hundreds of citations. (I have 639, plus or minus.)<\/p>\n
8. Eventually you’ll be at a point where you can do a simple find\/replace to change @@ to a new line and nuke all the {it} and so forth.\u00a0\u00a0This will be a great relief.<\/p>\n
9. Rename your final saved file from source12.txt to source12.ris and open with Endnote.<\/p>\n
10. Bonus material:\u00a0\u00a0if this was a bibliography to a paper using numbered<\/b> citations in order using eg [1], then in that paper you can do a find\/replace on [ -> { and ] -> }, then tell the Endnote plugin to format citations, and voila, the best magic ever.\u00a0\u00a0(If the paper uses author\/date citations then you’ll have to link them by hand, sorry.)<\/p>\n","protected":false},"excerpt":{"rendered":"
[Update 16\/7\/2011: See my more recent post on the topic, Launching Ref2RIS – convert your typed bibliography to Endnote format, which makes things even easier.] You won’t want to do this unless you’ve got literally hundreds of references. Any less, and these suggestions are way easier. 1. Format references so they’re each on their own […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[],"tags":[84,123,15],"_links":{"self":[{"href":"https:\/\/deborahfitchett.com\/blog\/wp-json\/wp\/v2\/posts\/116"}],"collection":[{"href":"https:\/\/deborahfitchett.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/deborahfitchett.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/deborahfitchett.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/deborahfitchett.com\/blog\/wp-json\/wp\/v2\/comments?post=116"}],"version-history":[{"count":0,"href":"https:\/\/deborahfitchett.com\/blog\/wp-json\/wp\/v2\/posts\/116\/revisions"}],"wp:attachment":[{"href":"https:\/\/deborahfitchett.com\/blog\/wp-json\/wp\/v2\/media?parent=116"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/deborahfitchett.com\/blog\/wp-json\/wp\/v2\/categories?post=116"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/deborahfitchett.com\/blog\/wp-json\/wp\/v2\/tags?post=116"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}