Home/ Questions/Q 6383727
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-25T02:41:08+00:00

I am using a file consisting of published scientific data. I’m using this file

  • 0

I am using a file consisting of published scientific data. I’m using this file with a program that reads in the first 5 space delimited data fields, and everything after that is considered a comment by the program.

2 example lines (of thousands):

FeII 1608.4511 0.521 55.36 -1300 M03 Journal of Physics
FeII 1611.23045 0.0321 55.36 1100 01J AJ

The program reads it as: 

FeII 1608.4511 0.521 55.36 -1300 
FeII 1611.23045 0.0321 55.36 1100

These numbers are each measurements and most (don’t get me started) have associated errors that are not listed in this file. I would like to store this information in a useful and updatable way. That is, say the first entry FeII 1608.4511 has an error of plus/minus 0.002. Consider when a new measurement is made and changes it to: FeII 1608.45034 plus/minus 0.0005. I would like to update the value, the error, and record some information about the publication that it came from.

The program that uses this file is legacy code and is both crucial and inflexible: and it needs the file to look like the above output when it’s read in. I would really like for there to be a way to update the input file to include things like errors on the values and publication hyperlinks in comments. I would also like a kind of version control ability to return the state of this large file today; or in 5 months after 20 more lines are updated with new values.

Any suggestions on how best to accomplish this? Should I store everything in some kind of database?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Databases are deeply tied to identity. If a database can’t identify a row by the data that’s in it, a database isn’t going to help you.

    If I were you, I’d start by storing the base file in a version control system, not a database. At 20 changes per 5 months, I’d probably make those changes manually and commit each batch of changes. (I don’t know what might constitute a batch for you. Could be a single change every time.)

    Since the format of the existing file is both crucial and brittle, I’m not sure whether modifying it is a good idea. I think I’d feel better about storing error ranges and publication hyperlinks in a separate file, and using a script to put the pieces together for applications that can use error ranges and hyperlinks.

    • 0