The .csv file in question is attached. It has the “magic line” defining the various branches that are to be filled. It also has two #commented lines for metadata.
It seems to read the file just fine, but when executing the ReadFile, I get a warning for each entry in the file:
" while reading data for branch mean_error on line 9
I don’t have any double- or single-quote characters in the file, so I am wondering what is causing this error. Is my .csv file mal-formed somehow?
// Error handling
6690 if (sToken.bad()) {
6691 // How could that happen for a stringstream?
6692 Warning("ReadStream",
6693 "Buffer error while reading data for branch %s on line %lld",
6694 branch->GetName(), nlines);
6695 } else if (!sToken.eof()) {
6696 if (sToken.fail()) {
6697 Warning("ReadStream",
6698 "Couldn't read formatted data in \"%s\" for branch %s on line %lld; ignoring line",
6699 tok.Data(), branch->GetName(), nlines);
6700 goodLine = kFALSE;
6701 } else {
6702 std::string remainder;
6703 std::getline(sToken, remainder, newline);
6704 if (!remainder.empty()) {
6705 Warning("ReadStream",
6706 "Ignoring trailing \"%s\" while reading data for branch %s on line %lld",
6707 remainder.c_str(), branch->GetName(), nlines);
6708 }
6709 }
6710 }
6711 } // tokenizer loop
Do you have "Ignoring trailing " or “Buffer error while” message? Or do you have exactly the chunk of message you demonstrated? In this case, it’s most probably “Ignoring trailing” part with something like ‘\r’ or similar in ‘remainder’. May be it can help if you save you cvs with ‘unix style’ new lines?
I just have exactly that chunk of the warning message, but 25 times (for each line from 2 to 26). No other warnings.
The .csv file was written using python’s csv module, which I just learned might give bad EOL characters on Windows machines (though I am on a modern Mac). They suggest opening the file in binary mode instead of text mode: stackoverflow.com/questions/1170 … terminator
I have tried this and the warnings still appear.
Oddly enough, if I try using the code in PyROOT using ROOT.gROOT.ProcessLine, the warning message lines look like this instead:
" while reading data for branch mean_error on line 2ng "
" while reading data for branch mean_error on line 3ng "
" while reading data for branch mean_error on line 4ng "
" while reading data for branch mean_error on line 5ng "
...
I don’t know where the “ng” and the extra closing quotation are coming from in PyROOT.
I think ReadFile does not work correctly with ‘\r\n’ (*), even if you fix ‘\r\r\n’ in your python script and even if you fix the first three lines (which in your case have different line endings).
One problem - do not mix line endings, since ReadFile tries to guess what line endings you are using and
The second - you’d better just process your csv to get rid of “\r\n” or ask python csv module to write ‘\n’.
Ok, at the moment I’m working on another issue you’ve reported in JIRA (related to TNtuple::ReadFile) and later I’ll revise TTree::ReadFile - we have some other reports/feature requests + I’ll fix the newline problems - so you’ll be able to use whatever you want without fixing your csv and with any newline characters you want (’\r’, ‘\n’, “\r\n” freely intermixed).
Thanks for working on it. I didn’t realize that my csv file was written with mixed EOL characters. I’ll look at how python is doing the writing to make it consistent.
[quote=“jfcaron”]Thanks for working on it. I didn’t realize that my csv file was written with mixed EOL characters. I’ll look at how python is doing the writing to make it consistent.
Jean-François[/quote]
In principle, we should process any of possible newline-characters - I fixed this in TNtuple (fix for TTree will be more complex though).
For example, if I read your file as TNtuple (commenting the “format” line, of course) I get this:
I use ROOT from MacPorts, so I’ll have to wait for a release to try out the fix. In the meantime, consistently using ‘\n’ as the line terminator seems to have removed the warnings.
Apparently python’s csv writer by default uses ‘\r\n’ independent of platform, but it has an optional keyword argument “lineterminator” that you can set. Setting this to ‘\n’ makes it consistent with the writing of the other strings (and Linux & MacOSX standard), so the warnings don’t show up anymore.
