Python log parser improvements #1547
Conversation
|
@sguada have a look and please confirm that I addressed all of your comments |
|
Hi drdan14, Do you know how to generate .log file for your log parser in Caffe? I'm stuck here without clue. I mean usually we run executable as "./build/tools/caffe train --solver=/path/to/my_solver.prototxt", do I need to add a flag to create the log file? Thank you very much for your help!! |
|
It's in Please ask these sorts of questions in the caffe users Google group at https://groups.google.com/forum/m/#!forum/caffe-users |
|
Thanks a lot! |
|
Hi drdan14, I used your python parser today and I think I found a bug. In parse_log(path_to_log) function, when you return train_dict_list and test_dict_list, it actually misses the last value stored in train_row and test_row. This will lead to the result that we lost last testing and training result. Thanks! |
|
@jamesguoxin you're quite right, that is a bug. It's a function of the fact that I didn't notice this because I usually have so many lines in my log that I don't care about the last one. But I'll ponder a fix. |
|
@jamesguoxin fix pushed |
|
Great! |
| re_output_loss = re.compile('output #\d+: loss = ([\.\d]+)') | ||
| re_lr = re.compile('lr = ([\.\d]+)') | ||
| regex_iteration = re.compile('Iteration (\d+)') | ||
| regex_train_output = re.compile('Train net output #(\d+): (\S+) = ([\.\d]+)') |
There was a problem hiding this comment.
Maybe it could be
regex_train_output = re.compile('Train net output #(\d+): (\S+) = ([0-9]*.?[0-9]+([eE][-+]?[0-9]+)?.)')
in order to deal with scientific notations when the loss is closer to zero.
There was a problem hiding this comment.
Update pushed, thanks for the suggestion, @zhfe99
* A sample code was added. * `slice_dim` and `slice_point` attributes were explained.
[docs] brief explanation of SLICE layer's attributes
Correct 'epochs' to 'iterations'
Next: release candidater
fix Imagenet example path
set the right rpath for tools and examples respectively thanks for the report @mees!
[build] fix dynamic linking of tools
… was overwritten with symlink created at build time and installed with install(DIRECTORY ...)
change resorce to resource
2.7.0 isn't really necessary - 2.3.0 is sufficient. This is the version available on Ubuntu 14.04 via apt-get, and seems to be a reasonable lowest common denominator in general. http://pillow.readthedocs.org/installation.html#old-versions
CUDNN_CONVOLUTION_FWD_PREFER_FASTEST requires a lot of GPU memory, which may not always be available. Add a fallback path that uses CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_GEMM when the allocation fails.
Fallback to different cuDNN algorithm when under memory pressure; fix BVLC#2197
Downgrade Pillow pip requirement
Remove scikit-learn dependency -- the need is noted in the relevant example
Previously, CUDA_VERSION would appear to be < 7 if there was no CUDA installed, and that would generate the wrong C++ flags for compiling on recent OSX versions. Instead, skip the CUDA version check if CPU_ONLY is set. This change only affects CPU_ONLY installations.
Fix Travis: no need to remove libm in new Miniconda
[build] check if CPU_ONLY is set when determining CUDA version
[example] change `import Image` for forward compatibility
[docs] fix brew command for OS X install
Output CSV format is unchanged Interface is changed; fields are no longer passed around, since the field order is self-contained within the OrderedDict Python 2.7 or higher is required for OrderedDict
E.g., if the top is called "accuracy", the output column name will also be "accuracy" We no longer make any assumptions about the name of the top
Only assumption for top name now is that it contains no white space; any other character is OK
This fixes a bug where the last row was not getting pushed because, after the row had been built, output_match was never true again
Otherwise the values are strings, which is silly, since they're all numbers
4da07e8 to
f74c649
Compare
|
Replaced by master-based version at #2350 |
Improvements to python log parser introduced in #1384
Highlights:
OrderedDictobjects instead ofdictobjects, which obviates the need to pass around a tuple with the column orders.loss, then the corresponding column header will also beloss; we no longer rename it to, e.g.,TrainingLossorTestLoss.You can use Matlab code from this gist to verify that your results are the same before and after the changes introduced in this pull request. That code assumes that your
topnames areaccuracyandloss, but you can modify the code if that's not true.One caveat: we expect that the output layer names consist only of alphanumeric characters and the underscore, and are therefore captured with the\wtoken in the Python regular expression syntax. If other characters are allowed in layer names, we'll have to change the\wtoken to something else.Update: any top name without white space is OK.