Cherry-pick add option to save multiple data in HDF5Output

[sguada]

This PR allows the HDF5Output layer to save multiple blobs, as many a bottoms, and during multiple batches, each one numbered consecutively.

This example would save fc7 and fc8 features plus the label. If there are split layers the names would be changed accordingly. Each forward pass will increment the batch_id and append it to the dataset_name

layers {
  name: "hdf5_output"
  type: HDF5_OUTPUT
  bottom: "fc7"
  bottom: "fc8"
  bottom: "label"
  hdf5_output_param { file_name: "temp.hdf5" }
}

[Tgaaly]

@sguada I get an error when trying to output my output layer results. See the error below. Any ideas on how to fix this?

My layer looks like this:

layer {
  name: "output"
  type: "HDF5Output"
  bottom: "ip"
  bottom: "label"
  hdf5_output_param {
    file_name: "/<path>/output_cnn.h5"
  }
}

Error here:

HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 140533133920256:
  #000: ../../../src/H5F.c line 1504 in H5Fcreate(): unable to create file
    major: File accessibilty
    minor: Unable to open file
  #001: ../../../src/H5F.c line 1307 in H5F_open(): unable to truncate a file which is already open
    major: File accessibilty
    minor: Unable to open file
F0630 20:28:44.699677 18387 hdf5_output_layer.cpp:20] Check failed: file_id_ >= 0 (-1 vs. 0) Failed to open HDF5 file/home/.../output_cnn.h5
*** Check failure stack trace: ***
    @     0x7fd06ace7daa  (unknown)
    @     0x7fd06ace7ce4  (unknown)
    @     0x7fd06ace76e6  (unknown)
    @     0x7fd06acea687  (unknown)
    @     0x7fd06b0d9f6b  caffe::HDF5OutputLayer<>::LayerSetUp()
    @     0x7fd06b02da02  caffe::Net<>::Init()
    @     0x7fd06b02f4c2  caffe::Net<>::Net()
    @     0x7fd06b0fd29c  caffe::Solver<>::InitTestNets()
    @     0x7fd06b0fd98b  caffe::Solver<>::Init()
    @     0x7fd06b0fdb56  caffe::Solver<>::Solver()
    @           0x40a360  caffe::GetSolver<>()
    @           0x4061a4  train()
    @           0x4040e1  main
    @     0x7fd06a1f9ec5  (unknown)
    @           0x404cf6  (unknown)
    @              (nil)  (unknown)

[twerdster]

Hi, we are getting the same error as @Tgaaly. Is there a simple fix for this?

[escorciav]

Sorry of we cannot discuss it here.
I have used this cherry-pick without problem. I recommend you to isolate the problem:

1. use this branch as is
2. use prototxt of this branch
3. delete any hdf5 files in your output directory

Is there an intention to integrate it on caffe soon?

[shelhamer]

Closing since the dev branch is deprecated. Please send PRs to master.

[lood339]

dev branch is deprecated. How can we get this pull?
I run:
git fetch origin pull/1392/head:sguada-hdf5output_multiple_input
git checkout sguada-hdf5output_multiple_input

but it has compile errors like this:
/usr/lib64/gcc/x86_64-suse-linux/4.8/include/mmintrin.h: In function ‘__m64 _mm_cvtsi32_si64(int)’:
/usr/lib64/gcc/x86_64-suse-linux/4.8/include/mmintrin.h:61:54: error: can’t convert between vector values of different size
return (__m64) __builtin_ia32_vec_init_v2si (__i, 0);