Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to take parameters from bottoms #2166

Closed
wants to merge 1 commit into from

Conversation

longjon
Copy link
Contributor

@longjon longjon commented Mar 20, 2015

After #2165.

This PR is the counterpart to #2079; together they propose a (temporary?) solution to #1474.

An option is added to LayerParameter called param_bottoms which gives a number of parameter blobs to take from (additional) bottoms. This allows one to break the artificial division between bottoms and parameters. It's implemented in the Layer base class, so one can use existing layers unmodified in this way.

One issue with this: while Forward_*, Backward_*, and LayerSetUp calls are wrapped by calls from Layer, Reshape is not, and is directly passed bottom. Thus we cannot perform any option-based finagling of bottoms without changing this call. The somewhat hacky solution here is to add an extra nullary wrapper method, ReshapeOnly, that lets the layer decide for itself what its bottoms and tops are. A better (but more aggressive) solution might be to go ahead and change all the method types to reflect the fact that bottom and top aren't actually allowed to change between set up and forward/backward/reshape calls.

This PR provides an immediate, but perhaps unsatisfying solution to #1474. The issue of extra semantic/naming information between bottoms and params is not addressed (but maybe it's not as important given a higher-level way to write nets, e.g., #2086). Neither is the issue of computing param sizes, which is a little less automatic when using this option. (Further, size checks don't exist for params-from-bottoms).

We could merge this PR, or something like it, as a temporary way of making layers more flexible until a final solution is built. Or we could keep this as a PR until the param/bottom distinction is cleanly removed, and layer code is updated accordingly.

Finally, a usage note: while you can write any number for param_bottoms, the only values that will work are zero or the number of parameter blobs expected by the layer (which is not otherwise specified). It's also possible to use a non-zero param_bottoms while also loading params, but that's not likely to work and should probably just be disabled.

@sguada
Copy link
Contributor

sguada commented Mar 20, 2015

@longjon why not change the signature of layers to (params, bottoms, tops) Otherwise it is going to be a bit confusing to keep the order of param and bottoms aligned.

Another option could be to use a map<string name, Blob bottom> that allows to refer to params and bottoms by name.

@shelhamer
Copy link
Member

Closing as #2219 was closed.

@shelhamer shelhamer closed this Apr 13, 2017
@itkinai
Copy link

itkinai commented Jun 28, 2017

@sguada can you explain how to set wieghts in InnerProduct as parameter layer with prototxt, or its works only in code with map<string name, Blob bottom> ?

@yuanyc06
Copy link

yuanyc06 commented Sep 4, 2017

@longjon @sguada would you mind give an example about how to set the weight of a layer (e.g. fc) as the bottom of another layer via the Parameter Layer? Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants