-
-
Notifications
You must be signed in to change notification settings - Fork 609
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add grouped convolution #1531
Add grouped convolution #1531
Conversation
Do the grouped convolutions run on the GPU as well? |
This PR adds the necessary changes to the dims objects that we can use to pass the groups to CuDNN. |
Seems like we are good to merge FluxML/NNlib.jl#289 |
Needs some tests. Manifest can be removed |
Going by how many releases of NNlib were needed to get the Embedding layer merged into Flux, it seemed to me that development could have been a lot faster and releases a lot safer had a Manifest been used. Releases aren't free since they affect unsuspecting users the most. |
development with a Manifest would have gone at exactly the same speed. Something that did slow down a bit development is the separation of NNlib and NNlibCUDA. Releases are free as long as they are non-breaking, they don't affect users that don't use the new features |
Anyways, Manifest should not be discussed here, this PR is meant to add groups support not to reinstate the manifest, let's not derail the PR |
Agreed, that I don't want to derail the PR but:
A manifest would have shown the issues with every CI run, without having to make disruptive releases. Not to mention the time it takes to make them, and the buggy code that is pushed out to users
Features are features, new or old. Sure bug fixes will come in faster for new ones than the old ones but if it is known that something isn't quite right, then why release?
The issue is the existence of the separation. The release schedule would have been the same even if in the same repo, and with a Manifest, the gap of where packages live is made significantly smaller |
but then you do. I disagree with everything you said, but again, this is not the right place to have that discussion |
a news entry would be good |
bors r+ |
Build succeeded: |
[Diff since v0.12.5](FluxML/Flux.jl@v0.12.5...v0.12.6) **Merged pull requests:** - Add grouped convolution (FluxML#1531) (@DhairyaLGandhi) - fix deprecations of zeros (FluxML#1670) (@DhairyaLGandhi) - Add GPU activation tests for grouped conv (FluxML#1672) (@DhairyaLGandhi)
Add support for grouped convolutions. Needed for a bunch of things. Depends on an NNlib branch where the support is present in the Dims Helpers, but needs to be chained through to the conv kernel. Although we now can generate the correct shaped kernels and filters here.