Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compile kernels and fix build #17

Merged
merged 2 commits into from
Apr 11, 2024
Merged

Compile kernels and fix build #17

merged 2 commits into from
Apr 11, 2024

Conversation

njhill
Copy link
Contributor

@njhill njhill commented Apr 11, 2024 •
edited by joerunde
Loading

These Dockerfile changes:

  • Update the release stage to work with the recently refactored requirements-common.txt / requirements-cuda.txt split
  • Fixup the kernel compilation in the build stage to correctly pick up cuda
  • Install the kernels from this docker build rather than pulling a precompiled wheel. We can swap that back once a new wheel is available with the correct pytorch version + updated interfaces

@njhill njhill force-pushed the kernel-compat-hack branch 2 times, most recently from b2dc7b7 to 318276c Compare April 11, 2024 17:18
@njhill
Temp hack to work with prebuilt 0.4.0-post kernels
ba27b4d 
Signed-off-by: Nick Hill <nickhill@us.ibm.com>
@njhill njhill force-pushed the kernel-compat-hack branch from 318276c to ba27b4d Compare April 11, 2024 19:48
@joerunde
🐛 compile it ourselves
e6ec7db 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
@joerunde joerunde changed the title [DO NOT MERGE] Temp hack to work with prebuilt 0.4.0-post kernels Compile kernels and fix build Apr 11, 2024
@joerunde joerunde marked this pull request as ready for review April 11, 2024 23:41
njhill
njhill commented Apr 11, 2024
View reviewed changes
Copy link
Contributor Author

@njhill njhill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @joerunde

@njhill njhill merged commit 15076fa into main Apr 11, 2024
4 checks passed
@njhill njhill deleted the kernel-compat-hack branch April 11, 2024 23:50
prashantgupta24 added a commit that referenced this pull request Apr 16, 2024
@prashantgupta24
Squashed commit of the following:
3a4c4ed 
commit 1c75dd5
Author: Prashant Gupta <prashantgupta@us.ibm.com>
Date:   Thu Apr 11 11:07:45 2024 -0700

    ✨ add console scripts to setup.py

    Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>

commit 4d1686d
Author: Prashant Gupta <prashantgupta@us.ibm.com>
Date:   Tue Apr 9 15:28:37 2024 -0700

    🎨 format

    Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>

commit c142e08
Author: Prashant Gupta <prashantgupta@us.ibm.com>
Date:   Tue Apr 9 15:26:15 2024 -0700

    ✅ add test cli file

    Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>

commit 0cb571d
Author: Prashant Gupta <prashantgupta@us.ibm.com>
Date:   Tue Apr 9 15:25:44 2024 -0700

    🎨 fix typo

    Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>

commit 9ff66fe
Author: Prashant Gupta <prashantgupta@us.ibm.com>
Date:   Tue Apr 9 11:45:01 2024 -0700

    🎨 fmt

    Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>

commit 7f3d0b6
Author: Prashant Gupta <prashantgupta@us.ibm.com>
Date:   Tue Apr 9 11:13:31 2024 -0700

    ✅ add test file

    Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>

commit 6691fd6
Author: Prashant Gupta <prashantgupta@us.ibm.com>
Date:   Fri Apr 5 11:57:52 2024 -0700

    ✨ add tgis-cli tools

    Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>

commit 15076fa
Author: Nick Hill <nickhill@us.ibm.com>
Date:   Fri Apr 12 00:50:25 2024 +0100

    Compile kernels and fix build (#17)

    These Dockerfile changes:
    - Update the release stage to work with the recently refactored
    `requirements-common.txt` / `requirements-cuda.txt` split
    - Fixup the kernel compilation in the `build` stage to correctly pick up
    cuda
    - Install the kernels from this docker build rather than pulling a
    precompiled wheel. We can swap that back once a new wheel is available
    with the correct pytorch version + updated interfaces

    ---------

    Signed-off-by: Nick Hill <nickhill@us.ibm.com>
    Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
    Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>

Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
prashantgupta24 added a commit that referenced this pull request Apr 17, 2024
@prashantgupta24
Squashed commit of the following:
38dbc92 
commit 82d2261
Author: Prashant Gupta <prashantgupta@us.ibm.com>
Date:   Wed Apr 17 15:44:35 2024 -0700

    ♻️ update dockerfile.ubi with vllm wheel installation

    Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>

commit 15076fa
Author: Nick Hill <nickhill@us.ibm.com>
Date:   Fri Apr 12 00:50:25 2024 +0100

    Compile kernels and fix build (#17)

    These Dockerfile changes:
    - Update the release stage to work with the recently refactored
    `requirements-common.txt` / `requirements-cuda.txt` split
    - Fixup the kernel compilation in the `build` stage to correctly pick up
    cuda
    - Install the kernels from this docker build rather than pulling a
    precompiled wheel. We can swap that back once a new wheel is available
    with the correct pytorch version + updated interfaces

    ---------

    Signed-off-by: Nick Hill <nickhill@us.ibm.com>
    Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
    Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>

Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
tdoublep added a commit that referenced this pull request Jan 20, 2025
@tdoublep @nikolaospapandreou @tjohnson31415
Initial support for AIU (#17)
44681e4 
Initial support for AIU in vLLM.

What is currently supported/tested:
- Single AIU
- Model: llama-7b-chat
- Offline inference (batch size 1) 
- Online inference (with `max-num-seq=1`)

---------

Signed-off-by: Nikolaos Papandreou <npo@zurich.ibm.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nikolaos Papandreou <npo@zurich.ibm.com>
Co-authored-by: TRAVIS JOHNSON <tsjohnso@us.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@njhill @joerunde