🌐 [i18n-KO] Translated `llm_optims.md` to Korean #32325

yijun-lee · 2024-07-30T13:32:20Z

What does this PR do?

Translated the llm_optims.md file of the documentation to Korean.
Thank you in advance for your review.

Part of #20179

Before reviewing

Check for missing / redundant translations (번역 누락/중복 검사)
Grammar Check (맞춤법 검사)
Review or Add new terms to glossary (용어 확인 및 추가)
Check Inline TOC (e.g. [[lowercased-header]])
Check live-preview for gotchas (live-preview로 정상작동 확인)

Who can review? (Initial)

@jun048098, @yijun-lee, @mreraser, @shinhyunji36, @heuristicwave

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review? (Final)

@stevhliu May you please review this PR?

docs/source/ko/llm_optims.md

jun048098 · 2024-08-06T08:49:09Z

docs/source/ko/llm_optims.md

+
+이를 최적화하기 위해, 이전 키(key)와 값(value)을 재계산하지 않고 저장하는 kv-cache를 사용할 수 있습니다. 그러나 kv-cache는 각 생성 단계에서 증가하며 동적이기 때문에 PyTorch 코드를 빠르고 최적화된 커널로 통합하는 강력한 최적화 도구인 [torch.compile](./perf_torch_compile)을 사용하는 데 제약이 있습니다.
+
+*정적 kv-cache*는 최대 값을 미리 할당하여 이 문제를 해결하여 torch.compile과 결합할 수 있게 합니다. 이를 통해 최대 4배의 속도 향상이 가능합니다.


Suggested change

*정적 kv-cache*는 최대 값을 미리 할당하여 이 문제를 해결하여 torch.compile과 결합할 수 있게 합니다. 이를 통해 최대 4배의 속도 향상이 가능합니다.

*정적 kv-cache*는 최대 값을 미리 할당하여 이 문제를 해결하여 torch.compile과 결합할 수 있게 합니다. 이를 통해 최대 4배의 속도 향상이 가능합니다. 속도 향상은 모델 크기(모델이 클수록 속도 향상이 더 작음) 및 하드웨어에 따라 달라질 수 있습니다.

원문 line29 Your speed up may vary depending on the model size (larger models have a smaller speed up) and hardware. 문장의 해석이 누락된 것 같습니다.

jun048098 · 2024-08-06T11:03:32Z

원문의 내용이 새로운 commit으로 변경되었습니다.

docs/source/ko/llm_optims.md

shinhyunji36 · 2024-08-06T11:06:39Z

docs/source/ko/llm_optims.md

+> [!TIP]
+> 보다 심층적인 설명을 원한다면, [Assisted Generation: a new direction toward low-latency text generation](https://hf.co/blog/assisted-generation) 블로그 게시물을 확인하십시오!
+
+자기 회귀의 또 다른 문제는 각 입력 토큰에 대해 순전파 중에 모델 가중치를 매번 로드해야 한다는 점입니다. 이는 수십억 개의 매개변수를 가진 LLM에는 느리고 번거롭습니다. 추정 디코딩은 더 작고 빠른 보조 모델을 사용하여 후보 토큰을 생성하고, 이를 큰 LLM이 단일 순전파에서 검증하여 이 속도 저하를 완화합니다. 검증된 토큰이 정확하다면, LLM은 본래 자체적으로 생성하는 것처럼 토큰을 얻을 수 있습니다. 전방 패스가 동일한 출력을 보장하기 때문에 정확도 저하가 없습니다.


Suggested change

자기 회귀의 또 다른 문제는 각 입력 토큰에 대해 순전파 중에 모델 가중치를 매번 로드해야 한다는 점입니다. 이는 수십억 개의 매개변수를 가진 LLM에는 느리고 번거롭습니다. 추정 디코딩은 더 작고 빠른 보조 모델을 사용하여 후보 토큰을 생성하고, 이를 큰 LLM이 단일 순전파에서 검증하여 이 속도 저하를 완화합니다. 검증된 토큰이 정확하다면, LLM은 본래 자체적으로 생성하는 것처럼 토큰을 얻을 수 있습니다. 전방 패스가 동일한 출력을 보장하기 때문에 정확도 저하가 없습니다.

자기 회귀의 또 다른 문제는 각 입력 토큰에 대해 순전파 중에 모델 가중치를 매번 로드해야 한다는 점입니다. 이는 수십억 개의 매개변수를 가진 LLM에는 느리고 번거롭습니다. 추정 디코딩은 더 작고 빠른 보조 모델을 사용하여 후보 토큰을 생성하고, 이를 큰 LLM이 단일 순전파에서 검증하여 이 속도 저하를 완화합니다. 검증된 토큰이 정확하다면, LLM은 이를 추가적인 연산 없이 획득하게 됩니다. 검증 순전파를 통해 LLM이 자체적으로 생성한 것과 동일한 출력이 생성되기 때문에 정확도가 저하되지 않습니다.

자체적으로 생성하는 것과 동일한 토큰을 연산없이 얻는다는 의미를 좀 더 강조해서 수정해봤습니다.

shinhyunji36 · 2024-08-06T11:09:50Z

긴 문서인데, 번역하느라 고생 많으셨습니다! 😄 @yijun-lee

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

Co-authored-by: HyunJi Shin <74661937+shinhyunji36@users.noreply.github.com>

HuggingFaceDocBuilderDev · 2024-08-06T16:17:51Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

stevhliu

Thanks for translating this! 🤗

docs/source/ko/llm_optims.md

stevhliu

Fantastic, LGTM! Thanks for adding those additional sections!

@mreraser @shinhyunji36 would you mind reviewing these new sections?

mreraser · 2024-08-28T16:40:52Z

Fantastic, LGTM! Thanks for adding those additional sections!

@mreraser @shinhyunji36 would you mind reviewing these new sections?

Hello @stevhliu! Yes, I will review the newly added sections. Thank you 😄

docs/source/ko/llm_optims.md

fix: resolve suggestions Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

mreraser

LGTM 😄

* docs: ko: llm_optims.md * feat: nmt draft * fix toc title * fix: manual edits * Update docs/source/ko/llm_optims.md Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: HyunJi Shin <74661937+shinhyunji36@users.noreply.github.com> * Update docs/source/ko/llm_optims.md Co-authored-by: HyunJi Shin <74661937+shinhyunji36@users.noreply.github.com> * Update llm_optims.md * fix: resolve suggestions * fix: resolve suggestions * Apply suggestions from code review fix: resolve suggestions Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> --------- Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com> Co-authored-by: HyunJi Shin <74661937+shinhyunji36@users.noreply.github.com>

yijun-lee and others added 5 commits July 19, 2024 10:54

docs: ko: llm_optims.md

19f9fd3

feat: nmt draft

a149982

fix toc title

22e64e4

Merge branch 'huggingface:main' into ko-llm_optims.md

3cba215

fix: manual edits

64f068a

yijun-lee marked this pull request as ready for review August 4, 2024 07:47

yijun-lee marked this pull request as draft August 4, 2024 07:47

Merge branch 'main' into ko-llm_optims.md

668f10f

mreraser reviewed Aug 4, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 4, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 4, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 4, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 4, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 4, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 5, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 5, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 6, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

jun048098 reviewed Aug 6, 2024

View reviewed changes

shinhyunji36 suggested changes Aug 6, 2024

View reviewed changes

yijun-lee and others added 9 commits August 6, 2024 20:26

Update docs/source/ko/llm_optims.md

005991b

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

Update docs/source/ko/llm_optims.md

5be498d

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

Update docs/source/ko/llm_optims.md

d64bc44

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

Update docs/source/ko/llm_optims.md

6f78352

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

Update docs/source/ko/llm_optims.md

a9b2232

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

Update docs/source/ko/llm_optims.md

482ac16

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

Update docs/source/ko/llm_optims.md

aa07933

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

Update docs/source/ko/llm_optims.md

4660adf

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

Update docs/source/ko/llm_optims.md

7b6999c

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

yijun-lee and others added 3 commits August 6, 2024 20:30

Update docs/source/ko/llm_optims.md

745c5eb

Co-authored-by: HyunJi Shin <74661937+shinhyunji36@users.noreply.github.com>

Update docs/source/ko/llm_optims.md

cc138dc

Co-authored-by: HyunJi Shin <74661937+shinhyunji36@users.noreply.github.com>

Update llm_optims.md

7ba0566

yijun-lee marked this pull request as ready for review August 6, 2024 11:43

stevhliu reviewed Aug 6, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

yijun-lee and others added 3 commits August 28, 2024 11:14

Merge branch 'huggingface:main' into ko-llm_optims.md

3a5d9db

fix: resolve suggestions

989f7f5

fix: resolve suggestions

aacb230

yijun-lee requested a review from stevhliu August 28, 2024 05:20

stevhliu reviewed Aug 28, 2024

View reviewed changes

mreraser reviewed Aug 28, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 28, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

mreraser reviewed Aug 28, 2024

View reviewed changes

docs/source/ko/llm_optims.md Outdated Show resolved Hide resolved

Apply suggestions from code review

649bd15

fix: resolve suggestions Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>

yijun-lee requested a review from mreraser August 29, 2024 00:14

mreraser approved these changes Aug 29, 2024

View reviewed changes

yijun-lee requested a review from stevhliu August 29, 2024 01:59

stevhliu approved these changes Aug 30, 2024

View reviewed changes

stevhliu merged commit db70426 into huggingface:main Aug 30, 2024
8 checks passed

yijun-lee deleted the ko-llm_optims.md branch September 27, 2024 06:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🌐 [i18n-KO] Translated `llm_optims.md` to Korean #32325

🌐 [i18n-KO] Translated `llm_optims.md` to Korean #32325

yijun-lee commented Jul 30, 2024 •

edited

Loading

jun048098 Aug 6, 2024

jun048098 commented Aug 6, 2024

shinhyunji36 Aug 6, 2024

shinhyunji36 Aug 6, 2024

shinhyunji36 commented Aug 6, 2024

HuggingFaceDocBuilderDev commented Aug 6, 2024

stevhliu left a comment

stevhliu left a comment

mreraser commented Aug 28, 2024

mreraser left a comment


		이를 최적화하기 위해, 이전 키(key)와 값(value)을 재계산하지 않고 저장하는 kv-cache를 사용할 수 있습니다. 그러나 kv-cache는 각 생성 단계에서 증가하며 동적이기 때문에 PyTorch 코드를 빠르고 최적화된 커널로 통합하는 강력한 최적화 도구인 [torch.compile](./perf_torch_compile)을 사용하는 데 제약이 있습니다.

		정적 kv-cache는 최대 값을 미리 할당하여 이 문제를 해결하여 torch.compile과 결합할 수 있게 합니다. 이를 통해 최대 4배의 속도 향상이 가능합니다.

🌐 [i18n-KO] Translated llm_optims.md to Korean #32325

🌐 [i18n-KO] Translated llm_optims.md to Korean #32325

Conversation

yijun-lee commented Jul 30, 2024 • edited Loading

What does this PR do?

Before reviewing

Who can review? (Initial)

Before submitting

Who can review? (Final)

jun048098 Aug 6, 2024

Choose a reason for hiding this comment

jun048098 commented Aug 6, 2024

shinhyunji36 Aug 6, 2024

Choose a reason for hiding this comment

shinhyunji36 Aug 6, 2024

Choose a reason for hiding this comment

shinhyunji36 commented Aug 6, 2024

HuggingFaceDocBuilderDev commented Aug 6, 2024

stevhliu left a comment

Choose a reason for hiding this comment

stevhliu left a comment

Choose a reason for hiding this comment

mreraser commented Aug 28, 2024

mreraser left a comment

Choose a reason for hiding this comment

🌐 [i18n-KO] Translated `llm_optims.md` to Korean #32325

🌐 [i18n-KO] Translated `llm_optims.md` to Korean #32325

yijun-lee commented Jul 30, 2024 •

edited

Loading