Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow render HTML with css/js external links #19017

Merged
merged 31 commits into from
Jun 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
581e415
Allow render HTML with css/js external links
lunny Mar 7, 2022
c727f99
Fix bug because of filename escape chars
lunny Mar 7, 2022
b04430d
Fix lint
lunny Mar 7, 2022
d980996
Update docs about new configuration item
lunny Mar 8, 2022
6e012d9
Fix bug of render HTML in sub directory
lunny May 31, 2022
7a4a58f
Add CSP head for displaying iframe in rendering file
lunny May 31, 2022
c3bfd5b
Fix test
lunny Jun 1, 2022
7ec5ac6
merge
lunny Jun 10, 2022
2006683
Apply suggestions from code review
lunny Jun 11, 2022
400308e
Some improvements
lunny Jun 12, 2022
9460386
Merge branch 'main' into lunny/render_iframe
lunny Jun 12, 2022
2c21f6f
merge
lunny Jun 12, 2022
feb290a
some improvement
lunny Jun 12, 2022
2987a7a
Merge branch 'main' into lunny/render_iframe
lunny Jun 12, 2022
4520ea9
revert change in SanitizerDisabled of external renderer
lunny Jun 12, 2022
a69ec31
Add sandbox for iframe and support allow-scripts and allow-same-origin
lunny Jun 13, 2022
05bc9b2
Merge branch 'main' into lunny/render_iframe
lunny Jun 13, 2022
7d0e74e
refactor
wxiaoguang Jun 14, 2022
069b236
Merge pull request #23 from wxiaoguang/lunny/render_iframe
lunny Jun 14, 2022
a929430
fix
wxiaoguang Jun 14, 2022
806e647
fix lint
wxiaoguang Jun 14, 2022
5a2416e
fine tune
wxiaoguang Jun 14, 2022
eb52c83
Merge branch 'main' into lunny/render_iframe
wxiaoguang Jun 14, 2022
d9679ce
use single option RENDER_CONTENT_MODE, use sandbox=allow-scripts
wxiaoguang Jun 14, 2022
a1cac2a
fine tune CSP
wxiaoguang Jun 14, 2022
cc18ebf
Apply suggestions from code review
lunny Jun 15, 2022
65443b2
Merge branch 'main' into lunny/render_iframe
wxiaoguang Jun 15, 2022
6c81597
Merge branch 'main' into lunny/render_iframe
lunny Jun 15, 2022
631fea3
Merge branch 'main' into lunny/render_iframe
lunny Jun 15, 2022
169362d
Merge branch 'main' into lunny/render_iframe
wxiaoguang Jun 15, 2022
c7df5bf
Merge branch 'main' into lunny/render_iframe
lunny Jun 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions custom/conf/app.example.ini
Original file line number Diff line number Diff line change
Expand Up @@ -2181,8 +2181,11 @@ PATH =
;RENDER_COMMAND = "asciidoc --out-file=- -"
;; Don't pass the file on STDIN, pass the filename as argument instead.
;IS_INPUT_FILE = false
; Don't filter html tags and attributes if true
;DISABLE_SANITIZER = false
;; How the content will be rendered.
;; * sanitized: Sanitize the content and render it inside current page, default to only allow a few HTML tags and attributes. Customized sanitizer rules can be defined in [markup.sanitizer.*] .
;; * no-sanitizer: Disable the sanitizer and render the content inside current page. It's **insecure** and may lead to XSS attack if the content contains malicious code.
;; * iframe: Render the content in a separate standalone page and embed it into current page by iframe. The iframe is in sandbox mode with same-origin disabled, and the JS code are safely isolated from parent page.
;RENDER_CONTENT_MODE=sanitized

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
Expand Down
7 changes: 5 additions & 2 deletions docs/content/doc/advanced/config-cheat-sheet.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -1026,13 +1026,16 @@ IS_INPUT_FILE = false
command. Multiple extensions needs a comma as splitter.
- RENDER\_COMMAND: External command to render all matching extensions.
- IS\_INPUT\_FILE: **false** Input is not a standard input but a file param followed `RENDER_COMMAND`.
- DISABLE_SANITIZER: **false** Don't filter html tags and attributes if true. Don't change this to true except you know what that means.
- RENDER_CONTENT_MODE: **sanitized** How the content will be rendered.
- sanitized: Sanitize the content and render it inside current page, default to only allow a few HTML tags and attributes. Customized sanitizer rules can be defined in `[markup.sanitizer.*]`.
- no-sanitizer: Disable the sanitizer and render the content inside current page. It's **insecure** and may lead to XSS attack if the content contains malicious code.
- iframe: Render the content in a separate standalone page and embed it into current page by iframe. The iframe is in sandbox mode with same-origin disabled, and the JS code are safely isolated from parent page.

Two special environment variables are passed to the render command:
- `GITEA_PREFIX_SRC`, which contains the current URL prefix in the `src` path tree. To be used as prefix for links.
- `GITEA_PREFIX_RAW`, which contains the current URL prefix in the `raw` path tree. To be used as prefix for image paths.

If `DISABLE_SANITIZER` is false, Gitea supports customizing the sanitization policy for rendered HTML. The example below will support KaTeX output from pandoc.
If `RENDER_CONTENT_MODE` is `sanitized`, Gitea supports customizing the sanitization policy for rendered HTML. The example below will support KaTeX output from pandoc.

```ini
[markup.sanitizer.TeX]
Expand Down
7 changes: 5 additions & 2 deletions docs/content/doc/advanced/config-cheat-sheet.zh-cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,14 +318,17 @@ IS_INPUT_FILE = false
- FILE_EXTENSIONS: 关联的文档的扩展名,多个扩展名用都好分隔。
- RENDER_COMMAND: 工具的命令行命令及参数。
- IS_INPUT_FILE: 输入方式是最后一个参数为文件路径还是从标准输入读取。
- DISABLE_SANITIZER: **false** 如果为 true 则不过滤 HTML 标签和属性。除非你知道这意味着什么,否则不要设置为 true。
- RENDER_CONTENT_MODE: **sanitized** 内容如何被渲染。
- sanitized: 对内容进行净化并渲染到当前页面中,仅有一部分 HTML 标签和属性是被允许的。
- no-sanitizer: 禁用净化器,把内容渲染到当前页面中。此模式是**不安全**的,如果内容中含有恶意代码,可能会导致 XSS 攻击。
- iframe: 把内容渲染在一个独立的页面中并使用 iframe 嵌入到当前页面中。使用的 iframe 工作在沙箱模式并禁用了同源请求,JS 代码被安全的从父页面中隔离出去。

以下两个环境变量将会被传递给渲染命令:

- `GITEA_PREFIX_SRC`:包含当前的`src`路径的URL前缀,可以被用于链接的前缀。
- `GITEA_PREFIX_RAW`:包含当前的`raw`路径的URL前缀,可以被用于图片的前缀。

如果 `DISABLE_SANITIZER` 为 false,则 Gitea 支持自定义渲染 HTML 的净化策略。以下例子将用 pandoc 支持 KaTeX 输出。
如果 `RENDER_CONTENT_MODE` 为 `sanitized`,则 Gitea 支持自定义渲染 HTML 的净化策略。以下例子将用 pandoc 支持 KaTeX 输出。

```ini
[markup.sanitizer.TeX]
Expand Down
2 changes: 1 addition & 1 deletion modules/csv/csv.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ func CreateReaderAndDetermineDelimiter(ctx *markup.RenderContext, rd io.Reader)
func determineDelimiter(ctx *markup.RenderContext, data []byte) rune {
extension := ".csv"
if ctx != nil {
extension = strings.ToLower(filepath.Ext(ctx.Filename))
extension = strings.ToLower(filepath.Ext(ctx.RelativePath))
}

var delimiter rune
Expand Down
2 changes: 1 addition & 1 deletion modules/csv/csv_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ John Doe john@doe.com This,note,had,a,lot,of,commas,to,test,delimiters`,
}

for n, c := range cases {
delimiter := determineDelimiter(&markup.RenderContext{Filename: c.filename}, []byte(decodeSlashes(t, c.csv)))
delimiter := determineDelimiter(&markup.RenderContext{RelativePath: c.filename}, []byte(decodeSlashes(t, c.csv)))
assert.EqualValues(t, c.expectedDelimiter, delimiter, "case %d: delimiter should be equal, expected '%c' got '%c'", n, c.expectedDelimiter, delimiter)
}
}
Expand Down
8 changes: 0 additions & 8 deletions modules/markup/console/console.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,6 @@ func (Renderer) Name() string {
return MarkupName
}

// NeedPostProcess implements markup.Renderer
func (Renderer) NeedPostProcess() bool { return false }

// Extensions implements markup.Renderer
func (Renderer) Extensions() []string {
return []string{".sh-session"}
Expand All @@ -48,11 +45,6 @@ func (Renderer) SanitizerRules() []setting.MarkupSanitizerRule {
}
}

// SanitizerDisabled disabled sanitize if return true
func (Renderer) SanitizerDisabled() bool {
return false
}

// CanRender implements markup.RendererContentDetector
func (Renderer) CanRender(filename string, input io.Reader) bool {
buf, err := io.ReadAll(input)
Expand Down
8 changes: 0 additions & 8 deletions modules/markup/csv/csv.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,6 @@ func (Renderer) Name() string {
return "csv"
}

// NeedPostProcess implements markup.Renderer
func (Renderer) NeedPostProcess() bool { return false }

// Extensions implements markup.Renderer
func (Renderer) Extensions() []string {
return []string{".csv", ".tsv"}
Expand All @@ -46,11 +43,6 @@ func (Renderer) SanitizerRules() []setting.MarkupSanitizerRule {
}
}

// SanitizerDisabled disabled sanitize if return true
func (Renderer) SanitizerDisabled() bool {
return false
}

func writeField(w io.Writer, element, class, field string) error {
if _, err := io.WriteString(w, "<"); err != nil {
return err
Expand Down
12 changes: 11 additions & 1 deletion modules/markup/external/external.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ type Renderer struct {
*setting.MarkupRenderer
}

var (
_ markup.PostProcessRenderer = (*Renderer)(nil)
_ markup.ExternalRenderer = (*Renderer)(nil)
)

// Name returns the external tool name
func (p *Renderer) Name() string {
return p.MarkupName
Expand All @@ -56,7 +61,12 @@ func (p *Renderer) SanitizerRules() []setting.MarkupSanitizerRule {

// SanitizerDisabled disabled sanitize if return true
func (p *Renderer) SanitizerDisabled() bool {
return p.DisableSanitizer
return p.RenderContentMode == setting.RenderContentModeNoSanitizer || p.RenderContentMode == setting.RenderContentModeIframe
}

// DisplayInIFrame represents whether render the content with an iframe
func (p *Renderer) DisplayInIFrame() bool {
return p.RenderContentMode == setting.RenderContentModeIframe
}

func envMark(envName string) string {
Expand Down
26 changes: 13 additions & 13 deletions modules/markup/html_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ func TestRender_Commits(t *testing.T) {
setting.AppURL = TestAppURL
test := func(input, expected string) {
buffer, err := RenderString(&RenderContext{
Ctx: git.DefaultContext,
Filename: ".md",
URLPrefix: TestRepoURL,
Metas: localMetas,
Ctx: git.DefaultContext,
RelativePath: ".md",
URLPrefix: TestRepoURL,
Metas: localMetas,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
Expand Down Expand Up @@ -80,9 +80,9 @@ func TestRender_CrossReferences(t *testing.T) {

test := func(input, expected string) {
buffer, err := RenderString(&RenderContext{
Filename: "a.md",
URLPrefix: setting.AppSubURL,
Metas: localMetas,
RelativePath: "a.md",
URLPrefix: setting.AppSubURL,
Metas: localMetas,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
Expand Down Expand Up @@ -124,8 +124,8 @@ func TestRender_links(t *testing.T) {

test := func(input, expected string) {
buffer, err := RenderString(&RenderContext{
Filename: "a.md",
URLPrefix: TestRepoURL,
RelativePath: "a.md",
URLPrefix: TestRepoURL,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
Expand Down Expand Up @@ -223,8 +223,8 @@ func TestRender_email(t *testing.T) {

test := func(input, expected string) {
res, err := RenderString(&RenderContext{
Filename: "a.md",
URLPrefix: TestRepoURL,
RelativePath: "a.md",
URLPrefix: TestRepoURL,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(res))
Expand Down Expand Up @@ -281,8 +281,8 @@ func TestRender_emoji(t *testing.T) {
test := func(input, expected string) {
expected = strings.ReplaceAll(expected, "&", "&amp;")
buffer, err := RenderString(&RenderContext{
Filename: "a.md",
URLPrefix: TestRepoURL,
RelativePath: "a.md",
URLPrefix: TestRepoURL,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
Expand Down
9 changes: 3 additions & 6 deletions modules/markup/markdown/markdown.go
Original file line number Diff line number Diff line change
Expand Up @@ -205,12 +205,14 @@ func init() {
// Renderer implements markup.Renderer
type Renderer struct{}

var _ markup.PostProcessRenderer = (*Renderer)(nil)

// Name implements markup.Renderer
func (Renderer) Name() string {
return MarkupName
}

// NeedPostProcess implements markup.Renderer
// NeedPostProcess implements markup.PostProcessRenderer
func (Renderer) NeedPostProcess() bool { return true }

// Extensions implements markup.Renderer
Expand All @@ -223,11 +225,6 @@ func (Renderer) SanitizerRules() []setting.MarkupSanitizerRule {
return []setting.MarkupSanitizerRule{}
}

// SanitizerDisabled disabled sanitize if return true
func (Renderer) SanitizerDisabled() bool {
return false
}

// Render implements markup.Renderer
func (Renderer) Render(ctx *markup.RenderContext, input io.Reader, output io.Writer) error {
return render(ctx, input, output)
Expand Down
9 changes: 3 additions & 6 deletions modules/markup/orgmode/orgmode.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,14 @@ func init() {
// Renderer implements markup.Renderer for orgmode
type Renderer struct{}

var _ markup.PostProcessRenderer = (*Renderer)(nil)

// Name implements markup.Renderer
func (Renderer) Name() string {
return "orgmode"
}

// NeedPostProcess implements markup.Renderer
// NeedPostProcess implements markup.PostProcessRenderer
func (Renderer) NeedPostProcess() bool { return true }

// Extensions implements markup.Renderer
Expand All @@ -47,11 +49,6 @@ func (Renderer) SanitizerRules() []setting.MarkupSanitizerRule {
return []setting.MarkupSanitizerRule{}
}

// SanitizerDisabled disabled sanitize if return true
func (Renderer) SanitizerDisabled() bool {
return false
}

// Render renders orgmode rawbytes to HTML
func Render(ctx *markup.RenderContext, input io.Reader, output io.Writer) error {
htmlWriter := org.NewHTMLWriter()
Expand Down
81 changes: 64 additions & 17 deletions modules/markup/renderer.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"errors"
"fmt"
"io"
"net/url"
"path/filepath"
"strings"
"sync"
Expand Down Expand Up @@ -43,17 +44,18 @@ type Header struct {

// RenderContext represents a render context
type RenderContext struct {
Ctx context.Context
Filename string
Type string
IsWiki bool
URLPrefix string
Metas map[string]string
DefaultLink string
GitRepo *git.Repository
ShaExistCache map[string]bool
cancelFn func()
TableOfContents []Header
Ctx context.Context
RelativePath string // relative path from tree root of the branch
Type string
IsWiki bool
URLPrefix string
Metas map[string]string
DefaultLink string
GitRepo *git.Repository
ShaExistCache map[string]bool
cancelFn func()
TableOfContents []Header
InStandalonePage bool // used by external render. the router "/org/repo/render/..." will output the rendered content in a standalone page
}

// Cancel runs any cleanup functions that have been registered for this Ctx
Expand Down Expand Up @@ -88,12 +90,24 @@ func (ctx *RenderContext) AddCancel(fn func()) {
type Renderer interface {
Name() string // markup format name
Extensions() []string
NeedPostProcess() bool
SanitizerRules() []setting.MarkupSanitizerRule
SanitizerDisabled() bool
Render(ctx *RenderContext, input io.Reader, output io.Writer) error
}

// PostProcessRenderer defines an interface for renderers who need post process
type PostProcessRenderer interface {
NeedPostProcess() bool
}

// PostProcessRenderer defines an interface for external renderers
type ExternalRenderer interface {
// SanitizerDisabled disabled sanitize if return true
SanitizerDisabled() bool

// DisplayInIFrame represents whether render the content with an iframe
DisplayInIFrame() bool
}

// RendererContentDetector detects if the content can be rendered
// by specified renderer
type RendererContentDetector interface {
Expand Down Expand Up @@ -142,7 +156,7 @@ func DetectRendererType(filename string, input io.Reader) string {
func Render(ctx *RenderContext, input io.Reader, output io.Writer) error {
if ctx.Type != "" {
return renderByType(ctx, input, output)
} else if ctx.Filename != "" {
} else if ctx.RelativePath != "" {
return renderFile(ctx, input, output)
}
return errors.New("Render options both filename and type missing")
Expand All @@ -163,6 +177,27 @@ type nopCloser struct {

func (nopCloser) Close() error { return nil }

func renderIFrame(ctx *RenderContext, output io.Writer) error {
// set height="0" ahead, otherwise the scrollHeight would be max(150, realHeight)
// at the moment, only "allow-scripts" is allowed for sandbox mode.
// "allow-same-origin" should never be used, it leads to XSS attack, and it makes the JS in iframe can access parent window's config and CSRF token
// TODO: when using dark theme, if the rendered content doesn't have proper style, the default text color is black, which is not easy to read
_, err := io.WriteString(output, fmt.Sprintf(`
<iframe src="%s/%s/%s/render/%s/%s"
name="giteaExternalRender"
onload="this.height=giteaExternalRender.document.documentElement.scrollHeight"
width="100%%" height="0" scrolling="no" frameborder="0" style="overflow: hidden"
sandbox="allow-scripts"
></iframe>`,
setting.AppSubURL,
url.PathEscape(ctx.Metas["user"]),
url.PathEscape(ctx.Metas["repo"]),
ctx.Metas["BranchNameSubURL"],
url.PathEscape(ctx.RelativePath),
))
return err
}

func render(ctx *RenderContext, renderer Renderer, input io.Reader, output io.Writer) error {
var wg sync.WaitGroup
var err error
Expand All @@ -175,7 +210,12 @@ func render(ctx *RenderContext, renderer Renderer, input io.Reader, output io.Wr
var pr2 io.ReadCloser
var pw2 io.WriteCloser

if !renderer.SanitizerDisabled() {
var sanitizerDisabled bool
if r, ok := renderer.(ExternalRenderer); ok {
sanitizerDisabled = r.SanitizerDisabled()
}

if !sanitizerDisabled {
pr2, pw2 = io.Pipe()
defer func() {
_ = pr2.Close()
Expand All @@ -194,7 +234,7 @@ func render(ctx *RenderContext, renderer Renderer, input io.Reader, output io.Wr

wg.Add(1)
go func() {
if renderer.NeedPostProcess() {
if r, ok := renderer.(PostProcessRenderer); ok && r.NeedPostProcess() {
err = PostProcess(ctx, pr, pw2)
} else {
_, err = io.Copy(pw2, pr)
Expand Down Expand Up @@ -239,8 +279,15 @@ func (err ErrUnsupportedRenderExtension) Error() string {
}

func renderFile(ctx *RenderContext, input io.Reader, output io.Writer) error {
extension := strings.ToLower(filepath.Ext(ctx.Filename))
extension := strings.ToLower(filepath.Ext(ctx.RelativePath))
if renderer, ok := extRenderers[extension]; ok {
if r, ok := renderer.(ExternalRenderer); ok && r.DisplayInIFrame() {
if !ctx.InStandalonePage {
// for an external render, it could only output its content in a standalone page
// otherwise, a <iframe> should be outputted to embed the external rendered page
return renderIFrame(ctx, output)
}
}
return render(ctx, renderer, input, output)
}
return ErrUnsupportedRenderExtension{extension}
Expand Down
Loading