Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow render HTML with css/js external links #19017

Merged
merged 31 commits into from
Jun 16, 2022
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
581e415
Allow render HTML with css/js external links
lunny Mar 7, 2022
c727f99
Fix bug because of filename escape chars
lunny Mar 7, 2022
b04430d
Fix lint
lunny Mar 7, 2022
d980996
Update docs about new configuration item
lunny Mar 8, 2022
6e012d9
Fix bug of render HTML in sub directory
lunny May 31, 2022
7a4a58f
Add CSP head for displaying iframe in rendering file
lunny May 31, 2022
c3bfd5b
Fix test
lunny Jun 1, 2022
7ec5ac6
merge
lunny Jun 10, 2022
2006683
Apply suggestions from code review
lunny Jun 11, 2022
400308e
Some improvements
lunny Jun 12, 2022
9460386
Merge branch 'main' into lunny/render_iframe
lunny Jun 12, 2022
2c21f6f
merge
lunny Jun 12, 2022
feb290a
some improvement
lunny Jun 12, 2022
2987a7a
Merge branch 'main' into lunny/render_iframe
lunny Jun 12, 2022
4520ea9
revert change in SanitizerDisabled of external renderer
lunny Jun 12, 2022
a69ec31
Add sandbox for iframe and support allow-scripts and allow-same-origin
lunny Jun 13, 2022
05bc9b2
Merge branch 'main' into lunny/render_iframe
lunny Jun 13, 2022
7d0e74e
refactor
wxiaoguang Jun 14, 2022
069b236
Merge pull request #23 from wxiaoguang/lunny/render_iframe
lunny Jun 14, 2022
a929430
fix
wxiaoguang Jun 14, 2022
806e647
fix lint
wxiaoguang Jun 14, 2022
5a2416e
fine tune
wxiaoguang Jun 14, 2022
eb52c83
Merge branch 'main' into lunny/render_iframe
wxiaoguang Jun 14, 2022
d9679ce
use single option RENDER_CONTENT_MODE, use sandbox=allow-scripts
wxiaoguang Jun 14, 2022
a1cac2a
fine tune CSP
wxiaoguang Jun 14, 2022
cc18ebf
Apply suggestions from code review
lunny Jun 15, 2022
65443b2
Merge branch 'main' into lunny/render_iframe
wxiaoguang Jun 15, 2022
6c81597
Merge branch 'main' into lunny/render_iframe
lunny Jun 15, 2022
631fea3
Merge branch 'main' into lunny/render_iframe
lunny Jun 15, 2022
169362d
Merge branch 'main' into lunny/render_iframe
wxiaoguang Jun 15, 2022
c7df5bf
Merge branch 'main' into lunny/render_iframe
lunny Jun 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/content/doc/advanced/config-cheat-sheet.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -1026,7 +1026,8 @@ IS_INPUT_FILE = false
command. Multiple extensions needs a comma as splitter.
- RENDER\_COMMAND: External command to render all matching extensions.
- IS\_INPUT\_FILE: **false** Input is not a standard input but a file param followed `RENDER_COMMAND`.
- DISABLE_SANITIZER: **false** Don't filter html tags and attributes if true. Don't change this to true except you know what that means.
- DISABLE_SANITIZER: **false** Don't filter html tags and attributes if true. This is insecure. Don't change this to true except you know what that means.
- USE_IFRAME: **false** Display the HTML with an embed iframe but not directly renderer.

Two special environment variables are passed to the render command:
- `GITEA_PREFIX_SRC`, which contains the current URL prefix in the `src` path tree. To be used as prefix for links.
Expand Down
3 changes: 2 additions & 1 deletion docs/content/doc/advanced/config-cheat-sheet.zh-cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,8 @@ IS_INPUT_FILE = false
- FILE_EXTENSIONS: 关联的文档的扩展名,多个扩展名用都好分隔。
- RENDER_COMMAND: 工具的命令行命令及参数。
- IS_INPUT_FILE: 输入方式是最后一个参数为文件路径还是从标准输入读取。
- DISABLE_SANITIZER: **false** 如果为 true 则不过滤 HTML 标签和属性。除非你知道这意味着什么,否则不要设置为 true。
- DISABLE_SANITIZER: **false** 如果为 true 则不过滤 HTML 标签和属性。本功能可能引起 XSS 安全问题。除非你知道这意味着什么,否则不要设置为 true。
- USE_IFRAME: **false** 采用iframe来渲染生成的HTML,还是直接在本页面渲染。

以下两个环境变量将会被传递给渲染命令:

Expand Down
2 changes: 1 addition & 1 deletion modules/csv/csv.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ func CreateReaderAndDetermineDelimiter(ctx *markup.RenderContext, rd io.Reader)
func determineDelimiter(ctx *markup.RenderContext, data []byte) rune {
extension := ".csv"
if ctx != nil {
extension = strings.ToLower(filepath.Ext(ctx.Filename))
extension = strings.ToLower(filepath.Ext(ctx.RelativePath))
}

var delimiter rune
Expand Down
2 changes: 1 addition & 1 deletion modules/csv/csv_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ John Doe john@doe.com This,note,had,a,lot,of,commas,to,test,delimiters`,
}

for n, c := range cases {
delimiter := determineDelimiter(&markup.RenderContext{Filename: c.filename}, []byte(decodeSlashes(t, c.csv)))
delimiter := determineDelimiter(&markup.RenderContext{RelativePath: c.filename}, []byte(decodeSlashes(t, c.csv)))
assert.EqualValues(t, c.expectedDelimiter, delimiter, "case %d: delimiter should be equal, expected '%c' got '%c'", n, c.expectedDelimiter, delimiter)
}
}
Expand Down
5 changes: 5 additions & 0 deletions modules/markup/console/console.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@ func (Renderer) SanitizerDisabled() bool {
return false
}

// DisplayInIFrame disabled sanitize if return true
func (Renderer) DisplayInIFrame() bool {
return false
}

// CanRender implements markup.RendererContentDetector
func (Renderer) CanRender(filename string, input io.Reader) bool {
buf, err := io.ReadAll(input)
Expand Down
5 changes: 5 additions & 0 deletions modules/markup/csv/csv.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,11 @@ func (Renderer) SanitizerDisabled() bool {
return false
}

// DisplayInIFrame disabled sanitize if return true
func (Renderer) DisplayInIFrame() bool {
return false
}

func writeField(w io.Writer, element, class, field string) error {
if _, err := io.WriteString(w, "<"); err != nil {
return err
Expand Down
5 changes: 5 additions & 0 deletions modules/markup/external/external.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,11 @@ func (p *Renderer) SanitizerDisabled() bool {
return p.DisableSanitizer
}

// DisplayInIFrame disabled sanitize if return true
func (p *Renderer) DisplayInIFrame() bool {
return p.UseIFrame
}

func envMark(envName string) string {
if runtime.GOOS == "windows" {
return "%" + envName + "%"
Expand Down
26 changes: 13 additions & 13 deletions modules/markup/html_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ func TestRender_Commits(t *testing.T) {
setting.AppURL = TestAppURL
test := func(input, expected string) {
buffer, err := RenderString(&RenderContext{
Ctx: git.DefaultContext,
Filename: ".md",
URLPrefix: TestRepoURL,
Metas: localMetas,
Ctx: git.DefaultContext,
RelativePath: ".md",
URLPrefix: TestRepoURL,
Metas: localMetas,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
Expand Down Expand Up @@ -80,9 +80,9 @@ func TestRender_CrossReferences(t *testing.T) {

test := func(input, expected string) {
buffer, err := RenderString(&RenderContext{
Filename: "a.md",
URLPrefix: setting.AppSubURL,
Metas: localMetas,
RelativePath: "a.md",
URLPrefix: setting.AppSubURL,
Metas: localMetas,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
Expand Down Expand Up @@ -124,8 +124,8 @@ func TestRender_links(t *testing.T) {

test := func(input, expected string) {
buffer, err := RenderString(&RenderContext{
Filename: "a.md",
URLPrefix: TestRepoURL,
RelativePath: "a.md",
URLPrefix: TestRepoURL,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
Expand Down Expand Up @@ -223,8 +223,8 @@ func TestRender_email(t *testing.T) {

test := func(input, expected string) {
res, err := RenderString(&RenderContext{
Filename: "a.md",
URLPrefix: TestRepoURL,
RelativePath: "a.md",
URLPrefix: TestRepoURL,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(res))
Expand Down Expand Up @@ -281,8 +281,8 @@ func TestRender_emoji(t *testing.T) {
test := func(input, expected string) {
expected = strings.ReplaceAll(expected, "&", "&amp;")
buffer, err := RenderString(&RenderContext{
Filename: "a.md",
URLPrefix: TestRepoURL,
RelativePath: "a.md",
URLPrefix: TestRepoURL,
}, input)
assert.NoError(t, err)
assert.Equal(t, strings.TrimSpace(expected), strings.TrimSpace(buffer))
Expand Down
5 changes: 5 additions & 0 deletions modules/markup/markdown/markdown.go
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,11 @@ func (Renderer) SanitizerDisabled() bool {
return false
}

// DisplayInIFrame disabled sanitize if return true
func (Renderer) DisplayInIFrame() bool {
return false
}

// Render implements markup.Renderer
func (Renderer) Render(ctx *markup.RenderContext, input io.Reader, output io.Writer) error {
return render(ctx, input, output)
Expand Down
5 changes: 5 additions & 0 deletions modules/markup/orgmode/orgmode.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,11 @@ func (Renderer) SanitizerDisabled() bool {
return false
}

// DisplayInIFrame disabled sanitize if return true
func (Renderer) DisplayInIFrame() bool {
return false
}

// Render renders orgmode rawbytes to HTML
func Render(ctx *markup.RenderContext, input io.Reader, output io.Writer) error {
htmlWriter := org.NewHTMLWriter()
Expand Down
24 changes: 21 additions & 3 deletions modules/markup/renderer.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"errors"
"fmt"
"io"
"net/url"
"path/filepath"
"strings"
"sync"
Expand Down Expand Up @@ -44,7 +45,7 @@ type Header struct {
// RenderContext represents a render context
type RenderContext struct {
Ctx context.Context
Filename string
RelativePath string // relative path from tree root of the branch
Type string
IsWiki bool
URLPrefix string
Expand All @@ -54,6 +55,7 @@ type RenderContext struct {
ShaExistCache map[string]bool
cancelFn func()
TableOfContents []Header
AllowIFrame bool
}

// Cancel runs any cleanup functions that have been registered for this Ctx
Expand Down Expand Up @@ -91,6 +93,7 @@ type Renderer interface {
NeedPostProcess() bool
SanitizerRules() []setting.MarkupSanitizerRule
SanitizerDisabled() bool
DisplayInIFrame() bool
Render(ctx *RenderContext, input io.Reader, output io.Writer) error
}

Expand Down Expand Up @@ -142,7 +145,7 @@ func DetectRendererType(filename string, input io.Reader) string {
func Render(ctx *RenderContext, input io.Reader, output io.Writer) error {
if ctx.Type != "" {
return renderByType(ctx, input, output)
} else if ctx.Filename != "" {
} else if ctx.RelativePath != "" {
return renderFile(ctx, input, output)
}
return errors.New("Render options both filename and type missing")
Expand All @@ -163,6 +166,18 @@ type nopCloser struct {

func (nopCloser) Close() error { return nil }

func renderIFrame(ctx *RenderContext, renderer Renderer, input io.Reader, output io.Writer) error {
_, err := io.WriteString(output, fmt.Sprintf(`<iframe src="%s%s/%s/render/%s/%s" name="ifd"
onload="this.height=ifd.document.body.scrollHeight" width="100%%" scrolling="no" frameborder="0"/>`,
setting.AppURL,
ctx.Metas["user"],
ctx.Metas["repo"],
ctx.Metas["BranchNameSubURL"],
url.PathEscape(ctx.RelativePath),
))
return err
}

func render(ctx *RenderContext, renderer Renderer, input io.Reader, output io.Writer) error {
var wg sync.WaitGroup
var err error
Expand Down Expand Up @@ -239,8 +254,11 @@ func (err ErrUnsupportedRenderExtension) Error() string {
}

func renderFile(ctx *RenderContext, input io.Reader, output io.Writer) error {
extension := strings.ToLower(filepath.Ext(ctx.Filename))
extension := strings.ToLower(filepath.Ext(ctx.RelativePath))
if renderer, ok := extRenderers[extension]; ok {
if renderer.DisplayInIFrame() && ctx.AllowIFrame {
return renderIFrame(ctx, renderer, input, output)
}
return render(ctx, renderer, input, output)
}
return ErrUnsupportedRenderExtension{extension}
Expand Down
2 changes: 2 additions & 0 deletions modules/setting/markup.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ type MarkupRenderer struct {
NeedPostProcess bool
MarkupSanitizerRules []MarkupSanitizerRule
DisableSanitizer bool
UseIFrame bool
}

// MarkupSanitizerRule defines the policy for whitelisting attributes on
Expand Down Expand Up @@ -152,5 +153,6 @@ func newMarkupRenderer(name string, sec *ini.Section) {
IsInputFile: sec.Key("IS_INPUT_FILE").MustBool(false),
NeedPostProcess: sec.Key("NEED_POSTPROCESS").MustBool(true),
DisableSanitizer: sec.Key("DISABLE_SANITIZER").MustBool(false),
UseIFrame: sec.Key("USE_IFRAME").MustBool(false),
})
}
4 changes: 2 additions & 2 deletions routers/web/repo/compare.go
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ func setCsvCompareContext(ctx *context.Context) {
return csvReader, reader, err
}

baseReader, baseBlobCloser, err := csvReaderFromCommit(&markup.RenderContext{Ctx: ctx, Filename: diffFile.OldName}, baseCommit)
baseReader, baseBlobCloser, err := csvReaderFromCommit(&markup.RenderContext{Ctx: ctx, RelativePath: diffFile.OldName}, baseCommit)
if baseBlobCloser != nil {
defer baseBlobCloser.Close()
}
Expand All @@ -149,7 +149,7 @@ func setCsvCompareContext(ctx *context.Context) {
return CsvDiffResult{nil, "unable to load file from base commit"}
}

headReader, headBlobCloser, err := csvReaderFromCommit(&markup.RenderContext{Ctx: ctx, Filename: diffFile.Name}, headCommit)
headReader, headBlobCloser, err := csvReaderFromCommit(&markup.RenderContext{Ctx: ctx, RelativePath: diffFile.Name}, headCommit)
if headBlobCloser != nil {
defer headBlobCloser.Close()
}
Expand Down
85 changes: 85 additions & 0 deletions routers/web/repo/render.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
// Copyright 2022 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package repo

import (
"bytes"
"io"
"net/http"
"path"
"strings"

"code.gitea.io/gitea/modules/charset"
"code.gitea.io/gitea/modules/context"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/markup"
"code.gitea.io/gitea/modules/typesniffer"
"code.gitea.io/gitea/modules/util"
)

// RenderFile renders a file by repos path
func RenderFile(ctx *context.Context) {
blob, err := ctx.Repo.Commit.GetBlobByPath(ctx.Repo.TreePath)
if err != nil {
if git.IsErrNotExist(err) {
ctx.NotFound("GetBlobByPath", nil)
} else {
ctx.ServerError("GetBlobByPath", err)
}
return
}

dataRc, err := blob.DataAsync()
if err != nil {
ctx.ServerError("DataAsync", err)
return
}
defer dataRc.Close()

buf := make([]byte, 1024)
n, _ := util.ReadAtMost(dataRc, buf)
buf = buf[:n]

st := typesniffer.DetectContentType(buf)
isTextFile := st.IsText()

rd := charset.ToUTF8WithFallbackReader(io.MultiReader(bytes.NewReader(buf), dataRc))

if markupType := markup.Type(blob.Name()); markupType == "" {
if isTextFile {
_, err = io.Copy(ctx.Resp, rd)
if err != nil {
ctx.ServerError("Copy", err)
}
return
}
ctx.Error(http.StatusInternalServerError, "Unsupported file type render")
return
}

treeLink := ctx.Repo.RepoLink + "/src/" + ctx.Repo.BranchNameSubURL()
if len(ctx.Repo.TreePath) > 0 {
treeLink += "/" + util.PathEscapeSegments(ctx.Repo.TreePath)
}

var result bytes.Buffer
err = markup.Render(&markup.RenderContext{
Ctx: ctx,
RelativePath: ctx.Repo.TreePath,
URLPrefix: path.Dir(treeLink),
Metas: ctx.Repo.Repository.ComposeDocumentMetas(),
GitRepo: ctx.Repo.GitRepo,
}, rd, &result)
if err != nil {
ctx.ServerError("Render", err)
return
}

_, err = charset.EscapeControlReader(strings.NewReader(result.String()), ctx.Resp)
if err != nil {
ctx.ServerError("EscapeControlReader", err)
return
}
}
Loading