🤗 What's the main difference in the throughout between v1.* and v2.*? #1139

fengnex · 2021-01-30T07:44:07Z

Question description
I had some stress tests on fiber v1.14.6 by accessing a local sqlite (query a few records) or remote redis (get a short string value) and found that the result throughtput can up to around 12000 requests/s.

import "github.com/gofiber/fiber"

But the above sentence actually downloads and uses fiber v1.14.6, rather than v2.*. What I wonder is how to let it use fiber v2 instead and the performance diference between the two major versions.

Code snippet Optional


import (
        "fmt"     

        "qc.com/m/v1/book"                    
        "qc.com/m/v1/database"             
        "github.com/gofiber/fiber"
        "github.com/jinzhu/gorm"
        _ "github.com/jinzhu/gorm/dialects/sqlite"
)


func setupRoutes(app *fiber.App) {  
        app.Get("/api/v1/book", book.GetBooks)
}

func initDatabase() {
        var err error
        database.DBConn, err = gorm.Open("sqlite3", "books.db")
        if err != nil {
                panic("failed to connect database")
        }
        fmt.Println("Connection Opened to Database")
        database.DBConn.AutoMigrate(&book.Book{})
        fmt.Println("Database Migrated")
}

func main() {
        app := fiber.New()
        initDatabase()
        setupRoutes(app)
        app.Listen(3000)
       
        defer database.DBConn.Close()
}

The text was updated successfully, but these errors were encountered:

welcome · 2021-01-30T07:44:08Z

Thanks for opening your first issue here! 🎉 Be sure to follow the issue template! If you need help or want to chat with us, join us on Discord https://gofiber.io/discord

kiyonlin · 2021-01-30T07:51:50Z

Please see v2 - Changelog

fengnex · 2021-01-30T07:52:49Z

go.mod's contents:
module qc.com/m/v1

go 1.15

require (
github.com/gofiber/fiber v1.14.6
github.com/gomodule/redigo v1.8.3 // indirect
github.com/jinzhu/gorm v1.9.16
)

Fenny · 2021-01-31T17:17:50Z

@fengnex, are you having problems using v2?

See #824 if it helps with initializing your go.mod

Make sure you are using github.com/gofiber/fiber/v2 for your import paths.

fengnex · 2021-02-01T09:25:55Z

@Fenny There are no any problems for me to correctly use v2.

The original go programs need many modificiations before they can successfully run, but what made me surprise is that the result throughput of v2 falls to 11600 requests/s from acround 12800 requests/s of v1 according to several stress tests. The tested url is http://server-ip:3000/api/v1/book/.

What is the root cause for the above test result? What should I do to improve it or how should I understand that?

The new program which can be compiled with fiber2 looks like the following:

package main

import (
        "fmt"
        "time"

        "qc.com/m/v1/book"
        "qc.com/m/v1/database"
        "github.com/gofiber/fiber/v2"
        "github.com/jinzhu/gorm"
        _ "github.com/jinzhu/gorm/dialects/sqlite"

        //"github.com/go-redis/redis"
)

func helloWorld(c *fiber.Ctx) error {
        return c.SendString("Hello, World!")
}

func rtest(c *fiber.Ctx) error {
        result,err := Exec("get","hello")
        if err != nil {
                fmt.Print(err.Error())
        }
        str,_:=red.String(result,err)
        return c.JSON(str)
}

func setupRoutes(app *fiber.App) {
        app.Get("/", helloWorld)

        app.Get("/api/v1/book", book.GetBooks)
        app.Get("/api/v1/book/:id", book.GetBook)
        app.Post("/api/v1/book", book.NewBook)
        app.Post("/api/v1/book_json", book.NewBookJson)
        app.Delete("/api/v1/book/:id", book.DeleteBook)

        app.Get("/api/v1/rtest",rtest)
}

func initDatabase() {
        var err error
        database.DBConn, err = gorm.Open("sqlite3", "books.db")
        if err != nil {
                panic("failed to connect database")
        }
        fmt.Println("Connection Opened to Database")
        database.DBConn.AutoMigrate(&book.Book{})
        fmt.Println("Database Migrated")
}

func main() {
        app := fiber.New()
        initDatabase()

        initRedis()
        Exec("set","hello","world")

        setupRoutes(app)
        app.Listen(":3000")
       
        defer database.DBConn.Close()
}

fengnex · 2021-02-01T09:31:58Z

Source code of book.GetBooks:

func GetBooks(c *fiber.Ctx) error {
        db := database.DBConn
        var books []Book
        db.Find(&books)
        return c.JSON(books)
        //c.Send("All Books")
}

There are only 3 simple records in the local sqlite database.

fengnex · 2021-02-04T02:42:29Z

Is there anyone who would be so kind as to explain the performance differences between fiber v1.* and v2.*?
Can we achieve an increase in fiber v2's performance and how to implement that?

kiyonlin · 2021-02-04T02:46:36Z

Could you provide an easy and runnable repo to reproduce your issue?

And tell us which benchmark loader you use.

fengnex · 2021-02-04T08:44:38Z

The source code of tested go application using fiber v1 and v2 can be found from the above code.
vegeta (https://github.com/tsenart/vegeta) was used to generate the test results mentioned here, you can obtain similar stress test records using a command like:

echo "GET http://fiber-app-ip:3000/api/v1/rtest/" | ./vegeta attack -rate=13000 -duration=10s > result.bin
./vegeta report -type=json result.bin

One corresponding output is:
{"latencies":{"total":6791532416620,"mean":52242557,"50th":40019999,"90th":62877540,"95th":72578148,"99th":1002333468,"max":1018241338,"min":31253},"bytes_in":{"total":876444,"mean":6.741876923076923},"bytes_out":{"total":0,"mean":0},"earliest":"2021-01-20T14:47:01.953702274+08:00","latest":"2021-01-20T14:47:11.953703133+08:00","end":"2021-01-20T14:47:11.958348869+08:00","duration":10000000859,"wait":4645736,"requests":130000,"rate":12999.998883300095,"throughput":12644.824462187811,"success":0.9731307692307692,"status_codes":{"0":3493,"200":126507},"errors":["Get "http://fiber-app-ip:3000/api/v1/rtest/": dial tcp 0.0.0.0:0-\u003e10.10.39.22:3000: socket: too many open files"]}

I have the both executable files of fiber application and vegeta, but it seems not a good idea to upload them and actually github does not accept such files here.

kiyonlin · 2021-02-04T09:00:31Z

The source code of tested go application using fiber v1 and v2 can be found from the above code.

Sorry, I don't think these code is complete and runnable. By any chance can you at least make /api/v1/rtest runnable?

kiyonlin · 2021-02-04T09:02:49Z

And I can see socket: too many open files error in your output, did you set limitation of socket(fd) correctly?

fengnex · 2021-02-04T09:17:16Z

The complete test programs that use fiber v2 are as follows:

main.go

package main

import (
        "fmt"
        "time"

        "qc.com/m/v1/book"
        "qc.com/m/v1/database"
        "github.com/gofiber/fiber/v2"
        "github.com/jinzhu/gorm"
        _ "github.com/jinzhu/gorm/dialects/sqlite"

        //"github.com/go-redis/redis"
        red "github.com/gomodule/redigo/redis"
)

type Redis struct {
        pool     *red.Pool
}

var redis *Redis

func initRedis() {
        redis = new(Redis)
        redis.pool = &red.Pool{
                MaxIdle:     256,
                MaxActive:   0,
                IdleTimeout: time.Duration(120),
                Dial: func() (red.Conn, error) {
                        return red.Dial(
                                "tcp",
                                "ip:6480",
                                red.DialReadTimeout(time.Duration(1000)*time.Millisecond),
                                red.DialWriteTimeout(time.Duration(1000)*time.Millisecond),
                                red.DialConnectTimeout(time.Duration(1000)*time.Millisecond),
                                red.DialDatabase(0),
                                red.DialPassword("pwd"),
                        )
                },
        }
}

func Exec(cmd string, key interface{}, args ...interface{}) (interface{}, error) {
        con := redis.pool.Get()
        if err := con.Err(); err != nil {
                return nil, err
        }
        defer con.Close()
        parmas := make([]interface{}, 0)
        parmas = append(parmas, key)

        if len(args) > 0 {
                for _, v := range args {
                        parmas = append(parmas, v)
                }
        }
        return con.Do(cmd, parmas...)
}

func helloWorld(c *fiber.Ctx) error {
        return c.SendString("Hello, World!")
}

func rtest(c *fiber.Ctx) error {
        result,err := Exec("get","hello")
        if err != nil {
                fmt.Print(err.Error())
        }
        str,_:=red.String(result,err)
        return c.JSON(str)
}

func setupRoutes(app *fiber.App) {
        app.Get("/", helloWorld)

        app.Get("/api/v1/book", book.GetBooks)
        app.Get("/api/v1/book/:id", book.GetBook)
        app.Post("/api/v1/book", book.NewBook)
        app.Post("/api/v1/book_json", book.NewBookJson)
        app.Delete("/api/v1/book/:id", book.DeleteBook)

        app.Get("/api/v1/rtest",rtest)
}

func initDatabase() {
        var err error
        database.DBConn, err = gorm.Open("sqlite3", "books.db")
        if err != nil {
                panic("failed to connect database")
        }
        fmt.Println("Connection Opened to Database")
        database.DBConn.AutoMigrate(&book.Book{})
        fmt.Println("Database Migrated")
}

func main() {
        app := fiber.New()
        initDatabase()

        initRedis()
        Exec("set","hello","world")

        setupRoutes(app)
        app.Listen(":3000")
       
        defer database.DBConn.Close()
}

book/book.go

package book

import (
        "qc.com/m/v1/database"
        "github.com/gofiber/fiber/v2"
        "github.com/jinzhu/gorm"
        _ "github.com/jinzhu/gorm/dialects/sqlite"
)

type Book struct {
        gorm.Model
        Title  string `json:"name"`
        Author string `json:"author"`
        Rating int    `json:"rating"`
}

func GetBooks(c *fiber.Ctx) error {
        db := database.DBConn
        var books []Book
        db.Find(&books)
        return c.JSON(books)
}

func GetBook(c *fiber.Ctx) error {
        id := c.Params("id")
        db := database.DBConn
        var book Book
        db.Find(&book, id)
        return c.JSON(book)
        //c.Send("Single Book")
}

func NewBook(c *fiber.Ctx) error {
        db := database.DBConn
        var book Book
        book.Title = "1984"
        book.Author = "George Orwell"
        book.Rating = 5
        db.Create(&book)
        return c.JSON(book)
}

func NewBookJson(c *fiber.Ctx) error {
        db := database.DBConn
        book := new(Book)
        if err := c.BodyParser(book); err != nil {
                return c.SendStatus(503)
        }
        db.Create(&book)
        return c.JSON(book)
}

func DeleteBook(c *fiber.Ctx) error {
        id := c.Params("id")
        db := database.DBConn

        var book Book
        db.First(&book, id)
        if book.Title == "" {
            return c.SendStatus(500)
        }
        db.Delete(&book)
        return c.SendString("Book Successfully deleted")
}

database/database.go

package database

import (
        "github.com/jinzhu/gorm"
        _ "github.com/jinzhu/gorm/dialects/sqlite"
)

var (
        DBConn *gorm.DB
)

go.mod

module qc.com/m/v1

go 1.15

require (
        github.com/gofiber/fiber/v2 v2.4.0
        github.com/gomodule/redigo v1.8.3 // indirect
        github.com/jinzhu/gorm v1.9.16
)

Related configuration of socket: too many open files:

ulimit -a|grep 'open files'
open files                      (-n) 65535

The url (http://fiber-app-ip:3000/api/v1/rtest/) is to obtain a string value from redis by calling redis's get. The result throughput is similar with that of accessing local sqlite; I guess this is because of capacity limitation of the fiber service.

fengnex · 2021-02-04T09:22:24Z

@kiyonlin
The complete test programs for fiber v1 can be quickly wrote according to the first few posts, and I can also paste them here if you want me to do that.

All the tests are executed on CentOS Linux release 7.8.2003. The test server has 6 cpu cores and 64Gb of memory.

Fiber v2 application has a different banner with that of fiber v1 application and it looks like the following:

┌───────────────────────────────────────────────────┐ 
 │                    Fiber v2.4.0                   │ 
 │               http://127.0.0.1:3000               │ 
 │                                                   │ 
 │ Handlers ............ 11  Processes ........... 1 │ 
 │ Prefork ....... Disabled  PID ............. 14106 │ 
 └───────────────────────────────────────────────────┘

kiyonlin · 2021-02-05T00:47:46Z

I think I found the reason.

This config IdleTimeout: time.Duration(120) in the redis pool means 120ns, so you'll dial redis almost all the time. Please change it to IdleTimeout: time.Second * 120, and set config Wait: true.

The vegeta result on my machine looks like

{"latencies":{"total":30829800748,"mean":237152,"50th":132145,"90th":159838,"95th":201268,"99th":816517,"max":32812825,"min":81229},"bytes_in":{"total":910000,"mean":7},"bytes_out":{"total":0,"mean":0},"earliest":"2021-02-05T08:42:43.680300663+08:00","latest":"2021-02-05T08:42:53.680191261+08:00","end":"2021-02-05T08:42:53.680388308+08:00","duration":9999890598,"wait":197047,"requests":130000,"rate":13000.14222415596,"throughput":12999.886062498605,"success":1,"status_codes":{"200":130000},"errors":[]}

fengnex · 2021-02-05T02:24:57Z

@kiyonlin You are right and I found the maximum throughput can reach 30880 requests/s in my environment where fiber application and Redis are located on different servers, which I think is a great test result.

But there is a throughput decline about 400 requests/s when I attacked http://fiber-app-ip:3000/api/v1/book/ accessing local sqlite in the same environment.

The detailed result of v1:
echo "GET http://fiber-app-ip:3000/api/v1/book/" | ./vt attack -rate=16000 -duration=10s | ./vt report -type=json
{"latencies":{"total":9698308528040,"mean":60614428,"50th":5174020,"90th":238913978,"95th":315546984,"99th":401291065,"max":632856422,"min":30253},"bytes_in":{"total":66534244,"mean":415.839025},"bytes_out":{"total":0,"mean":0},"earliest":"2021-01-30T12:03:16.205139026+08:00","latest":"2021-01-30T12:03:26.204931032+08:00","end":"2021-01-30T12:03:26.328178158+08:00","duration":9999792006,"wait":123247126,"requests":160000,"rate":16000.332797321986,"throughput":12862.14528089804,"success":0.813775,"status_codes":{"0":29796,"200":130204},"errors":["Get "http://fiber-app-ip:3000/api/v1/book/": dial tcp 0.0.0.0:0-\u003e10.10.39.22:3000: socket: too many open files"]}

Corresponding test result of v2:
{"latencies":{"total":9962809811985,"mean":62267561,"50th":4750012,"90th":246529619,"95th":318445771,"99th":412132071,"max":665999169,"min":30332},"bytes_in":{"total":64524481,"mean":403.27800625},"bytes_out":{"total":0,"mean":0},"earliest":"2021-02-05T09:40:39.87155801+08:00","latest":"2021-02-05T09:40:49.871467294+08:00","end":"2021-02-05T09:40:50.047015937+08:00","duration":9999909284,"wait":175548643,"requests":160000,"rate":16000.145146916717,"throughput":12409.367804956184,"success":0.78919375,"status_codes":{"0":33729,"200":126271},"errors":["Get "http://fiber-app-ip:3000/api/v1/book/": dial tcp 0.0.0.0:0-\u003e10.10.39.22:3000: socket: too many open files"]}

I suppose we can conclude that fiber is an outstanding web application from these results. But wheter is it fair if we say there is no noticeable performance improvement and actually there may be certain performance decline sometimes after upgrading to fiber v2?

kiyonlin · 2021-02-05T02:31:19Z

I think the limitation is on db(sqlite/redis), not the framework. You can test helloWorld to see if there has obvious performance difference.

fengnex · 2021-02-05T02:43:24Z

@kiyonlin I agree with you; but what's your opinion when comparing the test results of fiber v1 and v2?

kiyonlin · 2021-02-05T02:47:41Z

see #773 (v2 Benchmarks). But the most important thing is the breaking change of api.

fengnex · 2021-02-05T03:15:51Z

I still don't know what we should think about the performance degradation of fiber V2 in accessing the database. I think this is a general problem and I don't think such a problem should occur in the update of a large new version.

I see that fiber V2 supports more and more functions and features, but I still don't know what the main added value is and whether it is worth it. I will be glad to see other opinions.

kiyonlin · 2021-02-05T03:40:43Z

Maybe it is because fiber is faster and database can hardly to catch up.

hi019 · 2021-02-05T04:36:32Z

I still don't know what we should think about the performance degradation of fiber V2 in accessing the database. I think this is a general problem and I don't think such a problem should occur in the update of a large new version.

We have not observed any performance regressions from v1 to v2. I think it would be best for you to create a new issue with a small example (preferably one route) that demonstrates the regression, and we can discuss v2 in this issue.

I see that fiber V2 supports more and more functions and features, but I still don't know what the main added value is and whether it is worth it. I will be glad to see other opinions.

The main feature of v2 is the error handling. It is now more elegant and natural. Also, processing of many routes is faster, there is new middleware (monitor, rewritten session, compress improvements and more). All our middleware also uses a standardized storage interface, implemented in https://github.com/gofiber/storage.

In addition, Fiber v1 is considered legacy. While we will still update it with security fixes, eventually it will be unsupported. Like any other major package update, it is better to change your code sooner than later.

Fenny added the ♻ Wait for Response label Jan 31, 2021

kiyonlin removed the ♻ Wait for Response label Feb 3, 2021

hi019 closed this as completed Feb 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🤗 What's the main difference in the throughout between v1.* and v2.*? #1139

🤗 What's the main difference in the throughout between v1.* and v2.*? #1139

fengnex commented Jan 30, 2021 •

edited

Loading

welcome bot commented Jan 30, 2021

kiyonlin commented Jan 30, 2021

fengnex commented Jan 30, 2021

Fenny commented Jan 31, 2021

fengnex commented Feb 1, 2021 •

edited

Loading

fengnex commented Feb 1, 2021 •

edited

Loading

fengnex commented Feb 4, 2021

kiyonlin commented Feb 4, 2021

fengnex commented Feb 4, 2021 •

edited

Loading

kiyonlin commented Feb 4, 2021

kiyonlin commented Feb 4, 2021 •

edited

Loading

fengnex commented Feb 4, 2021 •

edited

Loading

fengnex commented Feb 4, 2021 •

edited

Loading

kiyonlin commented Feb 5, 2021

fengnex commented Feb 5, 2021 •

edited

Loading

kiyonlin commented Feb 5, 2021

fengnex commented Feb 5, 2021

kiyonlin commented Feb 5, 2021

fengnex commented Feb 5, 2021

kiyonlin commented Feb 5, 2021

hi019 commented Feb 5, 2021

🤗 What's the main difference in the throughout between v1.* and v2.*? #1139

🤗 What's the main difference in the throughout between v1.* and v2.*? #1139

Comments

fengnex commented Jan 30, 2021 • edited Loading

welcome bot commented Jan 30, 2021

kiyonlin commented Jan 30, 2021

fengnex commented Jan 30, 2021

Fenny commented Jan 31, 2021

fengnex commented Feb 1, 2021 • edited Loading

fengnex commented Feb 1, 2021 • edited Loading

fengnex commented Feb 4, 2021

kiyonlin commented Feb 4, 2021

fengnex commented Feb 4, 2021 • edited Loading

kiyonlin commented Feb 4, 2021

kiyonlin commented Feb 4, 2021 • edited Loading

fengnex commented Feb 4, 2021 • edited Loading

fengnex commented Feb 4, 2021 • edited Loading

kiyonlin commented Feb 5, 2021

fengnex commented Feb 5, 2021 • edited Loading

kiyonlin commented Feb 5, 2021

fengnex commented Feb 5, 2021

kiyonlin commented Feb 5, 2021

fengnex commented Feb 5, 2021

kiyonlin commented Feb 5, 2021

hi019 commented Feb 5, 2021

fengnex commented Jan 30, 2021 •

edited

Loading

fengnex commented Feb 1, 2021 •

edited

Loading

fengnex commented Feb 1, 2021 •

edited

Loading

fengnex commented Feb 4, 2021 •

edited

Loading

kiyonlin commented Feb 4, 2021 •

edited

Loading

fengnex commented Feb 4, 2021 •

edited

Loading

fengnex commented Feb 4, 2021 •

edited

Loading

fengnex commented Feb 5, 2021 •

edited

Loading