Skip to content

Commit

Permalink
add wal benchmark
Browse files Browse the repository at this point in the history
  • Loading branch information
jiacai2050 committed Nov 18, 2024
1 parent 5ed5869 commit a44febc
Show file tree
Hide file tree
Showing 5 changed files with 75 additions and 6 deletions.
30 changes: 30 additions & 0 deletions content/cn/blog/2024/release-2.1.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
title: 2.1.0 版本发布
date: 2024-11-18
tags: release
---

Apache HoraeDB(孵化中)团队很高兴地宣布,v2.1.0 版本已于 2024-11-18 发布,这个版本解决了 60 多个问题,并且包括两个主要功能:

## 1. 引入基于本地磁盘的新 WAL 实现

在之前的版本中,有一个基于 RocksDB 的 WAL。虽然它在大多数情况下运行良好,但存在以下问题:

- 从源代码编译可能是一项具有挑战性的任务,尤其是因为 RocksDB 主要是用 C++ 编写的。
- 对于 WAL 而言,RocksDB 可能有些矫枉过正。如果你对 RocksDB 不熟悉,那么对它进行调整可能会非常具有挑战性。

通过这个新的 WAL,就很好的解决了上面两个问题,而且在性能测试结果,新 WAL 的表现略优于之前的实现,给以后的优化打下了结实的基础。

![写入速率对比](/images/local-wal-write.png)
![回放速率对比](/images/local-wal-replay.png)

感兴趣的朋友可以参考[这里的设计文档]({{< ref "wal_on_disk.md" >}})了解更多这个特性的细节。

## 2. 使用 Apache OpenDAL 访问对象存储

## 其他

其他错误修复和改进请参见此处:
https://github.com/apache/horaedb/releases/tag/v2.1.0

> 我们一如既往地热忱欢迎您加入[我们的社区](/community),分享您的真知灼见。
37 changes: 31 additions & 6 deletions content/en/blog/2024/release-2.1.0.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,46 @@
---
title: Release 2.1.0
date: 2024-11-18
tags: release
---

The Apache HoraeDB(incubating) team are pleased to announce that v2.1.0 is released, which has closed over 60 issues, including two major features:

- Introduce a new WAL implementation based on local disk.
## 1. New WAL implementation based on local disk.

In previous version, there is a RocksDB-based WAL. Although it works well in most cases, it has following issues:
In previous version, there is a RocksDB-based WAL. Although it works well in most cases, it has following issues:

- Compiling from source can be a challenging task, especially since RocksDB is primarily written in C++.
- For WAL, RocksDB can be somewhat overkill. If you are not familiar with RocksDB, tuning it can be very challenging.
- Compiling from source can be a challenging task, especially since RocksDB is primarily written in C++.
- For WAL, RocksDB can be somewhat overkill. If you are not familiar with RocksDB, tuning it can be very challenging.

- Access object store with [Apache OpenDAL](https://github.com/apache/opendal)
With this new WAL, the above two problems are solved very well, and in performance test, the new WAL slightly outperforms the previous implementation, giving a solid foundation for future optimizations.

![Comparison of Write throughout](/images/local-wal-write.png)
![Comparison of Replay time](/images/local-wal-replay.png)

Interested readers can refer to the design documentation [here]({{< ref "wal_on_disk.md" >}}) for more details on this feature.

### How to enable

```
[analytic.wal]
type = "Local"
data_dir = "/path/to/local/wal"
```

## 2. Access object store with [Apache OpenDAL](https://github.com/apache/opendal)

OpenDAL (Open Data Access Layer) is a project that provides a unified API for accessing various data storage backends.
It offers several advantages for developers and organizations. Here are some key benefits:

- Unified API. OpenDAL provides a consistent and unified API for accessing different storage backends, such as AWS S3, Azure Blob Storage, and local file systems.
- Optimized for Efficiency: OpenDAL is built with performance in mind. It includes optimizations to ensure efficient data access and manipulation, making it suitable for high-performance applications.
- Comprehensive Documentation: The project provides detailed documentation, making it easier for developers to get started and understand how to use the library effectively.

## Others

Other bug fixes and improvements can be seen here:

- https://github.com/apache/horaedb/releases/tag/v2.1.0

As always, we warmly welcome you to join our [community](/community) and share your insights.
> As always, we warmly welcome you to join our [community](/community) and share your insights.
14 changes: 14 additions & 0 deletions layouts/_default/_markup/render-image.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<figure>
<img src="{{ .Destination | safeURL }}"
alt="{{ .Text | plainify }}"
{{ with .Title}}
title="{{ . | plainify }}"
{{ end }}
loading="lazy" />

{{ if .Title }}
<figcaption><center>{{ .Title | safeHTML }}</center></figcaption>
{{ else if .Text }}
<figcaption><center>{{ .Text | safeHTML }}</center></figcaption>
{{ end }}
</figure>
Binary file added static/images/local-wal-replay.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/images/local-wal-write.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit a44febc

Please sign in to comment.