Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

epub navigation #202

Closed
zeoint opened this issue Jul 15, 2023 · 33 comments
Closed

epub navigation #202

zeoint opened this issue Jul 15, 2023 · 33 comments
Labels
enhancement New feature or request

Comments

@zeoint
Copy link

zeoint commented Jul 15, 2023

Description of the new feature / enhancement

  • If any epub is opened, it will be unzipped to the temp folder and then displayed in yomikiru.
  • Since epub is the same as zip, cbz, 7z files. I mean archive.
  • So large archives cause the reader to slow down which is normal and it is the same for large zip, 7z, cbz files.

So I have an idea to solve this. I don't know if it is possible to do this.

  1. Epub chapters will be unzipped and placed in a folder with epub title as the folder's name. (Same way as manga files)
  2. Parent folder -> Epub files .xhtml
  3. For epub side list, take the epub and chapter title from the folder name and chapter.xhml files respectively.
  4. Same as manga side list. So no dependency on TOC.ncx, content.opf files.
  5. As the epub side list and its navigation would work the same way as it does for manga sidelist, navigation.
  6. If stylesheet is needed for the custom text, paragraph spacing, etc of the chapters then stylesheet.css will be in the epub folder alongside the .xhtml files or treat .css file in the epub folder as stylesheet, since there will only be one.
  7. If this works, there will be no loading times, no uzipping times, no lag at all.
  8. Please check.

Drawback

  1. Collapsible / Un-collapsible TOC would not work. But it is only one drawback and it does not matter.

Scenario when this would be used?

  • To read epub's lag free, unzipping times free.

Supporting information

  1. Basically the same navigation system as manga, but there will be a stylesheet.css (Only if needed).
  2. The chapter will be directly inside the parent folder rather than sub-folders.
  3. As for the image files in the epub, I extracted them to the epub folder and changed the respective image links to the local image links and it worked when I opened the .html file in browser (Even if the images do not work, its not a problem).
@zeoint zeoint added the enhancement New feature or request label Jul 15, 2023
@mienaiyami
Copy link
Owner

I was already planning to make option to not temp files after app close so they can be opened again without big loading time.
In case of epubs, most of the loading time is used in just extracting and not loading the content.

This does sound like a better option in term of overall performance but at same time its a lot more complex.
First I will try the method i mentioned above as it sound easier to implement and works for all archives and pdf as well. If there is still a need for your method after this then I might consider.

@mienaiyami
Copy link
Owner

Added in 2.15.1 as test.
Do you think there is a need for your way?

@zeoint
Copy link
Author

zeoint commented Jul 15, 2023

Do you think there is a need for your way?

Yes, please.
Mainly because of performance of large epub's.
Also there would be no need for the temp and other files in this way.


Even when a manga has 1000's of chapters, there is no effect on performance, because they are not in archive format.
I think the same will happen to epub's in this way.

@mienaiyami
Copy link
Owner

Also there would be no need for the temp and other files in this way.

It will though, since those new created files are kind of temp files. Also, it will be taking more than double the storage.

Even when a manga has 1000's of chapters, there is no effect on performance, because they are not in archive format.
I think the same will happen to epub's in this way.

Thats true.

@zeoint
Copy link
Author

zeoint commented Jul 15, 2023

It will though, since those new created files are kind of temp files.

How? There will only be the chapter .xhtml files.
If a stylesheet is needed to add custom line, paragraph spaces, etc.
Then the stylesheet will be in the respective epub's folder, so there wouldn't need to be a temp file.

Also, it will be taking more than double the storage.

Any unzipped archive would go back to its original size.

@zeoint
Copy link
Author

zeoint commented Jul 15, 2023

If a stylesheet is needed to add custom line, paragraph spaces, etc.
Then the stylesheet will be in the respective epub's folder, so there wouldn't need to be a temp file.

Or does it not work this way.

@mienaiyami
Copy link
Owner

Maybe I didnt get what you mean.
Arent you saying that epub should be unzipped and kept in same folder permanently, and have its content arrange folder as chapters in manga? If so, wont there be extra space usage of more than the epub itself.

If a stylesheet is needed to add custom line, paragraph spaces, etc.
Then the stylesheet will be in the respective epub's folder, so there wouldn't need to be a temp file.

Usually theres one or two stylesheets and they apply to all chapters. But they do not contain just line, paragraph space, etc., there are many other styles.

Any unzipped archive would go back to its original size.

What does that mean?

@zeoint
Copy link
Author

zeoint commented Jul 16, 2023

Any unzipped archive would go back to its original size.

What does that mean?

For example, yomikiru v2.15.3 has a download size of 88 mb, when unzipped it becomes 200mb which is its original size.

Maybe I didnt get what you mean.

It was a big misunderstanding. I should have explained it more clearly.

@zeoint
Copy link
Author

zeoint commented Jul 16, 2023

Arent you saying that epub should be unzipped and kept in same folder permanently, and have its content arrange folder as chapters in manga? If so, wont there be extra space usage of more than the epub itself.

The process I thought of,

  1. The user will unzip the epub themselves using winrar or 7zip.
  2. They will then place both the stylesheets, .html / .xhtml / .mhtml files in a folder having the epub name.
  3. Now, Yomikiru must be able to open that folder.
  4. The parent folder will be the epub title.
  5. The html files in that parent folder will be the chapter name.
  6. In this way, there would be no need to depend on toc.ncx, content.opf, other unnecessary files.
  7. Only .html files and stylesheets would be necessary and the overall performance would be great.
  8. The same way as manga side list, but the chapters here would directly be inside the parent folder.

@zeoint
Copy link
Author

zeoint commented Jul 16, 2023

Arent you saying that epub should be unzipped and kept in same folder permanently, and have its content arrange folder as chapters in manga? If so, wont there be extra space usage of more than the epub itself.

  • Yomikru doesn't need to do any complex process.
  • As in the above video, the user will unzip the text files, stylesheet and place it in a folder.
  • Yomikiru needs to be able to read that folder.
  • In the epub side list, the folder name as the epub's title and html / xhtml / mhtml file names as chapter title.
  • Just like it does for manga side list.
  • Since this is a folder and not archive, the overall performance of epub's will be improved greatly.

@zeoint
Copy link
Author

zeoint commented Jul 16, 2023

As in the above video, the user will unzip the text files, stylesheet and place it in a folder.

Yomikiru needs to be able to read that folder.

In the epub side list, the folder name as the epub's title and html / xhtml / mhtml file names as chapter title.

  • Also this type of parsing should only be done for .html, .xhtml, .mhtml files, when they are inside the parent folder directly without any sub-folders as in the above video.

  • If the parent folder has any of these format files inside it directly without any sub-folders, then do the above parsing.

@mienaiyami
Copy link
Owner

  • The user will unzip the epub themselves using winrar or 7zip.
  • They will then place both the stylesheets, .html / .xhtml / .mhtml files in a folder having the epub name.

I dont think its ideal for users to do that. As you just said, it is for epub with high chapter count.

Also in most cases one .xhtml file is not same as one chapter. A chapter can have more than one file and images as defined in content.opf and TOC

  • Yomikru doesn't need to do any complex process.
  • As in the above video, the user will unzip the text files, stylesheet and place it in a folder.
  • Yomikiru needs to be able to read that folder.

So this is not possible, unless Yomikiru itself make those folders with help of content.opf and TOC.ncx. Try looking at extract of different epubs.

Still the main issue will be order and content of chapter.
For example,
image
image

image

As you can see here, xhtml files dont really represent chapters. Real chapter files are in TOC and they internally import other .xhtml file which are outside of TOC (they are in content.opf and relationship if defined there).

@zeoint
Copy link
Author

zeoint commented Jul 16, 2023

What you said is true for official epub's. They have no performance issues. I should have said this sooner.

I am talking about the epub's that are downloaded manually from a website like I mentioned previously using webtoepub.

  • Those epub's do not have any images at all and each .xhtml file is a chapter. These are the one's I am talking about.
  • Can you make the above method work for these type of files since they have no images and each .xhtml file is a chapter?
  • They also only contain one stylesheet.
  • Because these are the only ones that have 1000's of chapters.

@mienaiyami
Copy link
Owner

It turned out to be harder than expected. Everything is there to make it work but it require me to make a separate file as manga and epub have their own. Which I dont want to because then I will have same code everywhere.
But I was already planning to make something like this that works with .html or .txt or .md, so I will try to find some solution, but dont expect this feature anytime soon.

@mienaiyami
Copy link
Owner

mienaiyami commented Jul 25, 2023

For clarity, you are not talking about creating temp files of the .xhtml files, right?

Yes, Im not talking about that. that part is actually very easy.

Btw, if you havent noted yet, the lag and stutter is because of the item count in side list (its present in manga side list as well).
Almost all epub reader have TOC hidden. If you hide it totally (not zen mode), there will be no lag or stutter.
I can make items in side list hidden when cursor is not over it but will result in a lag when you move cursor over it.

@zeoint
Copy link
Author

zeoint commented Jul 26, 2023

Btw, if you havent noted yet, the lag and stutter is because of the item count in side list (its present in manga side list as well).

Do you mean the lag that occurs when resizing the side list?

For clarity, you are not talking about creating temp files of the .xhtml files, right?

Yes, Im not talking about that. that part is actually very easy.

So temp files are needed even for the extracted .xhtml files.

@zeoint
Copy link
Author

zeoint commented Jul 26, 2023

Btw, if you havent noted yet, the lag and stutter is because of the item count in side list (its present in manga side list as well).

Or, are you talking about the reader lag? Yes, I have noticed it, compared to the version v2.12.3, the epub reader is laggy while going to another chapter in the present version.

While in v2.12.3, the chapter transition was smooth and lag free.

@mienaiyami
Copy link
Owner

mienaiyami commented Jul 26, 2023

Do you mean the lag that occurs when resizing the side list?

No, I mean that lag will be there as long as there is high chapter count in sidelist, its same for manga's side list.
The lag you see in that epub is because of side list and not the data in memory.

@mienaiyami
Copy link
Owner

Or, are you talking about the reader lag? Yes, I have noticed it, compared to the version v2.12.3, the epub reader is laggy while going to another chapter in the present version.

Thats because of #177 (comment) , it wasnt updating because whole sidelist was not rendered as at all in zen mode. But now that side list is rendered and just hidden. So sise-list is re-rendered every time chapter changes and because chapter count is too high, it cause big lags. It is also the reason for scrolling not being smooth.

#177 (comment), though I did say that, I just made side-list render even in zenmode. I even tested it on epub with 500chapters but didnt see any issue at that time, but maybe I need to use this approach instead.

@zeoint
Copy link
Author

zeoint commented Jul 26, 2023

#177 (comment), though I did say that, I just made side-list render even in zenmode. I even tested it on epub with 500chapters but didnt see any issue at that time, but maybe I need to use this approach instead.

So you are saying you can make the reader smooth while scrolling, chapter transition, like it was before, right?

@mienaiyami
Copy link
Owner

So you are saying you can make the reader smooth while scrolling, chapter transition, like it was before, right?

Yes, but it will result in the big lag when exiting zen mode. So, I think scrolling and chapter transition performance is preferred over this?

@zeoint
Copy link
Author

zeoint commented Jul 27, 2023

Yes.

@zeoint
Copy link
Author

zeoint commented Jul 27, 2023

If removing #177 would improve the overall performance of the epub reader, then please do.

@zeoint
Copy link
Author

zeoint commented Jul 28, 2023

Below is a video comparsion of chapter transition performance between v2.12.3 and v2.16.1.
The v2.12.3 chapter transition is very fluid and seamless.


@mienaiyami
Copy link
Owner

Thats probably because of #190. I will need to look for a better way to do this.
Generally, epub with high chapter count have a lot of issues, as you can see already, that's why there are many epub splitter online.
I have been reading ORV with chapter count of 551 and there's no issue in it. 2334 chapters is just too much, so neither current epub reader nor if I made a new for xhtml/txt will be able handle it.
As I mentioned above in #202 (comment), even if you extract and keep htmlx files in a folder and try to read them, it will still lag, both scrolling and chapter change. Unless you only keep like 500 xhtml in a folder, and that will be just like splitting a epub.
You can try 64-bit version for a little extra performance, but downside is higher RAM usage.

Though, Im still planning to implement this to support files with text.

@zeoint
Copy link
Author

zeoint commented Jul 28, 2023

Thats probably because of #190. I will need to look for a better way to do this.

Cool. If needed for performance, please remove it.

@mienaiyami
Copy link
Owner

As I said above.

I have been reading ORV with chapter count of 551 and there's no issue in it.

Even that is a lot for an epub.
I have fixed some performance issue related to #190, and lag will be less than before but won't be removing it just for a single epub file.

test.mp4

As you can see here, the lag in changing chapter is side list, and its better in zen mode.
Main reason for performance here is my lack of knowledge in epubs and the way im implementing features that need it.
Hope you understand.

@zeoint
Copy link
Author

zeoint commented Jul 28, 2023

and its better in zen mode.

Indeed it is.

Main reason for performance here is my lack of knowledge in epubs and the way im implementing features that need it.

If you found a way in future, please implement it.

Hope you understand.

Cool.

Yomikiru is already the best all in one reader.
This is only a minor setback.

@zeoint
Copy link
Author

zeoint commented Aug 9, 2023

Would this be in the upcoming update?

@mienaiyami
Copy link
Owner

Very unlikely, as i said before, it will require me to make a new type of reader just like manga and epub, So, Im trying to clean old code that caused big problems when making epub reader.

@zeoint
Copy link
Author

zeoint commented Aug 10, 2023

Cool.

mienaiyami added a commit that referenced this issue Aug 25, 2023
mienaiyami added a commit that referenced this issue Aug 25, 2023
@zeoint
Copy link
Author

zeoint commented Aug 27, 2023

As I had expected, the scrolling is very smooth even when there are 1000's of chapters.
As for the peformance, the chapter opens very fast as it is only in kb's.

Thank you for implementing this.

@zeoint
Copy link
Author

zeoint commented Sep 1, 2023

I will be closing this since it was completed.

@zeoint zeoint closed this as completed Sep 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants