Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] DiffusionToolkit not reading separately stored aesthetic score tags from A1111 files in certain circumstances #156

Open
curiousjp opened this issue Dec 18, 2023 · 3 comments

Comments

@curiousjp
Copy link
Contributor

curiousjp commented Dec 18, 2023

Describe the bug
PNG files from A1111 that embed a separate aesthetic_score tag in a PNG-tEXt block do not have this information read into their FileParameters structure, due to the OtherParameters field already being set elsewhere.

Version:
v1.4. The extension embedding the tag is stable-diffusion-webui-aesthetic-image-scorer, version 1c3ab7a1.

To Reproduce
Generate an image and have it loaded into Diffusion Toolkit. Set a search that uniquely identifies this image. Add a filter for aesthetic_score > 0. The image is filtered out.

Expected behavior
The image should remain, as it has an aesthetic score of 6.2.

Source Image
apple

Note that the parameters: tag starts at offset 33, and the aesthetic_score resides in a separate tag at offset 333.

Additional context
My hypothesis is that the problem is in the ReadFromFile function in Metadata.cs.

During execution, we encounter the "parameters:" tag and ReadA111Parameters is called. The tag is split and iterated over.
During this, the function moves to state 2 as the tag includes the line, "Steps: 20, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 43846133, Size: 512x512, Model hash: 5dd07a46d7, Model: richyrichmix_V2Fp16, VAE hash: df3c506e51, VAE: pastel-waifu-diffusion.vae.pt, Clip skip: 2, Token merging ratio: 0.5, Version: v1.7.0". Note that this tag fragment does not include the aesthetic score.

On line 866, this tag fragment is stored in fileParameters.OtherParameters, and eventually this fileParameters structure is returned to the outer function. Subsequently, the "aesthetic_score:" tEXt block may be encountered at line 103 and the value stored in aestheticScore, but this is not propagated into the fileParameters structure at line 149 because fileParameters.OtherParameters is no longer null and therefore the ??= assignment does not take place.

Suggestion
If I have understood correctly, the block of code at 148-149 should test in the alternative if OtherParameters is set - if not, the current behaviour is correct, but otherwise the aesthetic score should be appended if it is not already present from an earlier read operation.

@curiousjp
Copy link
Contributor Author

Well - I feel silly now. After build a local copy and doing some tests, it seems the issue is not the non-writing of OtherParameters but that the aestheticScore variable doesn't appear to be written to fileParameters.AestheticScore. Adding a line to do that at 1001 and 149 seems to have fixed the problem.

@RupertAvery
Copy link
Owner

Thanks, do you have a pull request for this?

@curiousjp
Copy link
Contributor Author

Unfortunately, I don't know much about git (or version control), but hopefully what I've proposed as request #163 is what you need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants