Implement Caballero et al. spectral factor model #1296

Jacc0027 · 2021-08-26T16:50:46Z

Closes Add a model for spectral corrections #1278
I am familiar with the contributing guidelines
Tests added
Updates entries to docs/sphinx/source/api.rst for API changes.
Adds description and name entries in the appropriate "what's new" file in docs/sphinx/source/whatsnew for all changes. Includes link to the GitHub Issue with :issue:`num` or this Pull Request with :pull:`num`. Includes contributor name and/or GitHub username (link with :ghuser:`user`).
New code is fully documented. Includes numpydoc compliant docstrings, examples, and comments where necessary.
Pull request is nearly complete and ready for detailed review.
Maintainer: Appropriate GitHub Labels and Milestone are assigned to the Pull Request and linked Issue.

https://github.com//issues/1278

kandersolar · 2021-09-24T18:50:31Z

Hi @Jacc0027 -- sorry for the delay if you were waiting for feedback on this PR! It looks great so far. To make a more detailed review easier, could you address the code style issues noted by "Stickler CI" on this page? https://github.com/pvlib/pvlib-python/pull/1296/files

This will also need some tests (see the contributing docs) to check that the code works on all the various python versions and package versions we support. Please reach out if you need input from us to move this PR forward :) And thanks again for the contribution!

…lib-python into Spectral-corrections

Jacc0027 · 2021-09-25T11:40:23Z

Hi @Jacc0027 -- sorry for the delay if you were waiting for feedback on this PR! It looks great so far. To make a more detailed review easier, could you address the code style issues noted by "Stickler CI" on this page? https://github.com/pvlib/pvlib-python/pull/1296/files

This will also need some tests (see the contributing docs) to check that the code works on all the various python versions and package versions we support. Please reach out if you need input from us to move this PR forward :) And thanks again for the contribution!

Hi @kanderso-nrel , thank you for your help.

After a bit of time and testing, I finally managed to adapt the code and pass the Stickler CI test.

It is a pleasure for me to contribute to this wonderful project.

I remain at your disposal for any further issues.

kandersolar · 2023-05-19T00:24:35Z

Thanks @Jacc0027 for the effort here! I think I see how to get the tests working; would you mind if I pushed some commits to this PR for that? Then after a bit of documentation I think this PR will be ready.

Jacc0027 · 2023-05-19T18:16:02Z

Thanks @Jacc0027 for the effort here! I think I see how to get the tests working; would you mind if I pushed some commits to this PR for that? Then after a bit of documentation I think this PR will be ready.

Hi @kandersolar , no problem. Go ahead with the solution. Thanks for the support!

kandersolar · 2023-05-19T18:50:36Z

Looks like things are working! I think these are the last main things, @Jacc0027 can you take a look:

Fill in the two "TODO" parameter descriptions
Add an entry in /docs/sphinx/source/reference/airmass_atmospheric.rst for this new function
Add a new item in /docs/sphinx/source/whatsnew/v0.9.6.rst for this enhancement, and yourself as a contributor at the bottom

Updated both "TODO" issues.

Jacc0027 · 2023-05-19T19:40:57Z

Looks like things are working! I think these are the last main things, @Jacc0027 can you take a look:

Fill in the two "TODO" parameter descriptions

Add an entry in /docs/sphinx/source/reference/airmass_atmospheric.rst for this new function

Add a new item in /docs/sphinx/source/whatsnew/v0.9.6.rst for this enhancement, and yourself as a contributor at the bottom

Thanks @kandersolar you did it!. I have done the 3 things you've requested me. Please, let me know if you need something else.

kandersolar · 2023-05-19T20:23:59Z

Hmm, for some reason this PR is not showing any changes to the reference and whatsnew pages: https://github.com/pvlib/pvlib-python/pull/1296/files

Maybe you only changed them locally but did not include them in a commit?

Jacc0027 · 2023-05-20T07:49:50Z

Hmm, for some reason this PR is not showing any changes to the reference and whatsnew pages: https://github.com/pvlib/pvlib-python/pull/1296/files

Maybe you only changed them locally but did not include them in a commit?

I requested to update both files after modifying them via Github online. After requesting the Commit of the changes, I got an option to create a PR for them. Should I create these new PRs?

kandersolar · 2023-05-20T13:47:55Z

It would be better to include them in this PR so that all the changes are in one place. If you are editing them with Github online, then I think you can add the changes to this PR by editing the files on the specific git branch associated with this PR (the Spectral-corrections branch in your fork of pvlib). For example this is the link for the "what's new" file on the PR branch: https://github.com/Jacc0027/pvlib-python/blob/Spectral-corrections/docs/sphinx/source/whatsnew/v0.9.6.rst

If you edit that file and go to commit the change, it should give you the option "Commit directly to the Spectral-corrections branch", and then I think the commit will show up in this PR.

adriesse

I have two very old comments that show up as pending. Does that mean nobody else saw them? I'm going to click on submit now but will take another look when I get back to the office next week.

adriesse · 2021-11-11T18:55:33Z

pvlib/atmosphere.py

+    as a function of AM, AOD and PW was performed according to [2] and [3].
+    As such, the polynomial adjustment coefficients included in [3]
+    were obtained.
+


I think it is very good to have a couple of paragraphs explaining the basics here, but unfortunately I find them hard to understand. If you recorded AM, AOD, PW and spectra, then where does smarts come in? I guess I should read the paper, but this text should also be consistent and comprehensible on its own.

Hi @adriesse, the reason for using synthetic spectra, generated through SMARTS2, was to ascertain the ideal polynomial type of the experimental equations related to the AM, AOD and PW parameters.

adriesse · 2021-11-11T18:57:44Z

pvlib/atmosphere.py

+    Spain for one year synchronously with both, broadband and
+    spectroradiometric measurements of 30º tilted global irradiance
+    south-facing logged in 5-min intervals. AM was estimated through
+    elevation data.


Does this estimation refer to solar elevation? Airmass calculations are pretty standard.

Hi @adriesse, yes it does. The solar elevation was measured by means of a solar spectral irradiance meter (SolarSIM-D2) from Spectrafy Inc. Then the AM was computed by using the sun’s zenith angle (z). Reference: F. Kasten and A. T. Young, “Revised optical air mass tables and approxi-mation formula,” Appl. Opt., vol. 28, pp. 4735–4738, Nov. 15, 1989.

My thinking was that your responses might make it into the doc string at some point...

Added caballero spectral correction as a new function.

Added a new function: atmosphere.caballero_spectral_correction

Jacc0027 · 2023-05-23T19:45:39Z

It would be better to include them in this PR so that all the changes are in one place. If you are editing them with Github online, then I think you can add the changes to this PR by editing the files on the specific git branch associated with this PR (the Spectral-corrections branch in your fork of pvlib). For example this is the link for the "what's new" file on the PR branch: https://github.com/Jacc0027/pvlib-python/blob/Spectral-corrections/docs/sphinx/source/whatsnew/v0.9.6.rst

If you edit that file and go to commit the change, it should give you the option "Commit directly to the Spectral-corrections branch", and then I think the commit will show up in this PR.

Hi @kandersolar, done. Thank you for the guidelines!

pvlib/atmosphere.py

Co-authored-by: Anton Driesse <anton.driesse@pvperformancelabs.com>

following the conventions set out in pvlib#1628

- reorder and rename parameters to match pvlib#1768 - remove reference parameters - shorten/simplify docstring - restructure calculation to be more readable

kandersolar · 2023-06-12T14:58:20Z

Hi @Jacc0027, I just added a few changes. The main one is to move the new function to pvlib.spectrum, since we are consolidating the spectral modifier functions to that module (see #1628). I also made some changes to the function to better match typical pvlib style, and address @adriesse's comments above.

I think this PR is ready to merge, but I'll leave it open for a few days to allow a chance for anyone else to review again.

Jacc0027 · 2023-06-18T21:31:17Z

Hi @Jacc0027, I just added a few changes. The main one is to move the new function to pvlib.spectrum, since we are consolidating the spectral modifier functions to that module (see #1628). I also made some changes to the function to better match typical pvlib style, and address @adriesse's comments above.

I think this PR is ready to merge, but I'll leave it open for a few days to allow a chance for anyone else to review again.

Thank you @kandersolar .

Hi @Jacc0027, I just added a few changes. The main one is to move the new function to pvlib.spectrum, since we are consolidating the spectral modifier functions to that module (see #1628). I also made some changes to the function to better match typical pvlib style, and address @adriesse's comments above.

I think this PR is ready to merge, but I'll leave it open for a few days to allow a chance for anyone else to review again.

Hi @kandersolar, thank you so much for the effort! There have been two major solar energy events during the last week: Intersolar Europe 2023 and PVSC50. Perhaps during the week ahead there will be some additional comments. We'll see.

adriesse · 2023-06-19T11:19:41Z

pvlib/spectrum/mismatch.py

+    f_AM = (
+        coeff[0]
+        + coeff[1] * ama
+        + coeff[2] * ama**2
+        + coeff[3] * ama**3
+        + coeff[4] * ama**4
+    )
+    # Eq 6, with Table 1
+    f_AOD = (aod500 - aod500_ref) * (
+        coeff[5]
+        + coeff[10] * coeff[6] * ama
+        + coeff[11] * coeff[6] * np.log(ama)
+        + coeff[7] * ama**2
+    )
+    # Eq 7, with Table 1
+    f_PW = (precipitable_water - pw_ref) * (
+        coeff[8]
+        + coeff[9] * np.log(ama)
+    )


There are a several options for making this more efficient: Horner's method, np.polyval, pre-calculating ama powers, expanding **3 and **4, adding coefficients before multiplying where applicable. Just thought I'd mention that since it came up in other places in the past. They would all reduce the much-appreciated code clarity to varying degrees, so I'll stop short of making a recommendation.

I felt the same conflict between code clarity and efficiency. I settled in favor of clarity after seeing that this inefficient implementation is still plenty fast by my standards (computes ~10 million values in one second).

Still, it does offend the sensibilities to calculate x**3 on one line and x**4 on the next :(

kandersolar · 2023-06-23T18:09:34Z

Thanks again @Jacc0027 for this contribution, and my apologies that it took nearly two years to get it merged. Better late than never, hopefully :)

Jacc0027 · 2023-07-05T19:06:03Z

Thank you so much @kandersolar, @adriesse, @cwhanse, @mikofski for your support during this time! To a lesser or greater extent, the closing of this contribution has also been possible thanks to your assistance. I would really wish I could have made this contribution in other circumstances with more time available.
See you!

Jacc0027 added 2 commits August 26, 2021 18:47

Update atmosphere.py

f634e39

Update atmosphere.py

b5d7fec

Jacc0027 added 26 commits September 25, 2021 11:14

Update atmosphere.py

eb948e0

Update atmosphere.py

e27c688

Update atmosphere.py

03920a6

Merge branch 'Spectral-corrections' of https://github.com/Jacc0027/pv…

be39d2e

…lib-python into Spectral-corrections

Update atmosphere.py

c9fefeb

Update atmosphere.py

125bc16

Update atmosphere.py

6a43156

Update atmosphere.py

5b890ab

Update atmosphere.py

c602a32

Update atmosphere.py

97d0e65

Update atmosphere.py

0e2d832

Update atmosphere.py

8a473ba

Update atmosphere.py

397c3e8

Update atmosphere.py

7382e36

Update atmosphere.py

ed2d4f6

Update atmosphere.py

394c4bc

Update atmosphere.py

e89248f

Update atmosphere.py

25b43df

Update atmosphere.py

d9f466e

Update atmosphere.py

cdc64fc

Update atmosphere.py

422f49f

Update atmosphere.py

48474a7

Update atmosphere.py

6e9b9d8

Update atmosphere.py

43a188c

Update atmosphere.py

710cdd3

Update atmosphere.py

46519a8

Jacc0027 added 2 commits May 19, 2023 01:13

Update test_atmosphere.py

3ad15bb

Update test_atmosphere.py

c314d3c

update function logic and get tests working

b56307b

Jacc0027 added 2 commits May 19, 2023 21:14

Update atmosphere.py

8fa44af

Updated both "TODO" issues.

Update atmosphere.py

cda92be

adriesse reviewed May 20, 2023

View reviewed changes

Jacc0027 added 2 commits May 23, 2023 21:21

Update v0.9.6.rst

c872a3d

Added caballero spectral correction as a new function.

Update airmass_atmospheric.rst

3bb26c4

Added a new function: atmosphere.caballero_spectral_correction

adriesse requested changes May 24, 2023

View reviewed changes

pvlib/atmosphere.py Outdated Show resolved Hide resolved

pvlib/atmosphere.py Outdated Show resolved Hide resolved

pvlib/atmosphere.py Outdated Show resolved Hide resolved

Jacc0027 and others added 5 commits May 24, 2023 20:41

Update pvlib/atmosphere.py

b6a6a08

Co-authored-by: Anton Driesse <anton.driesse@pvperformancelabs.com>

Merge branch 'main' into pr/1296

5b10b84

Merge remote-tracking branch 'upstream/main' into pr/1296

440ef38

rename function and move to pvlib.spectrum

f229255

following the conventions set out in pvlib#1628

misc cleanup

4ac3a89

- reorder and rename parameters to match pvlib#1768 - remove reference parameters - shorten/simplify docstring - restructure calculation to be more readable

kandersolar changed the title ~~Update atmosphere.py - Add a model for spectral corrections~~ Implement Caballero et al. spectral factor model Jun 12, 2023

adriesse reviewed Jun 19, 2023

View reviewed changes

Merge branch 'main' into Spectral-corrections

dcf7891

kandersolar merged commit 948d820 into pvlib:main Jun 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Caballero et al. spectral factor model #1296

Implement Caballero et al. spectral factor model #1296

Jacc0027 commented Aug 26, 2021 •

edited

Loading

kandersolar commented Sep 24, 2021

Jacc0027 commented Sep 25, 2021

kandersolar commented May 19, 2023

Jacc0027 commented May 19, 2023

kandersolar commented May 19, 2023

Jacc0027 commented May 19, 2023

kandersolar commented May 19, 2023

Jacc0027 commented May 20, 2023

kandersolar commented May 20, 2023

adriesse left a comment

adriesse Nov 11, 2021

Jacc0027 May 23, 2023

adriesse Nov 11, 2021

Jacc0027 May 23, 2023

adriesse Jun 22, 2023

Jacc0027 commented May 23, 2023

kandersolar commented Jun 12, 2023

Jacc0027 commented Jun 18, 2023

adriesse Jun 19, 2023

kandersolar Jun 19, 2023

kandersolar commented Jun 23, 2023

Jacc0027 commented Jul 5, 2023

Implement Caballero et al. spectral factor model #1296

Implement Caballero et al. spectral factor model #1296

Conversation

Jacc0027 commented Aug 26, 2021 • edited Loading

kandersolar commented Sep 24, 2021

Jacc0027 commented Sep 25, 2021

kandersolar commented May 19, 2023

Jacc0027 commented May 19, 2023

kandersolar commented May 19, 2023

Jacc0027 commented May 19, 2023

kandersolar commented May 19, 2023

Jacc0027 commented May 20, 2023

kandersolar commented May 20, 2023

adriesse left a comment

Choose a reason for hiding this comment

adriesse Nov 11, 2021

Choose a reason for hiding this comment

Jacc0027 May 23, 2023

Choose a reason for hiding this comment

adriesse Nov 11, 2021

Choose a reason for hiding this comment

Jacc0027 May 23, 2023

Choose a reason for hiding this comment

adriesse Jun 22, 2023

Choose a reason for hiding this comment

Jacc0027 commented May 23, 2023

kandersolar commented Jun 12, 2023

Jacc0027 commented Jun 18, 2023

adriesse Jun 19, 2023

Choose a reason for hiding this comment

kandersolar Jun 19, 2023

Choose a reason for hiding this comment

kandersolar commented Jun 23, 2023

Jacc0027 commented Jul 5, 2023

Jacc0027 commented Aug 26, 2021 •

edited

Loading