Macro-based baked data

[robertbastian]

When reviewing, look at:

• icu_globaldata rustdoc
• The usage in icu_list
• This commit that tries to split data into separate files again, which does not work because macro-lookup in icu_testdata (or anywhere that include!s baked data) cannot use the $crate prefix, but macro-lookup in icu_globaldata requires it

Turn off deleted files in the diff view to make it not lag.

#2743 #2945

[robertbastian]

I've rescoped this PR, it now only changes the baked format.

[sffc]

Question: why the rename and re-export?

[robertbastian]

Because macros are weird. macro_export will put them at the root of the crate, so we hide that, and pub use respects module structure. This is explained in a comment in line 465.

[sffc]

Here's an example of what it generates for a single key

#[doc(hidden)]
#[macro_export]
macro_rules! __singleton_calendar_japanese_v1 {
    () => {
        icu_calendar::provider::JapaneseErasV1 {
            dates_to_eras: unsafe { zerovec::ZeroVec::from_bytes_unchecked(b"blahblahblah") },
        }
    }
}

#[doc(hidden)]
pub use __singleton_calendar_japanese_v1 as singleton_calendar_japanese_v1;

#[doc = " Implement [`DataProvider<JapaneseErasV1Marker>`](icu_provider::DataProvider) on the given struct using the data"]
#[doc = r" hardcoded in this file. This allows the struct to be used with"]
#[doc = r" `icu`'s `_unstable` constructors."]
#[doc(hidden)]
#[macro_export]
macro_rules! __impl_calendar_japanese_v1 {
    ($provider: path) => {
        # [clippy :: msrv = "1.61"]
        impl icu_provider :: DataProvider < icu_calendar :: provider :: JapaneseErasV1Marker > for $ provider {
            fn load (& self , req : icu_provider :: DataRequest ,) -> Result < icu_provider :: DataResponse < icu_calendar :: provider :: JapaneseErasV1Marker > , icu_provider :: DataError > {
                req . locale . is_empty ()
                    . then (|| {
                        static ANCHOR : < icu_calendar :: provider :: JapaneseErasV1Marker as icu_provider :: DataMarker > :: Yokeable = singleton_calendar_japanese_v1 ! () ; & ANCHOR
                    }) . map (icu_provider :: prelude :: zerofrom :: ZeroFrom :: zero_from)
                    . map (icu_provider :: DataPayload :: from_owned)
                    . map (| payload | {
                        icu_provider :: DataResponse { metadata : Default :: default () , payload : Some (payload) , }
                    }) . ok_or_else (|| icu_provider :: DataErrorKind :: MissingLocale . with_req (< icu_calendar :: provider :: JapaneseErasV1Marker as icu_provider :: KeyedDataMarker > :: KEY , req))
            }
        }
    }
}

#[doc(inline)]
pub use __impl_calendar_japanese_v1 as impl_calendar_japanese_v1;

[Manishearth]

I'm fine with this landing. I have a couple concerns:

• I would eventually like these files to get nicely rustfmted again. It may mean moving off of quote for the final construction of code.
  • Reasoning: I have found myself looking at baked data when debugging (and I expect people unfamiliar with our architecture will do this even more often). It's useful. If we're using macros, it's going to be even more useful; people want to look at macro source to figure out how to invoke it.
• I would like this to be one-file-per-macro. I recognize that this is not easy to achieve, but I am skeptical that there is no way to do so.
  • Reasoning: Smaller files, easier to read when necessary. More importantly keeps the door open for multi-crate solutions, and patching a key is far cleaner on people who vendor our code.

[robertbastian]

I would eventually like these files to get nicely rustfmted again. It may mean moving off of quote for the final construction of code.
I would like this to be one-file-per-macro. I recognize that this is not easy to achieve, but I am skeptical that there is no way to do so.

Me too. As discussed I'll try to make this work after globaldata when the constraints are clearer

patching a key is far cleaner on people who vendor our code.

Oh no we should not support that. Just regen.

[Manishearth]

@robertbastian not manually patching, but even in a regen you want manageable diffs

[sffc]

Thanks for working on this!