Xcode project corruption due to UUID conflict

[jmkk]

We run into a race condition/UUID conflict that seems to be related to changes made in #627

Our sizeable project uses Cocoapods, and when testing with 1.7.0.beta.2 we run into an issue where a specific number of files included in the Pods project resulted in an apparent UUID conflict, which results in a corrupted Pods.xcodeproj generated.

The issue seems to reproduce only when:

• Pods project getting constructed in the memory has a specific number of files (meaning adding or removing a file fixes the issue)
• Before saving the Pods project, we add a new file ref in post-install hook:
  common_ref = project.new_file("Some.common.xcconfig")

After the project has been saved, the newly added file reference gets assigned a already existing UUID (46EB2E00000000) and replaces rootObject. With this, the project gets corrupted as root object no longer points to an expected data type (PBXFileReference instead of PBXProject).

% xcodeproj show Pods/Pods.xcodeproj
Traceback (most recent call last):
    7: from /Users/jsuliga/.rvm/gems/ruby-2.6.1/bin/xcodeproj:23:in `<main>'
    6: from /Users/jsuliga/.rvm/gems/ruby-2.6.1/bin/xcodeproj:23:in `load'
    5: from /Users/jsuliga/.rvm/gems/ruby-2.6.1/gems/xcodeproj-1.8.2/bin/xcodeproj:10:in `<top (required)>'
    4: from /Users/jsuliga/.rvm/gems/ruby-2.6.1/gems/claide-1.0.2/lib/claide/command.rb:334:in `run'
    3: from /Users/jsuliga/.rvm/gems/ruby-2.6.1/gems/xcodeproj-1.8.2/lib/xcodeproj/command/show.rb:46:in `run'
    2: from /Users/jsuliga/.rvm/gems/ruby-2.6.1/gems/xcodeproj-1.8.2/lib/xcodeproj/command.rb:60:in `xcodeproj'
    1: from /Users/jsuliga/.rvm/gems/ruby-2.6.1/gems/xcodeproj-1.8.2/lib/xcodeproj/project.rb:112:in `open'
/Users/jsuliga/.rvm/gems/ruby-2.6.1/gems/xcodeproj-1.8.2/lib/xcodeproj/project.rb:231:in `initialize_from_file': undefined method `product_ref_group' for #<Xcodeproj::Project::Object::PBXFileReference:0x00007fb77bee82e0> (NoMethodError)

I'm still trying to find a self-contained project that reproduces the issue, but it's been quite challenging. Saving the project before adding the new reference seems to work around the issue, but causes us to double-save the project which is wasteful.

[sebastianv1]

@jmkk Does your project have deterministic_uuids enabled? The PR you linked should only affect projects with that option enabled.

How do you reference project in your post-install hook? Is it from installer or some other means (i.e Xcodeproj::Project#open or Pod::Project#open)

[jmkk]

We have deterministic_uuids disabled. The logic in the PR seems to be kicking off for that case. When enabling deterministic_uuids I cannot reproduce this issue.

We're referencing the project from installer.pods_project.

[jmkk]

@sebastianv1 any updates? I'll be happy to provide more context as needed. Thank you!

[sebastianv1]

@jmkk No update here. I haven't been able to reproduce this issue and a sample project would help a lot.

If you can't get a repro project, do you mind getting the raw number of objects from a project that breaks. You can do this through irb and using the Xcodeproj gem to open the project and then grab the count from #objects. You mentioned it was specific to a number of objects, so if there is some magic number where it breaks it might help me reproduce.

[jmkk]

I just verified and I see the issue persisted in Cocoapods 1.7.0 beta 3, but I do not see it reproduce in version 1.7.0.rc.1 and later. So whatever it was (be it in Cocoapods or Xcodeproj 1.9.0 release) it seems fixed (or the race conditions changed...)

[jmkk]

Spoke too soon. At a different commit # for our project, 1.7.1 still is broken the same way. I will spend more time trying to reproduce with a test project.

[jmkk]

@sebastianv1 Here's how I hope you can reproduce the issue using your large project. Just add this to your post_install loop hook:

  installer.generated_projects.each do |proj|
    xcconfigsPath = "../../../build/buildSettings/xcconfigs"
    common_ref = proj.new_file("#{xcconfigsPath}/X.common.xcconfig")
    debug_ref = proj.new_file("#{xcconfigsPath}/X.debug.xcconfig")
    release_ref = proj.new_file("#{xcconfigsPath}/X.release.xcconfig")
    warnings_ref = proj.new_file("#{xcconfigsPath}/X.common.warnings.xcconfig")

    proj.build_configurations.each do |bc|
      if bc.name == 'Release'
        bc.base_configuration_reference = release_ref
      else
        bc.base_configuration_reference = debug_ref
      end
    end
  end

(The paths referenced there do not matter, and you do not need to have these xcconfigs files in the right locations - the refs will be broken but that’s ok for this purpose).

Make sure generate_multiple_pod_projects option is on and do a clean pod install (not incremental). For our project which has over 200 pods, a handful of generated individual projects are corrupted. In Xcode they show without the expansion icon, and you cannot open them. The broken project files all have the same issue with the root object pointing to one of the xcconfig refs added in that post_install step.

Hope this helps a bit.

[dnkoutso]

@sebastianv1 lets try to review this again this week?

[jmkk]

Any updates? @dnkoutso @sebastianv1 Thank you

[dnkoutso]

not much from me, currently on vacation in August.

[sebastianv1]

@jmkk I've tried plugging your post install hook into one of our larger projects and couldn't seem to repro. A reproducible project is ideal here to get this fixed.

Based on your description above though it still seems to be a magic number causing this bug (since it disappeared then reappeared). One step that might help debug is if you can the use xcodeproj gem to open the project and post the total size of objects in the project.

[jmkk]

@sebastianv1 I've spent some more time debugging this and this is what I've found - at least this is my understanding based on my limited familiarity with Ruby.

TL;DR the issue can be reproduced when adding new file references to the project in post-install, when the number of existing file refs almost exhausts available_uuids in the given Project object. In my particular case when the number of available_uuids is 3, and I'm adding 4 file references as quoted above, the 4th one will get assigned a previously existing UUID. The sequence of events seems to be:

1. During a Pod project generation, uuids get generated sequentially using Cocoapod's override for generate_available_uuid_list. It continuously keeps expanding generated_uuids collection, which is important for the sequential generation part (array's size is used to ensure no conflicts).

2. At the end of the Pod project generation phase, stabilize_target_uuids is called, which calls to TargetUUIDGenerator's generate function. In there, among other things, generated_uuids instance variable is mutated:

Xcodeproj/lib/xcodeproj/project/uuid_generator.rb

Line 23 in 15c9005

project.instance_variable_set(:@generated_uuids, project.instance_variable_get(:@available_uuids))

3. This line of code seems to be effectively binding generated_uuids instance variable to available_uuids since it is an object reference, so all mutations made to available_uuids moving forward will have same impact on the generated_uuids. Since available_uuids gets reduced in size for every generate_uuid call - it in turn impacts generated_uuids in the same way. Not to mention that available_uuids will be smaller than generated_uuids in that assignment anyway, so even when changed to a copy it would still break further down the road.

4. So the next time generate_available_uuid_list is called (again, the Cocoapod's override, not the base class implementation), the size of generated_uuids array would be smaller than earlier calls, which will make the function generate duplicate UUIDs.

https://github.com/CocoaPods/CocoaPods/blob/24af5c6a691083d7a8f18594edf0d426ed0f5cff/lib/cocoapods/project.rb#L70

What purpose does the @generated_uuids to @available_uuids assignment serve in UUIDGenerator?

[sebastianv1]

@segiddins Do you remember why the line @jmkk inlined above: project.instance_variable_set(:@generated_uuids, project.instance_variable_get(:@available_uuids)) was important to set during UUID generation? The particular line existed before I generalized the code for multi project generation, but I don't recall if there was a gotcha here that we had to account for.

In order to fix this in the short term, maybe we can add another installation flag to disable stabilizing target UUIDs since @jmkk 's project already disabled deterministic UUIDs. We would just have to add another validation to incremental installation flag. Thoughts @dnkoutso ?

Sorry this is taking so long to resolve @jmkk and thanks for being so patient.

[dnkoutso]

@jmkk if #627 is reverted locally do you experience the same issue? Not planning on reverting it but I wanted to narrow down the issue to #627 or not.

[jmkk]

At the time of reporting the issue I did confirm that without #627 things worked fine. It was a while ago so I do not remember all the details. The code changed enough since then that reverting from master today is not trivial.

[LWX124]

I have the same issue in cocoapods 1.8.4.
Calling new_group , new_reference, new_file in post-install hooks will cause this issue.
It begins from version 1.7.0.beta.1, I've tried the following:
file: installer.rb
method: create_and_save_projects
`

    predictabilize_uuids(generated_projects) if installation_options.deterministic_uuids?

    run_podfile_post_install_hooks

    stabilize_target_uuids(generated_projects)

`
put "stabilize_target_uuids" after post install hook
It works for our project, but, I've found that somebody need uuid in post-install hooks.
It's related to commit(CocoaPods/CocoaPods@7030b34)
hope for better solution

[cuongvoong]

I just spent all day debugging this issue and found this post. The problem still exists in cocoapods 1.15.2. When adding 4 package_references using post_install, the pbxproj becomes corrupted. When only adding 3, it is working as expected.