[llvm-readobj][COFF] Dump hybrid object for ARM64X files.

[cjacek]

Hybrid ARM64X files may theoretically change anything in the object (it may change both the PE headers and each sections), so I just print everything for both the primary and hybrid object. In practice, changing things like section table would be questionable (and would probably fail at runtime), so I considered skipping it for the hybrid objects, but decided against it for consistency.

My primary reason to implement it was to be able to write proper tests for lld-link, but since then it has proven to be neat tool when working with ARM64X files.

[llvmbot]

@llvm/pr-subscribers-llvm-binary-utilities

Author: Jacek Caban (cjacek)

Changes

Hybrid ARM64X files may theoretically change anything in the object (it may change both the PE headers and each sections), so I just print everything for both the primary and hybrid object. In practice, changing things like section table would be questionable (and would probably fail at runtime), so I considered skipping it for the hybrid objects, but decided against it for consistency.

My primary reason to implement it was to be able to write proper tests for lld-link, but since then it has proven to be neat tool when working with ARM64X files.

---

Full diff: https://github.com/llvm/llvm-project/pull/102245.diff

7 Files Affected:

• (modified) llvm/include/llvm/Object/COFF.h (+1)
• (modified) llvm/lib/Object/COFFObjectFile.cpp (+48)
• (modified) llvm/test/tools/llvm-readobj/COFF/arm64x-reloc.yaml (+144)
• (modified) llvm/tools/llvm-readobj/COFFDumper.cpp (+13)
• (modified) llvm/tools/llvm-readobj/ObjDumper.cpp (+4-2)
• (modified) llvm/tools/llvm-readobj/ObjDumper.h (+5)
• (modified) llvm/tools/llvm-readobj/llvm-readobj.cpp (+8-5)

diff --git a/llvm/include/llvm/Object/COFF.h b/llvm/include/llvm/Object/COFF.h
index b084754c5f8fb..dc43c87c4125c 100644
--- a/llvm/include/llvm/Object/COFF.h
+++ b/llvm/include/llvm/Object/COFF.h
@@ -1085,6 +1085,7 @@ class COFFObjectFile : public ObjectFile {
   Expected<SubtargetFeatures> getFeatures() const override {
     return SubtargetFeatures();
   }
+  std::unique_ptr<MemoryBuffer> getHybridObjectView() const;
 
   import_directory_iterator import_directory_begin() const;
   import_directory_iterator import_directory_end() const;
diff --git a/llvm/lib/Object/COFFObjectFile.cpp b/llvm/lib/Object/COFFObjectFile.cpp
index 34347e5681358..f8a177879b319 100644
--- a/llvm/lib/Object/COFFObjectFile.cpp
+++ b/llvm/lib/Object/COFFObjectFile.cpp
@@ -1489,6 +1489,54 @@ StringRef COFFObjectFile::mapDebugSectionName(StringRef Name) const {
       .Default(Name);
 }
 
+std::unique_ptr<MemoryBuffer> COFFObjectFile::getHybridObjectView() const {
+  if (getMachine() != COFF::IMAGE_FILE_MACHINE_ARM64X)
+    return nullptr;
+
+  std::unique_ptr<WritableMemoryBuffer> HybridView;
+
+  for (auto DynReloc : dynamic_relocs()) {
+    if (DynReloc.getType() != COFF::IMAGE_DYNAMIC_RELOCATION_ARM64X)
+      continue;
+
+    for (auto reloc : DynReloc.arm64x_relocs()) {
+      if (!HybridView) {
+        HybridView =
+            WritableMemoryBuffer::getNewUninitMemBuffer(Data.getBufferSize());
+        memcpy(HybridView->getBufferStart(), Data.getBufferStart(),
+               Data.getBufferSize());
+      }
+
+      uint32_t RVA = reloc.getRVA();
+      void *Ptr;
+      uintptr_t IntPtr;
+      if (RVA & ~0xfff) {
+        cantFail(getRvaPtr(RVA, IntPtr));
+        Ptr = HybridView->getBufferStart() + IntPtr -
+              reinterpret_cast<uintptr_t>(base());
+      } else {
+        // PE header relocation.
+        Ptr = HybridView->getBufferStart() + RVA;
+      }
+
+      switch (reloc.getType()) {
+      case COFF::IMAGE_DVRT_ARM64X_FIXUP_TYPE_ZEROFILL:
+        memset(Ptr, 0, reloc.getSize());
+        break;
+      case COFF::IMAGE_DVRT_ARM64X_FIXUP_TYPE_VALUE: {
+        auto Value = static_cast<ulittle64_t>(reloc.getValue());
+        memcpy(Ptr, &Value, reloc.getSize());
+        break;
+      }
+      case COFF::IMAGE_DVRT_ARM64X_FIXUP_TYPE_DELTA:
+        *reinterpret_cast<ulittle32_t *>(Ptr) += reloc.getValue();
+        break;
+      }
+    }
+  }
+  return HybridView;
+}
+
 bool ImportDirectoryEntryRef::
 operator==(const ImportDirectoryEntryRef &Other) const {
   return ImportTable == Other.ImportTable && Index == Other.Index;
diff --git a/llvm/test/tools/llvm-readobj/COFF/arm64x-reloc.yaml b/llvm/test/tools/llvm-readobj/COFF/arm64x-reloc.yaml
index df2d83db3a6a1..851dba17879c4 100644
--- a/llvm/test/tools/llvm-readobj/COFF/arm64x-reloc.yaml
+++ b/llvm/test/tools/llvm-readobj/COFF/arm64x-reloc.yaml
@@ -86,6 +86,32 @@
 # CHECK-NEXT:   ]
 # CHECK-NEXT: ]
 
+# RUN: llvm-readobj --hex-dump=.test %t.dll | FileCheck --check-prefix=HEX %s
+# HEX:      Format: COFF-ARM64X
+# HEX-NEXT: Arch: aarch64
+# HEX-NEXT: AddressSize: 64bit
+# HEX-EMPTY:
+# HEX-NEXT: Hex dump of section '.test':
+# HEX-NEXT: 0x180006000 11112222 33334444 55556666 77778888 ..""33DDUUffww..
+# HEX-NEXT: 0x180006010 9999aaaa bbbbcccc ddddeeee ffff0000 ................
+# HEX-NEXT: 0x180006020 00000000 00000000 00000000 00000000 ................
+# HEX-NEXT: 0x180006030 00000000 00000000 00000000 00000000 ................
+# HEX-NEXT: 0x180006040 10101010 20202020 30303030 40404040 ....    0000@@@@
+# HEX-NEXT: 0x180006050 50505050 60606060 70707070 80808080 PPPP````pppp....
+# HEX-NEXT: HybridObject {
+# HEX-NEXT:   Format: COFF-ARM64EC
+# HEX-NEXT:   Arch: aarch64
+# HEX-NEXT:   AddressSize: 64bit
+# HEX-EMPTY:
+# HEX-NEXT:   Hex dump of section '.test':
+# HEX-NEXT:   0x180006000 00002222 00000000 55556666 77778888 ..""....UUffww..
+# HEX-NEXT:   0x180006010 00000000 00000000 ddddeeee ffff0000 ................
+# HEX-NEXT:   0x180006020 12340000 23456789 11223344 55667788 .4..#Eg.."3DUfw.
+# HEX-NEXT:   0x180006030 00000000 00000000 00000000 00000000 ................
+# HEX-NEXT:   0x180006040 941c1110 28392220 20303030 20404040 ....(9"  000 @@@
+# HEX-NEXT:   0x180006050 50505050 60606060 70707070 80808080 PPPP````pppp....
+# HEX-NEXT: }
+
 
 --- !COFF
 OptionalHeader:
@@ -203,6 +229,7 @@ symbols:         []
 
 # RUN: yaml2obj %s --docnum=2 -o %t2.dll
 # RUN: llvm-readobj --coff-load-config %t2.dll | FileCheck  --check-prefixes=CHECK,V2 %s
+# RUN: llvm-readobj --hex-dump=.test %t2.dll | FileCheck --check-prefix=HEX %s
 
 --- !COFF
 OptionalHeader:
@@ -320,3 +347,120 @@ sections:
       - Binary: 0000      # terminator
 symbols:         []
 ...
+
+# RUN: yaml2obj %s --docnum=3 -o %t3.dll
+# RUN: llvm-readobj --coff-exports %t3.dll | FileCheck  --check-prefix=EXP %s
+
+# EXP:      Format: COFF-ARM64X
+# EXP-NEXT: Arch: aarch64
+# EXP-NEXT: AddressSize: 64bit
+# EXP-NEXT: Export {
+# EXP-NEXT:   Ordinal: 1
+# EXP-NEXT:   Name: test
+# EXP-NEXT:   RVA: 0x2000
+# EXP-NEXT: }
+# EXP-NEXT: HybridObject {
+# EXP-NEXT:   Format: COFF-ARM64EC
+# EXP-NEXT:   Arch: aarch64
+# EXP-NEXT:   AddressSize: 64bit
+# EXP-NEXT:   Export {
+# EXP-NEXT:     Ordinal: 1
+# EXP-NEXT:     Name: test
+# EXP-NEXT:     RVA: 0x2004
+# EXP-NEXT:   }
+# EXP-NEXT: }
+
+--- !COFF
+OptionalHeader:
+  ImageBase:        0x180000000
+  SectionAlignment: 4096
+  FileAlignment:    512
+  DLLCharacteristics: [ ]
+  AddressOfEntryPoint: 0
+  ExportTable:
+    RelativeVirtualAddress: 0x1000
+    Size:                   64
+  LoadConfigTable:
+    RelativeVirtualAddress: 0x3000
+    Size:                   320
+header:
+  Machine:         IMAGE_FILE_MACHINE_ARM64
+  Characteristics: [ IMAGE_FILE_EXECUTABLE_IMAGE, IMAGE_FILE_LARGE_ADDRESS_AWARE, IMAGE_FILE_DLL ]
+sections:
+  - Name:            .rdata
+    Characteristics: [ IMAGE_SCN_CNT_INITIALIZED_DATA, IMAGE_SCN_MEM_READ ]
+    VirtualAddress:  0x1000
+    VirtualSize:     64
+    StructuredData:
+      - UInt32: 0       # ExportFlags
+      - UInt32: 0       # TimeDateStamp
+      - UInt32: 0       # Version
+      - UInt32: 0x1028  # NameRVA
+      - UInt32: 1       # OrdinalBase
+      - UInt32: 1       # AddressTableEntries
+      - UInt32: 1       # NumberOfNamePointers
+      - UInt32: 0x1030  # ExportAddressTableRVA
+      - UInt32: 0x1034  # NamePointerRVA
+      - UInt32: 0x1038  # OrdinalTableRVA
+      - Binary: 7473742E646C6C00  # "tst.dll"
+      - UInt32: 0x2000  # export RVA
+      - UInt32: 0x103A  # name RVA
+      - Binary: 0000    # ordinal
+      - Binary: 7465737400 # "test"
+  - Name:            .data
+    Characteristics: [ IMAGE_SCN_CNT_INITIALIZED_DATA, IMAGE_SCN_MEM_READ, IMAGE_SCN_MEM_WRITE ]
+    VirtualAddress:  0x2000
+    VirtualSize:     80
+    StructuredData:
+      - UInt32: 1       # Version
+      - UInt32: 0       # CodeMap
+      - UInt32: 0       # CodeMapCount
+      - UInt32: 0       # CodeRangesToEntryPoints
+      - UInt32: 0       # RedirectionMetadata
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0       # CodeRangesToEntryPointsCount
+      - UInt32: 0       # RedirectionMetadataCount
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0
+      - UInt32: 0
+  - Name:            .cfg
+    Characteristics: [ IMAGE_SCN_CNT_INITIALIZED_DATA, IMAGE_SCN_MEM_READ ]
+    VirtualAddress:  0x3000
+    VirtualSize:     328
+    StructuredData:
+      - LoadConfig:
+          CHPEMetadataPointer: 0x180002000
+          DynamicValueRelocTableOffset: 0
+          DynamicValueRelocTableSection: 4
+  - Name:            .arm64x
+    Characteristics: [ IMAGE_SCN_CNT_INITIALIZED_DATA, IMAGE_SCN_MEM_READ, IMAGE_SCN_MEM_DISCARDABLE ]
+    VirtualAddress:  0x4000
+    VirtualSize:     56
+    StructuredData:
+      - UInt32: 2         # coff_dynamic_reloc_table.Version
+      - UInt32: 48        # coff_dynamic_reloc_table.Size
+      - UInt32: 24        # coff_dynamic_relocation64_v2.HeaderSize
+      - UInt32: 24        # coff_dynamic_relocation64_v2.FixupInfoSize
+      - UInt32: 6         # coff_dynamic_relocation64_v2.Symbol(low) = IMAGE_DYNAMIC_RELOCATION_ARM64X
+      - UInt32: 0         # coff_dynamic_relocation64_v2.Symbol(high)
+      - UInt32: 0         # coff_dynamic_relocation64_v2.SymbolGroup
+      - UInt32: 0         # coff_dynamic_relocation64_v2.Flags
+      - UInt32: 0         # coff_base_reloc_block_header[0].PageRVA
+      - UInt32: 12        # coff_base_reloc_block_header[0].BlockSize
+      - Binary: 8450      # VALUE offset 0x84 (PE header Machine), size 2
+      - Binary: 6486      # IMAGE_FILE_MACHINE_AMD64
+      - UInt32: 0x1000    # coff_base_reloc_block_header[1].PageRVA
+      - UInt32: 12        # coff_base_reloc_block_header[1].BlockSize
+      - Binary: 3020      # DELTA offset 0x30, mul 4
+      - Binary: 0100
+symbols:         []
+...
diff --git a/llvm/tools/llvm-readobj/COFFDumper.cpp b/llvm/tools/llvm-readobj/COFFDumper.cpp
index 65d67d29a5aa3..cb8750f377416 100644
--- a/llvm/tools/llvm-readobj/COFFDumper.cpp
+++ b/llvm/tools/llvm-readobj/COFFDumper.cpp
@@ -99,6 +99,7 @@ class COFFDumper : public ObjDumper {
   void printCOFFTLSDirectory() override;
   void printCOFFResources() override;
   void printCOFFLoadConfig() override;
+  void printCOFFHybridObject() override;
   void printCodeViewDebugInfo() override;
   void mergeCodeViewTypes(llvm::codeview::MergingTypeTableBuilder &CVIDs,
                           llvm::codeview::MergingTypeTableBuilder &CVTypes,
@@ -2271,3 +2272,15 @@ void COFFDumper::printCOFFTLSDirectory(
                ArrayRef(ImageSectionCharacteristics),
                COFF::SectionCharacteristics(COFF::IMAGE_SCN_ALIGN_MASK));
 }
+
+void COFFDumper::printCOFFHybridObject() {
+  std::unique_ptr<MemoryBuffer> HybridView = Obj->getHybridObjectView();
+  if (!HybridView)
+    return;
+  Expected<std::unique_ptr<COFFObjectFile>> HybridObjOrErr =
+      COFFObjectFile::create(*HybridView);
+  if (!HybridObjOrErr)
+    reportError(HybridObjOrErr.takeError(), Obj->getFileName().str());
+  DictScope D(Writer, "HybridObject");
+  ObjDumper::dumpObject(**HybridObjOrErr, Writer, nullptr, true);
+}
diff --git a/llvm/tools/llvm-readobj/ObjDumper.cpp b/llvm/tools/llvm-readobj/ObjDumper.cpp
index 20e99d9d97f3a..d02d1e543d31d 100644
--- a/llvm/tools/llvm-readobj/ObjDumper.cpp
+++ b/llvm/tools/llvm-readobj/ObjDumper.cpp
@@ -91,8 +91,10 @@ void ObjDumper::printAsStringList(StringRef StringContent,
 void ObjDumper::printFileSummary(StringRef FileStr, object::ObjectFile &Obj,
                                  ArrayRef<std::string> InputFilenames,
                                  const object::Archive *A) {
-  W.getOStream() << "\n";
-  W.printString("File", FileStr);
+  if (!FileStr.empty()) {
+    W.getOStream() << "\n";
+    W.printString("File", FileStr);
+  }
   W.printString("Format", Obj.getFileFormatName());
   W.printString("Arch", Triple::getArchTypeName(Obj.getArch()));
   W.printString("AddressSize",
diff --git a/llvm/tools/llvm-readobj/ObjDumper.h b/llvm/tools/llvm-readobj/ObjDumper.h
index cd744e3bbfb71..ffe6797711c8b 100644
--- a/llvm/tools/llvm-readobj/ObjDumper.h
+++ b/llvm/tools/llvm-readobj/ObjDumper.h
@@ -149,6 +149,7 @@ class ObjDumper {
   virtual void printCOFFTLSDirectory() {}
   virtual void printCOFFResources() {}
   virtual void printCOFFLoadConfig() { }
+  virtual void printCOFFHybridObject() {}
   virtual void printCodeViewDebugInfo() { }
   virtual void
   mergeCodeViewTypes(llvm::codeview::MergingTypeTableBuilder &CVIDs,
@@ -185,6 +186,10 @@ class ObjDumper {
   void reportUniqueWarning(Error Err) const;
   void reportUniqueWarning(const Twine &Msg) const;
 
+  static void dumpObject(object::ObjectFile &Obj, ScopedPrinter &Writer,
+                         const object::Archive *A = nullptr,
+                         bool IsHybrid = false);
+
 protected:
   ScopedPrinter &W;
 
diff --git a/llvm/tools/llvm-readobj/llvm-readobj.cpp b/llvm/tools/llvm-readobj/llvm-readobj.cpp
index 15d838617063b..b3df584a410d9 100644
--- a/llvm/tools/llvm-readobj/llvm-readobj.cpp
+++ b/llvm/tools/llvm-readobj/llvm-readobj.cpp
@@ -360,8 +360,8 @@ createDumper(const ObjectFile &Obj, ScopedPrinter &Writer) {
 }
 
 /// Dumps the specified object file.
-static void dumpObject(ObjectFile &Obj, ScopedPrinter &Writer,
-                       const Archive *A = nullptr) {
+void ObjDumper::dumpObject(ObjectFile &Obj, ScopedPrinter &Writer,
+                           const Archive *A, bool IsHybrid) {
   std::string FileStr =
       A ? Twine(A->getFileName() + "(" + Obj.getFileName() + ")").str()
         : Obj.getFileName().str();
@@ -538,6 +538,9 @@ static void dumpObject(ObjectFile &Obj, ScopedPrinter &Writer,
     Dumper->printStackMap();
   if (opts::PrintStackSizes)
     Dumper->printStackSizes();
+
+  if (!IsHybrid && Obj.isCOFF())
+    Dumper->printCOFFHybridObject();
 }
 
 /// Dumps each object file in \a Arc;
@@ -553,7 +556,7 @@ static void dumpArchive(const Archive *Arc, ScopedPrinter &Writer) {
 
     Binary *Bin = ChildOrErr->get();
     if (ObjectFile *Obj = dyn_cast<ObjectFile>(Bin))
-      dumpObject(*Obj, Writer, Arc);
+      ObjDumper::dumpObject(*Obj, Writer, Arc);
     else if (COFFImportFile *Imp = dyn_cast<COFFImportFile>(Bin))
       dumpCOFFImportFile(Imp, Writer);
     else
@@ -572,7 +575,7 @@ static void dumpMachOUniversalBinary(const MachOUniversalBinary *UBinary,
   for (const MachOUniversalBinary::ObjectForArch &Obj : UBinary->objects()) {
     Expected<std::unique_ptr<MachOObjectFile>> ObjOrErr = Obj.getAsObjectFile();
     if (ObjOrErr)
-      dumpObject(*ObjOrErr.get(), Writer);
+      ObjDumper::dumpObject(*ObjOrErr.get(), Writer);
     else if (auto E = isNotObjectErrorInvalidFileType(ObjOrErr.takeError()))
       reportError(ObjOrErr.takeError(), UBinary->getFileName());
     else if (Expected<std::unique_ptr<Archive>> AOrErr = Obj.getAsArchive())
@@ -618,7 +621,7 @@ static void dumpInput(StringRef File, ScopedPrinter &Writer) {
                dyn_cast<MachOUniversalBinary>(Bin.get()))
     dumpMachOUniversalBinary(UBinary, Writer);
   else if (ObjectFile *Obj = dyn_cast<ObjectFile>(Bin.get()))
-    dumpObject(*Obj, Writer);
+    ObjDumper::dumpObject(*Obj, Writer);
   else if (COFFImportFile *Import = dyn_cast<COFFImportFile>(Bin.get()))
     dumpCOFFImportFile(Import, Writer);
   else if (WindowsResource *WinRes = dyn_cast<WindowsResource>(Bin.get()))

[jh7370]

I don't like this: you're polluting the generic interface with parameters that are specific to one file format. It's okay to check Obj.isCOFF() in this function, and I'm also okay with it becoming an ObjDumper member (if moved to ObjDumper.cpp), but the hybrid object check really needs to be within the COFF-specific code.

One alternative approach might be to make the second dumpObject call, then call printCOFFHybridObject after that call. A variation of this might even be to introduce a new high-level function dumpCOFFHybridObject, much like the existing dumpCOFFImportFile etc that does this. (NB: I haven't given this a thorough thinking through, so this might not quite work, but hopefully it makes the principle clear).

[cjacek]

Thanks for the feedback. I tried moving dumpObject to ObjDumper.cpp, but that would require making a number of opts variables non-static and became a bit messy. Instead, I kept is static and changed it to be more similar to dumpMachOUniversalBinary: I used a separate branch in dumpInput for COFF files, where I call dumpObject twice in case for hybrid objects. That made the change quite simple.

Also, following the suggestion from #102071, I changed tests to use --match-full-lines and --strict-whitespace to make sure that indention works as expected.

[jh7370]

Common code areas look good to me. I'll leave someone else to review the COFF-specific code.

[mstorsjo]

The COFF bits look good to me, thanks James for the review of the rest of it!

[cjacek]

Merged, thanks for reviews!