allocating frames

In order to create new page tables, we need to create a proper frame
allocator. For that we use the `memory_map` that is passed by the
bootloader as part of the `BootInfo` struct.

Before we implement the `FrameAllocator` trait, we add an auxiliary
`usable_frames` method  that converts the memory map into an
iterator of usable frames.

We use the `BootInfoFrameAllocator` in `main.rs`. We modify
our `kernel_main` function to pass a `BootInfoFrameAllocator`
instance instead of an `EmptyFrameAllocator`.

src/main.rs

@@ -19,7 +19,7 @@ fn kernel_main(boot_info: &'static BootInfo) -> ! {
         structures::paging::Page,
         VirtAddr,
     }; // need to import the `Translate` trait in order to use the `translate_addr` method it provides.
-    use tiny_os::memory;
+    use tiny_os::memory::{ self, BootInfoFrameAllocator };
 
     // Write some characters to the screen.
     print!("H");
@@ -81,7 +81,9 @@ fn kernel_main(boot_info: &'static BootInfo) -> ! {
 
     let phys_mem_offset = VirtAddr::new(boot_info.physical_memory_offset);
     let mut mapper = unsafe { memory::init(phys_mem_offset) };
-    let mut frame_allocator = memory::EmptyFrameAllocator;
+    // With the `BootInfoFrameAllocator`, behind the scenes, the `map_to` method
+    // creates the missing page tables.
+    let mut frame_allocator = unsafe { BootInfoFrameAllocator::init(&boot_info.memory_map) };
     // Map an unused page.
     // This maps the page to the VGA text buffer frame, so we should see any
     // write to it on the screen.
@@ -187,6 +189,25 @@ fn trivial_assertion() {
 
 // ********** Sidenote **********
 // 
+// # Boot information
+//
+// The `bootloader` crate defines a `BootInfo` struct that contains all the
+// information it passes to our kernel. With the `map_physical_memory` feature
+// enabled, it currently has the two fields `memory_map` and
+// `physical_memory_offset`:
+//
+// - The `memory_map` field contains an overview of the available physical
+//   memory. This tells our kernel how much physical memory is available in the
+//   system and which memory regions are reserved for devices such as the VGA
+//   hardware. The memory map can be queried from the BIOS or UEFI firmware, but
+//   only very early in the boot process. For this reason, it must be provided
+//   by the bootloader because there is no way for the kernel to retrieve it
+//   later.
+// - The `physical_memory_offset` tells us the virtual start address of the
+//   physical memory mapping. By adding this offset to a physical address, we
+//   get the corresponding virtual address. This allows us to access arbitrary
+//   physical memory from our kernel.
+//
 // # The `entry_point` macro
 //
 // Since our `_start` function is called externally from the bootloader, no

src/memory.rs

@@ -17,6 +17,7 @@ use x86_64::{
     },
     VirtAddr, PhysAddr,
 };
+use bootloader::bootinfo::{ MemoryMap, MemoryRegionType };
 
 /// Initialize a new `OffsetPageTable`.
 ///
@@ -119,6 +120,78 @@ unsafe impl FrameAllocator<Size4KiB> for EmptyFrameAllocator {
     }
 }
 
+/// A `FrameAllocator` that returns usable frames from the bootloader's memory
+/// map.
+pub struct BootInfoFrameAllocator {
+    /// A `'static` reference to the memory map passed by the bootloader.
+    memory_map: &'static MemoryMap,
+    /// Keeps track of number of the next frame that the allocator should
+    /// return.
+    next: usize,
+}
+
+impl BootInfoFrameAllocator {
+    /// Create a `FrameAllocator` from the passed memory map.
+    ///
+    /// This function is unsafe because the caller must guarantee that the
+    /// passed memory map is valid. The main requirement is that all frames that
+    /// are marked as `USABLE` in it are really unused.
+    pub unsafe fn init(memory_map: &'static MemoryMap) -> Self {
+        BootInfoFrameAllocator {
+            memory_map,
+            // Initialized with 0 and will be increased for every frame
+            // allocation to avoid returning the same frame twice.
+            next: 0,
+        }
+    }
+
+    /// An auxiliary method that returns an iterator over the usable frames
+    /// specified in the memory map.
+    fn usable_frames(&self) -> impl Iterator<Item = PhysFrame> {
+        // Get usable regions from memory map.
+        //
+        // Note: The `iter` method convert the memory map to an iterator of
+        // `MemoryRegions`. The `filter` method to skip any reserved or
+        // otherwise unavailable regions. The bootloader updates the memory map
+        // for all the mappings it creates, so frames that are used by our
+        // kernel (code, data or stack) or to store the boot information are
+        // already marked as InUse or similar. Thus we can be sure that Usable
+        // frames are not used somewhere else.
+        let regions = self.memory_map.iter();
+        let usable_regions = regions
+            .filter(|r| r.region_type == MemoryRegionType::Usable);
+        // Map each region to its address range.
+        //
+        // Note: `map` combinator transform our iterator of memory regions to an
+        // iterator of address ranges.
+        //
+        // `start_addr` method returns the physical start address of the memory
+        // region.
+        let addr_ranges = usable_regions
+            .map(|r| r.range.start_addr()..r.range.end_addr());
+        // Transform to an iterator of frame start addresses.
+        //
+        // Note: `flat_map` to transform the address ranges into an iterator of
+        // frame start addresses, choosing every 4096th address using `step_by`.
+        // Since 4096 bytes (= 4 KiB) is the page size, we get the start address
+        // of each frame. The bootloader page aligns all usable memory areas so
+        // that we don’t need any alignment or rounding code here.
+        let frame_addresses = addr_ranges.flat_map(|r| r.step_by(4096));
+        // Create `PhysFrame` types from the start addresses.
+        frame_addresses.map(|addr| PhysFrame::containing_address(PhysAddr::new(addr)))
+    }
+}
+
+unsafe impl FrameAllocator<Size4KiB> for BootInfoFrameAllocator {
+    fn allocate_frame(&mut self) -> Option<PhysFrame> {
+        let frame = self.usable_frames().nth(self.next);
+        // Before returning that frame, we increase `self.next` by one so that
+        // we return the following frame on the next call.
+        self.next += 1;
+        frame
+    }
+}
+
 /*
 
 /// Translates the given virtual address to the mapped physical address, or
@@ -185,4 +258,38 @@ fn translate_addr_inner(addr: VirtAddr, physical_memory_offset: VirtAddr)
     Some(frame.start_address() + u64::from(addr.page_offset()))
 }
 
-*/
+*/
+
+// ********** Sidenote **********
+// 
+// # Allocating frames
+//
+// The memory map is passed by the bootloader. It is provided by the BIOS/UEFI
+// firmware. It can only be queried very early in the boot process, so the
+// bootloader already calls the respective functions for us.
+//
+// The memory map is a map of the physical memory regions of the underlying
+// machine. Thus it consists of a list of `MemoryRegion` structs, which contain
+// the start address, the length, and the type (e.g. unused, reserved, etc.) of
+// each memory region.
+//
+// ## Implementing the `FrameAllocator` trait
+//
+// This implementation is not quite optimal since it recreates the
+// `usable_frame` allocator on every allocation. It would be better to directly
+// store the iterator as a struct field instead. Then we wouldn’t need the `nth`
+// method and could just call `next` on every allocation. The problem with this
+// approach is that it’s not possible to store an `impl Trait` type in a struct
+// field currently. It might work someday when [named existential
+// types](https://github.com/rust-lang/rfcs/pull/2071) are fully implemented.
+//
+// With the boot info frame allocator, the mapping succeeds. Behind the scenes,
+// the `map_to` method creates the missing page tables in the following way:
+// - Allocate an unused frame from the passed `frame_allocator`.
+// - Zero the frame to create a new, empty page table.
+// - Map the entry of the higher level table to that frame.
+// - Continue with the next table level.
+//
+// While our `create_example_mapping` function is just some example code, we are
+// now able to create new mappings for arbitrary pages. This will be essential
+// for allocating memory or implementing multithreading in future.