HACKS for testing

Change-Id: I3f5405137cd8c5d561a166252955906db63d06e7

docs/objects.md

@@ -35,6 +35,7 @@ Magenta actively manages the following resources:
 ### Memory and address space
 + [Virtual Memory Object](objects/vm_object.md)
 + [Virtual Memory Address Region](objects/vm_address_region.md)
++ [bus_transaction_initiator](objects/bus_transaction_initiator.md)
 
 ### Waiting
 + [Port](objects/port.md)

docs/objects/bus_transaction_initiator.md

@@ -0,0 +1,36 @@
+# Bus Transaction Initiator
+
+## NAME
+
+bus_transaction_initiator - DMA configuration capability
+
+## SYNOPSIS
+
+Bus Transaction Initiators (BTIs) represent the bus mastering/DMA capability
+of a device, and can be used for granting a device access to memory.
+
+## DESCRIPTION
+
+Device drivers are provided one BTI for each bus transaction ID each of its
+devices can use.  A bus transaction ID in this context is a hardware transaction
+identifier that may be used by an IOMMU (e.g. PCI addresses on Intel's IOMMU
+and StreamIDs on ARM's SMMU).
+
+A BTI can be used to pin and unpin memory used in a Virtual Memory Object (VMO).
+If a caller pins memory from a VMO, they are given device-physical addresses
+that can be used to issue memory transactions to the VMO (provided the
+transaction has the correct bus transaction ID).  If transactions affecting
+these addresses are issued with a different transaction ID, the transaction
+may fail and the issuing device may need a reset in order to continue functioning.
+
+TODO(teisenbe): Add details about failed transaction notification.
+
+## SEE ALSO
+
++ [vm_object](vm_object.md) - Virtual Memory Objects
+
+## SYSCALLS
+
++ [bti_create](../syscalls/bti_create.md) - create a new bus transaction initiator
++ [bti_pin](../syscalls/bti_pin.md) - pin memory and grant access to it to the BTI
++ [bti_unpin](../syscalls/bti_unpin.md) - revoke access and unpin memory

docs/syscalls/bti_create.md

@@ -0,0 +1,46 @@
+# mx_bti_create
+
+## NAME
+
+bti_create - create a new bus transaction initiator
+
+## SYNOPSIS
+
+```
+#include <magenta/syscalls.h>
+
+mx_status_t mx_bti_create(mx_handle_t iommu, uint64_t bti_id, mx_handle_t* out);
+
+```
+
+## DESCRIPTION
+
+**bti_create**() creates a new [bus transaction initiator](../objects/bus_transaction_initiator.md)
+given a handle to an IOMMU and a hardware transaction identifier for a device
+downstream of that IOMMU.
+
+Upon success a handle for the new BTI is returned.  This handle will have rights
+**MX_RIGHT_READ**, **MX_RIGHT_MAP**, **MX_RIGHT_DUPLICATE**, and
+**MX_RIGHT_TRANSFER**.
+
+## RETURN VALUE
+
+**bti_create**() returns MX_OK and a handle to the new BTI
+(via *out*) on success.  In the event of failure, a negative error value
+is returned.
+
+## ERRORS
+
+**MX_ERR_BAD_HANDLE**  *iommu_resource* is not a valid handle.
+
+**MX_ERR_WRONG_TYPE**  *iommu_resource* is not a resource handle.
+
+**MX_ERR_INVALID_ARGS**  *bti_id* is invalid on the given I/O MMU,
+or *out* is an invalid pointer.
+
+**MX_ERR_NO_MEMORY**  (Temporary) Failure due to lack of memory.
+
+## SEE ALSO
+
+[bti_pin](bti_pin.md),
+[bti_unpin](bti_unpin.md).

docs/syscalls/bti_pin.md

@@ -0,0 +1,70 @@
+# mx_bti_pin
+
+## NAME
+
+bti_pin - pin pages and grant devices access to them
+
+## SYNOPSIS
+
+```
+#include <magenta/syscalls.h>
+
+mx_status_t mx_bti_pin(mx_handle_t bti, mx_handle_t vmo, uint64_t offset,
+                       uint64_t size, uint32_t perms, uint64_t* addrs,
+                       uint32_t addrs_len, uint32_t* actual_addrs_len);
+```
+
+## DESCRIPTION
+
+**bti_pin**() pins pages of a VMO (i.e. prevents them from being decommitted)
+and grants the hardware transaction ID represented by the BTI the ability to
+access these pages, with the permissions specified in *perms*.
+
+*offset* must be aligned to page boundaries.  *perms* is a bitfield
+that may contain one or more of *MX_VM_FLAG_PERM_READ*, *MX_VM_FLAG_PERM_WRITE*,
+and *MX_VM_FLAG_PERM_EXECUTE*.  In order for the call to succeed, *vmo* must
+have the READ/WRITE/EXECUTE rights corresponding to the flags set in *perms*.
+
+*addrs* will be populated with the device-physical addresses of the requested
+VMO pages.  These addresses may be given to devices that issue memory
+transactions with the hardware transaction ID associated with the BTI.  The
+maximum number of addresses returned is one for each page in the VMO range, so
+*actual_addrs_len* will be at most *size*/PAGE_SIZE, rounded up.  If the VMO
+was contiguous memory, only one address will be returned, the start of the range.
+
+## RETURN VALUE
+
+On success, **bti_pin**() returns MX_OK.  The device-physical addresses of the
+requested VMO pages will be written in *addrs* and the number of entries will
+be written in *actual_addrs_len*.
+
+In the event of failure, a negative error value is returned.
+
+## ERRORS
+
+**MX_ERR_BAD_HANDLE**  *bti* or *vmo* is not a valid handle.
+
+**MX_ERR_WRONG_TYPE**  *bti* is not a BTI handle or *vmo* is not a VMO handle.
+
+**MX_ERR_ACCESS_DENIED** *bti* or *vmo* does not have the *MX_RIGHT_MAP*, or
+*perms* contained a flag corresponding to a right that *vmo* does not have.
+
+**MX_ERR_NOT_FOUND**  The requested range contains at least one uncommitted page.
+
+**MX_ERR_BUFFER_TOO_SMALL** *addrs* is not big enough to hold the result.
+
+**MX_ERR_INVALID_ARGS** *perms* is 0 or contains an undefined flag, or *addrs* or
+*actual_addrs_len* is not a valid pointer, or *offset* is not page-aligned.
+
+**MX_ERR_ALREADY_BOUND** The requested range contains a page already pinned by this
+BTI.
+
+**MX_ERR_UNAVAILABLE** (Temporary) At least one page in the requested range could
+not be pinned at this time.
+
+**MX_ERR_NO_MEMORY**  (Temporary) Failure due to lack of memory.
+
+## SEE ALSO
+
+[bti_create](bti_create.md),
+[bti_unpin](bti_unpin.md).

docs/syscalls/bti_unpin.md

@@ -0,0 +1,42 @@
+# mx_bti_unpin
+
+## NAME
+
+bti_unpin - unpin pages and revoke device access to them
+
+## SYNOPSIS
+
+```
+#include <magenta/syscalls.h>
+
+mx_status_t mx_bti_unpin(mx_handle_t bti, uint64_t* addrs, uint32_t addrs_len);
+```
+
+## DESCRIPTION
+
+**bti_unpin**() unpins pages that were previously pinned by **mx_bti_pin**(),
+and revokes the access that was granted by the pin call.
+
+*addrs* and *addrs_len* must refer to an array with exactly the same elements
+that were returned by a call to **mx_btin_pin**().
+
+## RETURN VALUE
+
+On success, **bti_unpin**() returns MX_OK.
+In the event of failure, a negative error value is returned.
+
+## ERRORS
+
+**MX_ERR_BAD_HANDLE**  *bti* is not a valid handle.
+
+**MX_ERR_WRONG_TYPE**  *bti* is not a BTI handle.
+
+**MX_ERR_ACCESS_DENIED** *bti* does not have the *MX_RIGHT_MAP*.
+
+**MX_ERR_INVALID_ARGS**  The requested range was not previously returned by
+**mx_bti_pin**().
+
+## SEE ALSO
+
+[bti_create](bti_create.md),
+[bti_pin](bti_pin.md).

kernel/arch/arm64/rules.mk

@@ -35,6 +35,7 @@ MODULE_SRCS += \
 	$(LOCAL_DIR)/uspace_entry.S
 
 MODULE_DEPS += \
+	kernel/dev/iommu/dummy \
 	third_party/lib/fdt \
 
 KERNEL_DEFINES += \

kernel/arch/x86/include/arch/x86/interrupts.h

@@ -6,6 +6,8 @@
 
 #pragma once
 
+#include <assert.h>
+
 #ifdef ASSEMBLY
 #define X86_MAX_INT 0xff
 #else
@@ -42,7 +44,9 @@ enum x86_interrupt_vector {
     X86_INT_IPI_GENERIC,
     X86_INT_IPI_RESCHEDULE,
     X86_INT_IPI_HALT,
+    X86_INT_RESERVED_SENTINEL,
 
     X86_MAX_INT = 0xff,
 };
+static_assert(X86_INT_RESERVED_SENTINEL <= 0x100, "value overflow");
 #endif

kernel/arch/x86/rules.mk

@@ -83,6 +83,8 @@ MODULE_SRCS += \
 	$(LOCAL_DIR)/hypervisor/vmx_cpu_state.cpp \
 
 MODULE_DEPS += \
+	kernel/dev/iommu/dummy \
+	kernel/dev/iommu/intel \
 	kernel/lib/bitmap \
 	kernel/lib/hypervisor \
 	kernel/lib/mxtl \

kernel/dev/iommu/dummy/dummy_iommu.cpp

@@ -0,0 +1,64 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#include <dev/iommu/dummy.h>
+
+#include <err.h>
+#include <kernel/vm.h>
+#include <mxtl/new.h>
+#include <mxtl/ref_ptr.h>
+
+DummyIommu::DummyIommu() {
+}
+
+status_t DummyIommu::Create(mxtl::unique_ptr<const uint8_t[]> desc, uint32_t desc_len,
+                            mxtl::RefPtr<Iommu>* out) {
+    if (desc_len != sizeof(mx_iommu_desc_dummy_t)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+
+    mxtl::AllocChecker ac;
+    auto instance = mxtl::AdoptRef<DummyIommu>(new (&ac) DummyIommu());
+    if (!ac.check()) {
+        return MX_ERR_NO_MEMORY;
+    }
+    *out = mxtl::move(instance);
+    return MX_OK;
+}
+
+DummyIommu::~DummyIommu() {
+}
+
+bool DummyIommu::IsValidBusTxnId(uint64_t bus_txn_id) const {
+    return true;
+}
+
+status_t DummyIommu::Map(uint64_t bus_txn_id, paddr_t paddr, size_t size, uint32_t perms,
+                         dev_vaddr_t* vaddr) {
+    DEBUG_ASSERT(vaddr);
+    if (!IS_PAGE_ALIGNED(paddr) || !IS_PAGE_ALIGNED(size)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+    if (perms & ~(IOMMU_FLAG_PERM_READ | IOMMU_FLAG_PERM_WRITE | IOMMU_FLAG_PERM_EXECUTE)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+    if (perms == 0) {
+        return MX_ERR_INVALID_ARGS;
+    }
+    *vaddr = paddr;
+    return MX_OK;
+}
+
+status_t DummyIommu::Unmap(uint64_t bus_txn_id, dev_vaddr_t vaddr, size_t size) {
+    if (!IS_PAGE_ALIGNED(vaddr) || !IS_PAGE_ALIGNED(size)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+    return MX_OK;
+}
+
+status_t DummyIommu::ClearMappingsForBusTxnId(uint64_t bus_txn_id) {
+    return MX_OK;
+}

kernel/dev/iommu/dummy/include/dev/iommu/dummy.h

@@ -0,0 +1,32 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <dev/iommu.h>
+#include <magenta/compiler.h>
+#include <magenta/syscalls/iommu.h>
+#include <mxtl/unique_ptr.h>
+
+class DummyIommu final : public Iommu {
+public:
+    static status_t Create(mxtl::unique_ptr<const uint8_t[]> desc, uint32_t desc_len,
+                           mxtl::RefPtr<Iommu>* out);
+
+    bool IsValidBusTxnId(uint64_t bus_txn_id) const final;
+
+    status_t Map(uint64_t bus_txn_id, paddr_t paddr, size_t size, uint32_t perms,
+                 dev_vaddr_t* vaddr) final;
+    status_t Unmap(uint64_t bus_txn_id, dev_vaddr_t vaddr, size_t size) final;
+
+    status_t ClearMappingsForBusTxnId(uint64_t bus_txn_id) final;
+
+    ~DummyIommu() final;
+
+    DISALLOW_COPY_ASSIGN_AND_MOVE(DummyIommu);
+private:
+    DummyIommu();
+};

kernel/dev/iommu/dummy/rules.mk

@@ -0,0 +1,14 @@
+# Copyright 2016 The Fuchsia Authors
+# Copyright (c) 2008-2015 Travis Geiselbrecht
+#
+# Use of this source code is governed by a MIT-style
+# license that can be found in the LICENSE file or at
+# https://opensource.org/licenses/MIT
+
+LOCAL_DIR := $(GET_LOCAL_DIR)
+MODULE := $(LOCAL_DIR)
+
+MODULE_SRCS := \
+    $(LOCAL_DIR)/dummy_iommu.cpp
+
+include make/module.mk

kernel/dev/iommu/intel/context_table_state.cpp

@@ -0,0 +1,95 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#include "context_table_state.h"
+
+#include <mxcpp/new.h>
+#include <mxtl/unique_ptr.h>
+
+#include "device_context.h"
+#include "hw.h"
+#include "iommu_impl.h"
+
+namespace intel_iommu {
+
+ContextTableState::ContextTableState(uint8_t bus, bool extended, bool upper,
+                                     IommuImpl* parent, volatile ds::RootEntrySubentry* root_entry,
+                                     IommuPage page)
+        : parent_(parent), root_entry_(root_entry), page_(mxtl::move(page)),
+          bus_(bus), extended_(extended), upper_(upper) {
+}
+
+ContextTableState::~ContextTableState() {
+    ds::RootEntrySubentry entry;
+    entry.ReadFrom(root_entry_);
+    entry.set_present(0);
+    entry.WriteTo(root_entry_);
+    // TODO(teisenbe): Perform a context cache flush
+}
+
+status_t ContextTableState::Create(uint8_t bus, bool extended, bool upper,
+                                   IommuImpl* parent, volatile ds::RootEntrySubentry* root_entry,
+                                   mxtl::unique_ptr<ContextTableState>* table) {
+    ds::RootEntrySubentry entry;
+    entry.ReadFrom(root_entry);
+    DEBUG_ASSERT(!entry.present());
+
+    IommuPage page;
+    status_t status = IommuPage::AllocatePage(&page);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    mxtl::AllocChecker ac;
+    mxtl::unique_ptr<ContextTableState> tbl(new (&ac) ContextTableState(bus, extended, upper,
+                                                                        parent, root_entry,
+                                                                        mxtl::move(page)));
+    if (!ac.check()) {
+        return MX_ERR_NO_MEMORY;
+    }
+
+    entry.set_present(1);
+    entry.set_context_table(tbl->page_.paddr() >> 12);
+    entry.WriteTo(root_entry);
+
+    *table = mxtl::move(tbl);
+    return MX_OK;
+}
+
+status_t ContextTableState::CreateDeviceContext(uint8_t bus, uint8_t dev_func, uint32_t domain_id,
+                                                DeviceContext** context) {
+    mxtl::unique_ptr<DeviceContext> dev;
+    status_t status;
+    if (extended_) {
+        volatile ds::ExtendedContextTable* tbl = extended_table();
+        volatile ds::ExtendedContextEntry* entry = &tbl->entry[dev_func & 0x7f];
+        status = DeviceContext::Create(bus, dev_func, domain_id, parent_, entry, &dev);
+    } else {
+        volatile ds::ContextTable* tbl = table();
+        volatile ds::ContextEntry* entry = &tbl->entry[dev_func];
+        status = DeviceContext::Create(bus, dev_func, domain_id, parent_, entry, &dev);
+    }
+    if (status != MX_OK) {
+        return status;
+    }
+
+    *context = dev.get();
+    devices_.push_back(mxtl::move(dev));
+    return MX_OK;
+}
+
+status_t ContextTableState::GetDeviceContext(uint8_t bus, uint8_t dev_func,
+                                             DeviceContext** context) {
+    for (auto& dev : devices_) {
+        if (dev.is_bdf(bus, dev_func)) {
+            *context = &dev;
+            return MX_OK;
+        }
+    }
+    return MX_ERR_NOT_FOUND;
+}
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/context_table_state.h

@@ -0,0 +1,80 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <mxtl/intrusive_double_list.h>
+#include <mxtl/macros.h>
+#include <mxtl/unique_ptr.h>
+
+#include "hw.h"
+#include "iommu_page.h"
+
+namespace intel_iommu {
+
+class DeviceContext;
+class IommuImpl;
+
+class ContextTableState : public mxtl::DoublyLinkedListable<mxtl::unique_ptr<ContextTableState>> {
+public:
+    ~ContextTableState();
+
+    // Create a ContextTableState for the given bus.
+    // If extended is true, then this will represent a reg::ExtendedContextTable,
+    // and the table will handle translations for either the lower (dev<16) or
+    // upper half of this bus.  Otherwise it represents a reg::ContextTable.
+    static status_t Create(uint8_t bus, bool extended, bool upper,
+                           IommuImpl* parent, volatile ds::RootEntrySubentry* root_entry,
+                           mxtl::unique_ptr<ContextTableState>* table);
+
+    // Check if this ContextTableState is for the given BDF
+    bool includes_bdf(uint8_t bus, uint8_t dev_func) const {
+        if (bus != bus_) return false;
+        if (!extended_) return true;
+        return (dev_func >= 0x80) == upper_;
+    }
+
+    // Create a new DeviceContext representing the given BDF, and give it the specified domain_id.
+    // It is a fatal error to try to create a context for a BDF that already has one.
+    status_t CreateDeviceContext(uint8_t bus, uint8_t dev_func, uint32_t domain_id,
+                                 DeviceContext** context);
+
+    status_t GetDeviceContext(uint8_t bus, uint8_t dev_func, DeviceContext** context);
+private:
+    ContextTableState(uint8_t bus, bool extended, bool upper, IommuImpl* parent,
+                      volatile ds::RootEntrySubentry* root_entry, IommuPage page);
+
+    DISALLOW_COPY_ASSIGN_AND_MOVE(ContextTableState);
+
+    volatile ds::ContextTable* table() const {
+        DEBUG_ASSERT(!extended_);
+        return reinterpret_cast<volatile ds::ContextTable*>(page_.vaddr());
+    }
+
+    volatile ds::ExtendedContextTable* extended_table() const {
+        DEBUG_ASSERT(extended_);
+        return reinterpret_cast<volatile ds::ExtendedContextTable*>(page_.vaddr());
+    }
+
+    // Pointer to IOMMU that owns this ContextTableState
+    IommuImpl* const parent_;
+    // Pointer to the half of the Root Table Entry that decodes to this
+    // ContextTable.
+    volatile ds::RootEntrySubentry* const root_entry_;
+
+    // Page backing the ContextTable/ExtendedContextTable
+    const IommuPage page_;
+
+    // List of device configurations beneath this ContextTable.
+    mxtl::DoublyLinkedList<mxtl::unique_ptr<DeviceContext>> devices_;
+
+    const uint8_t bus_;
+    const bool extended_;
+    // Only valid if extended_ is true
+    const bool upper_;
+};
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/device_context.cpp

@@ -0,0 +1,194 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#include "device_context.h"
+
+#include <kernel/vm.h>
+#include <mxcpp/new.h>
+#include <mxtl/unique_ptr.h>
+#include <trace.h>
+
+#include "hw.h"
+#include "iommu_impl.h"
+
+#define LOCAL_TRACE 0
+
+namespace intel_iommu {
+
+DeviceContext::DeviceContext(uint8_t bus, uint8_t dev_func, IommuImpl* parent,
+                         volatile ds::ExtendedContextEntry* context_entry)
+        : parent_(parent), extended_context_entry_(context_entry), initialized_(false),
+          bus_(bus), dev_func_(dev_func), extended_(true) {
+}
+
+DeviceContext::DeviceContext(uint8_t bus, uint8_t dev_func, IommuImpl* parent,
+                         volatile ds::ContextEntry* context_entry)
+        : parent_(parent), context_entry_(context_entry), initialized_(false),
+          bus_(bus), dev_func_(dev_func), extended_(false) {
+}
+
+DeviceContext::~DeviceContext() {
+    if (extended_) {
+        ds::ExtendedContextEntry entry;
+        entry.ReadFrom(extended_context_entry_);
+        entry.set_present(0);
+        entry.WriteTo(extended_context_entry_);
+    } else {
+        ds::ContextEntry entry;
+        entry.ReadFrom(context_entry_);
+        entry.set_present(0);
+        entry.WriteTo(context_entry_);
+    }
+
+    // TODO(teisenbe): Perform a context cache flush
+
+    if (initialized_) {
+        status_t status = second_level_pt_.Destroy();
+        ASSERT(status == MX_OK);
+    }
+}
+
+status_t DeviceContext::Create(uint8_t bus, uint8_t dev_func, uint32_t domain_id, IommuImpl* parent,
+                               volatile ds::ContextEntry* context_entry,
+                               mxtl::unique_ptr<DeviceContext>* device) {
+    uint8_t aspace_width = 0;
+    auto caps = parent->caps();
+    if (caps->supports_48_bit_agaw()) {
+        aspace_width = 48;
+    } else if (caps->supports_39_bit_agaw()) {
+        aspace_width = 39;
+    }
+
+    ds::ContextEntry entry;
+    entry.ReadFrom(context_entry);
+
+    // It's a bug if we're trying to re-initialize an existing entry
+    ASSERT(!entry.present());
+
+    mxtl::AllocChecker ac;
+    mxtl::unique_ptr<DeviceContext> dev(new (&ac) DeviceContext(bus, dev_func,
+                                                                parent, context_entry));
+    if (!ac.check()) {
+        return MX_ERR_NO_MEMORY;
+    }
+
+    status_t status = dev->second_level_pt_.Init(0, 1ull << aspace_width,
+                                                 ARCH_ASPACE_FLAG_GUEST_PASPACE);
+    if (status != MX_OK) {
+        return status;
+    }
+    dev->initialized_ = true;
+
+    entry.set_present(1);
+    entry.set_fault_processing_disable(0);
+    entry.set_translation_type(ds::ContextEntry::kDeviceTlbDisabled);
+    // TODO(teisenbe): don't hardcode this
+    entry.set_address_width(ds::ContextEntry::k48Bit);
+    entry.set_domain_id(domain_id);
+    entry.set_second_level_pt_ptr(dev->second_level_pt_.pt_phys() >> 12);
+
+    entry.WriteTo(context_entry);
+
+    *device = mxtl::move(dev);
+    return MX_OK;
+}
+
+status_t DeviceContext::Create(uint8_t bus, uint8_t dev_func, uint32_t domain_id, IommuImpl* parent,
+                               volatile ds::ExtendedContextEntry* context_entry,
+                               mxtl::unique_ptr<DeviceContext>* device) {
+
+    uint8_t aspace_width = 0;
+    auto caps = parent->caps();
+    if (caps->supports_48_bit_agaw()) {
+        aspace_width = 48;
+    } else if (caps->supports_39_bit_agaw()) {
+        aspace_width = 39;
+    }
+
+    ds::ExtendedContextEntry entry;
+    entry.ReadFrom(context_entry);
+
+    // It's a bug if we're trying to re-initialize an existing entry
+    ASSERT(!entry.present());
+
+    mxtl::AllocChecker ac;
+    mxtl::unique_ptr<DeviceContext> dev(new (&ac) DeviceContext(bus, dev_func,
+                                                                parent, context_entry));
+    if (!ac.check()) {
+        return MX_ERR_NO_MEMORY;
+    }
+
+    status_t status = dev->second_level_pt_.Init(0, 1ull << aspace_width,
+                                                 ARCH_ASPACE_FLAG_GUEST_PASPACE);
+    if (status != MX_OK) {
+        return status;
+    }
+    dev->initialized_ = true;
+
+    entry.set_present(1);
+    entry.set_fault_processing_disable(0);
+    entry.set_translation_type(ds::ExtendedContextEntry::kHostModeWithDeviceTlbDisabled);
+    entry.set_deferred_invld_enable(0);
+    entry.set_page_request_enable(0);
+    entry.set_nested_translation_enable(0);
+    entry.set_pasid_enable(0);
+    entry.set_global_page_enable(0);
+    // TODO(teisenbe): don't hardcode this
+    entry.set_address_width(ds::ExtendedContextEntry::k48Bit);
+    entry.set_no_exec_enable(1);
+    entry.set_write_protect_enable(1);
+    // TODO: reconsider
+    entry.set_cache_disable(0);
+    entry.set_extended_mem_type_enable(0);
+    entry.set_domain_id(domain_id);
+    entry.set_smep_enable(1);
+    entry.set_extended_accessed_flag_enable(0);
+    entry.set_execute_requests_enable(0);
+    entry.set_second_level_execute_bit_enable(0);
+    entry.set_second_level_pt_ptr(dev->second_level_pt_.pt_phys() >> 12);
+
+    entry.WriteTo(context_entry);
+
+    *device = mxtl::move(dev);
+    return MX_OK;
+}
+
+status_t DeviceContext::SecondLevelMap(paddr_t paddr, size_t size, uint32_t perms,
+                                       paddr_t* virt_paddr) {
+    DEBUG_ASSERT(IS_PAGE_ALIGNED(size));
+    DEBUG_ASSERT(IS_PAGE_ALIGNED(paddr));
+    size_t mapped;
+    uint flags = 0;
+    // TODO: Don't use ARCH_MMU_FLAGs here, should be arch agnostic
+    if (perms & IOMMU_FLAG_PERM_READ) {
+        flags |= ARCH_MMU_FLAG_PERM_READ;
+    }
+    if (perms & IOMMU_FLAG_PERM_WRITE) {
+        flags |= ARCH_MMU_FLAG_PERM_WRITE;
+    }
+    if (perms & IOMMU_FLAG_PERM_EXECUTE) {
+        flags |= ARCH_MMU_FLAG_PERM_EXECUTE;
+    }
+    status_t status = second_level_pt_.Map(paddr, paddr, size / PAGE_SIZE, flags,
+                                           &mapped);
+    if (status != MX_OK) {
+        return status;
+    }
+    ASSERT(mapped == size / PAGE_SIZE);
+    *virt_paddr = paddr;
+    LTRACEF("Map(%02x:%02x.%1x): [%p, %p) -> %p %#x\n", bus_, (unsigned int)dev_func_ >> 3, (unsigned int)dev_func_ & 0x7, (void*)paddr, (void*)(paddr + size), (void*)paddr, flags);
+    return MX_OK;
+}
+
+status_t DeviceContext::SecondLevelUnmap(paddr_t virt_paddr, size_t size) {
+    DEBUG_ASSERT(IS_PAGE_ALIGNED(virt_paddr));
+    DEBUG_ASSERT(IS_PAGE_ALIGNED(size));
+    size_t unmapped;
+    LTRACEF("Unmap(%02x:%02x.%1x): [%p, %p)\n", bus_, (unsigned int)dev_func_ >> 3, (unsigned int)dev_func_ & 0x7, (void*)virt_paddr, (void*)(virt_paddr + size));
+    return second_level_pt_.Unmap(virt_paddr, size / PAGE_SIZE, &unmapped);
+}
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/device_context.h

@@ -0,0 +1,72 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+// TODO(teisenbe): Move x86 mmu stuff into a lib, and use a different
+// parameterization for the IOMMU.  Slightly different support for things like
+// large pages and dynamic multilevel translation.
+#include <arch/aspace.h>
+
+#include <mxtl/intrusive_double_list.h>
+#include <mxtl/macros.h>
+#include <mxtl/unique_ptr.h>
+
+#include "hw.h"
+
+namespace intel_iommu {
+
+class IommuImpl;
+
+class DeviceContext : public mxtl::DoublyLinkedListable<mxtl::unique_ptr<DeviceContext>> {
+public:
+    ~DeviceContext();
+
+    // Create a new DeviceContext representing the given BDF.  It is a fatal error
+    // to try to create a context for a BDF that already has one.
+    static status_t Create(uint8_t bus, uint8_t dev_func, uint32_t domain_id, IommuImpl* parent,
+                           volatile ds::ExtendedContextEntry* context_entry,
+                           mxtl::unique_ptr<DeviceContext>* device);
+    static status_t Create(uint8_t bus, uint8_t dev_func, uint32_t domain_id, IommuImpl* parent,
+                           volatile ds::ContextEntry* context_entry,
+                           mxtl::unique_ptr<DeviceContext>* device);
+
+    // Check if this DeviceContext is for the given BDF
+    bool is_bdf(uint8_t bus, uint8_t dev_func) const {
+        return bus == bus_ && dev_func == dev_func_;
+    }
+
+    // Use the second-level translation table to map the host address |paddr| to the
+    // guest's address |*virt_paddr|.  |size| is in bytes.
+    status_t SecondLevelMap(paddr_t paddr, size_t size, uint32_t perms, paddr_t* virt_paddr);
+    status_t SecondLevelUnmap(paddr_t virt_paddr, size_t size);
+
+private:
+    DeviceContext(uint8_t bus, uint8_t dev_func, IommuImpl* parent,
+                  volatile ds::ExtendedContextEntry* context_entry);
+    DeviceContext(uint8_t bus, uint8_t dev_func, IommuImpl* parent,
+                  volatile ds::ContextEntry* context_entry);
+
+    DISALLOW_COPY_ASSIGN_AND_MOVE(DeviceContext);
+
+    IommuImpl* const parent_;
+    union {
+        volatile ds::ExtendedContextEntry* const extended_context_entry_;
+        volatile ds::ContextEntry* const context_entry_;
+    };
+
+    // Page tables used for translating requests-without-PASID and for nested
+    // translation of requests-with-PASID.
+    ArchVmAspace second_level_pt_;
+
+    bool initialized_;
+
+    const uint8_t bus_;
+    const uint8_t dev_func_;
+    const bool extended_;
+};
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/domain_allocator.cpp

@@ -0,0 +1,36 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#include "domain_allocator.h"
+
+#include <assert.h>
+
+namespace intel_iommu {
+
+DomainAllocator::DomainAllocator()
+    // Note that next_domain_id_ starts at 1, since under some conditions 0 is
+    // an invalid domain ID (i.e. if CM is set in the capability register).
+    : num_domains_(0), next_domain_id_(1) { }
+
+status_t DomainAllocator::Allocate(uint32_t* domain_id) {
+    if (next_domain_id_ >= num_domains_) {
+        return MX_ERR_NO_RESOURCES;
+    }
+
+    // This allocator should be enough, since the hardware should have enough
+    // domain IDs for each device hanging off of it.  If we start deallocating
+    // Context Entries, we'll need to make this allocator more sophisticated to
+    // manage the ID reuse.
+    *domain_id = next_domain_id_++;
+    return MX_OK;
+}
+
+void DomainAllocator::set_num_domains(uint32_t num) {
+    ASSERT(num >= next_domain_id_);
+    num_domains_ = num;
+}
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/domain_allocator.h

@@ -0,0 +1,36 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <err.h>
+#include <stdint.h>
+
+#include <mxtl/macros.h>
+
+namespace intel_iommu {
+
+// Manages the domain ID space for a given IOMMU
+class DomainAllocator {
+public:
+    DomainAllocator();
+
+    // Get an unused domain ID.
+    // Returns MX_ERR_NO_RESOURCES if one cannot be found.
+    status_t Allocate(uint32_t* domain_id);
+
+    // Set the number of domain IDs this instance manages.  Panics if this call
+    // would reduce the number of domain IDs below the highest allocated one.
+    void set_num_domains(uint32_t num);
+
+private:
+    DISALLOW_COPY_ASSIGN_AND_MOVE(DomainAllocator);
+
+    uint32_t num_domains_;
+    uint32_t next_domain_id_;
+};
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/hw.h

@@ -0,0 +1,469 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <err.h>
+#include <kernel/atomic.h>
+#include <magenta/compiler.h>
+#include <hwreg/bitfields.h>
+#include <stdint.h>
+
+namespace intel_iommu {
+
+namespace reg {
+
+class Version : public hwreg::RegisterBase<uint32_t> {
+ public:
+    static constexpr uint32_t kAddr = 0x0;
+    static auto Get() { return hwreg::RegisterAddr<Version>(kAddr); }
+
+    DEF_FIELD(3, 0, minor);
+    DEF_FIELD(7, 4, major);
+    DEF_RSVDZ_FIELD(31, 8);
+};
+
+class Capability : public hwreg::RegisterBase<uint64_t> {
+ public:
+    static constexpr uint32_t kAddr = 0x8;
+    static auto Get() { return hwreg::RegisterAddr<Capability>(kAddr); }
+
+    DEF_FIELD(2, 0, num_domains);
+    DEF_BIT(3, adv_fault_logging);
+    DEF_BIT(4, required_write_buf_flushing);
+    DEF_BIT(5, supports_protected_low_mem);
+    DEF_BIT(6, supports_protected_high_mem);
+    DEF_BIT(7, caching_mode);
+    DEF_RSVDZ_BIT(8);
+    DEF_BIT(9, supports_39_bit_agaw);
+    DEF_BIT(10, supports_48_bit_agaw);
+    DEF_RSVDZ_BIT(11);
+    DEF_RSVDZ_BIT(12);
+    DEF_RSVDZ_FIELD(15, 13);
+    DEF_FIELD(21, 16, max_guest_addr_width);
+    DEF_BIT(22, supports_zero_length_read);
+    DEF_RSVDZ_BIT(23);
+    DEF_FIELD(33, 24, fault_recording_register_offset);
+    DEF_BIT(34, supports_second_level_2mb_page);
+    DEF_BIT(35, supports_second_level_1gb_page);
+    DEF_RSVDZ_FIELD(37, 36);
+    DEF_RSVDZ_BIT(38);
+    DEF_BIT(39, supports_page_selective_invld);
+    DEF_FIELD(47, 40, num_fault_recording_reg);
+    DEF_FIELD(53, 48, max_addr_mask_value);
+    DEF_BIT(54, supports_write_draining);
+    DEF_BIT(55, supports_read_draining);
+    DEF_BIT(56, supports_first_level_1gb_page);
+    DEF_RSVDZ_FIELD(58, 57);
+    DEF_BIT(59, supports_posted_interrupts);
+    DEF_RSVDZ_FIELD(63, 60);
+};
+
+class ExtendedCapability : public hwreg::RegisterBase<uint64_t> {
+ public:
+    static constexpr uint32_t kAddr = 0x10;
+    static auto Get() { return hwreg::RegisterAddr<ExtendedCapability>(kAddr); }
+
+    DEF_BIT(0, page_walk_coherency);
+    DEF_BIT(1, supports_queued_invld);
+    DEF_BIT(2, supports_device_tlb);
+    DEF_BIT(3, supports_interrupt_remapping);
+    DEF_BIT(4, supports_extended_interrupt_mode);
+    DEF_BIT(6, supports_pass_through);
+    DEF_BIT(7, supports_snoop_control);
+    DEF_FIELD(17, 8, iotlb_register_offset);
+    DEF_RSVDZ_FIELD(19, 18);
+    DEF_FIELD(23, 20, max_handle_mask_value);
+    DEF_BIT(24, supports_extended_context);
+    DEF_BIT(25, supports_memory_type);
+    DEF_BIT(26, supports_nested_translation);
+    DEF_BIT(27, supports_deferred_invld);
+    DEF_BIT(28, supports_pasid);
+    DEF_BIT(29, supports_page_requests);
+    DEF_BIT(30, supports_execute_requests);
+    DEF_BIT(31, supports_supervisor_requests);
+    DEF_RSVDZ_BIT(32);
+    DEF_BIT(33, supports_no_write_flag);
+    DEF_BIT(34, supports_extended_accessed_flag);
+    DEF_FIELD(39, 35, pasid_size);
+    DEF_RSVDZ_FIELD(63, 40);
+};
+
+// This is a merger of the Global Command and Global Status registers.
+class GlobalControl : public hwreg::RegisterBase<uint32_t> {
+ public:
+    static constexpr uint32_t kWriteAddr = 0x18;
+    static constexpr uint32_t kReadAddr = 0x1c;
+    static auto Get() { return hwreg::RegisterAddr<GlobalControl>(kReadAddr); }
+
+    DEF_RSVDZ_FIELD(22, 0);
+    DEF_BIT(23, compat_format_interrupt);
+    DEF_BIT(24, interrupt_remap_table_ptr);
+    DEF_BIT(25, interrupt_remap_enable);
+    DEF_BIT(26, queued_invld_enable);
+    DEF_BIT(27, write_buffer_flush);
+    DEF_BIT(28, adv_fault_logging_enable);
+    DEF_BIT(29, fault_log);
+    DEF_BIT(30, root_table_ptr);
+    DEF_BIT(31, translation_enable);
+
+    // This redefines functions from RegisterBase which are not virtual.
+    // This is safe, since no callers operate on this type as its base class.
+    void ReadFrom(hwreg::RegisterIo* reg_io) {
+        hwreg::RegisterBase<uint32_t>::set_reg_addr(kReadAddr);
+        return hwreg::RegisterBase<uint32_t>::ReadFrom(reg_io);
+    }
+    void WriteTo(hwreg::RegisterIo* reg_io) {
+        hwreg::RegisterBase<uint32_t>::set_reg_addr(kWriteAddr);
+        return hwreg::RegisterBase<uint32_t>::WriteTo(reg_io);
+    }
+};
+
+class RootTableAddress : public hwreg::RegisterBase<uint64_t> {
+ public:
+    static constexpr uint32_t kAddr = 0x20;
+    static auto Get() { return hwreg::RegisterAddr<RootTableAddress>(kAddr); }
+
+    DEF_RSVDZ_FIELD(10, 0);
+    DEF_BIT(11, root_table_type);
+    DEF_FIELD(63, 12, root_table_address);
+};
+
+class ContextCommand : public hwreg::RegisterBase<uint64_t> {
+public:
+    static constexpr uint32_t kAddr = 0x28;
+    static auto Get() { return hwreg::RegisterAddr<ContextCommand>(kAddr); }
+
+    DEF_FIELD(15, 0, domain_id);
+    DEF_FIELD(31, 16, source_id);
+    DEF_FIELD(33, 32, function_mask);
+    DEF_RSVDZ_FIELD(58, 34);
+    DEF_FIELD(60, 59, actual_invld_granularity);
+    DEF_FIELD(62, 61, invld_request_granularity);
+    DEF_BIT(63, invld_context_cache);
+
+    enum Granularity {
+        kGlobalInvld = 0b01,
+        kDomainInvld = 0b10,
+        kDeviceInvld = 0b11,
+    };
+};
+
+class InvalidateAddress : public hwreg::RegisterBase<uint64_t> {
+ public:
+    static constexpr uint32_t kInstanceOffset = 0x0;
+    static auto Get(uint32_t iotlb_base) {
+        return hwreg::RegisterAddr<InvalidateAddress>(iotlb_base + kInstanceOffset);
+    }
+
+    DEF_FIELD(5, 0, address_mask);
+    DEF_BIT(6, invld_hint);
+    DEF_RSVDZ_FIELD(11, 7);
+    DEF_FIELD(63, 12, address);
+};
+
+class IotlbInvalidate : public hwreg::RegisterBase<uint64_t> {
+ public:
+    static constexpr uint32_t kInstanceOffset = 0x08;
+    static auto Get(uint32_t iotlb_base) {
+        return hwreg::RegisterAddr<IotlbInvalidate>(iotlb_base + kInstanceOffset);
+    }
+
+    DEF_FIELD(47, 32, domain_id);
+    DEF_BIT(48, drain_writes);
+    DEF_BIT(49, drain_reads);
+    DEF_RSVDZ_FIELD(56, 50);
+    DEF_FIELD(58, 57, actual_invld_granularity);
+    DEF_RSVDZ_BIT(59);
+    DEF_FIELD(61, 60, invld_request_granularity);
+    DEF_RSVDZ_BIT(62);
+    DEF_BIT(63, invld_iotlb);
+
+    enum Granularity {
+        kGlobalInvld = 0b01,
+        kDomainInvld = 0b10,
+        kDeviceInvld = 0b11,
+    };
+};
+
+class FaultStatus : public hwreg::RegisterBase<uint32_t> {
+public:
+    static constexpr uint32_t kAddr = 0x34;
+    static auto Get() { return hwreg::RegisterAddr<FaultStatus>(kAddr); }
+
+    DEF_BIT(0, primary_fault_overflow);
+    DEF_BIT(1, primary_pending_fault);
+    DEF_BIT(2, adv_fault_overflow);
+    DEF_BIT(3, adv_pending_fault);
+    DEF_BIT(4, invld_queue_error);
+    DEF_BIT(5, invld_completion_error);
+    DEF_BIT(6, invld_timeout_error);
+    DEF_BIT(7, page_request_overflow);
+    DEF_FIELD(15, 8, fault_record_index);
+    DEF_RSVDZ_FIELD(31, 16);
+};
+
+class FaultEventControl : public hwreg::RegisterBase<uint32_t> {
+public:
+    static constexpr uint32_t kAddr = 0x38;
+    static auto Get() { return hwreg::RegisterAddr<FaultEventControl>(kAddr); }
+
+    DEF_BIT(30, interrupt_pending);
+    DEF_BIT(31, interrupt_mask);
+};
+
+class FaultEventData : public hwreg::RegisterBase<uint32_t> {
+public:
+    static constexpr uint32_t kAddr = 0x3c;
+    static auto Get() { return hwreg::RegisterAddr<FaultEventData>(kAddr); }
+
+    DEF_FIELD(15, 0, interrupt_message_data);
+    DEF_FIELD(31, 16, extended_interrupt_message_data);
+};
+
+class FaultEventAddress : public hwreg::RegisterBase<uint32_t> {
+public:
+    static constexpr uint32_t kAddr = 0x40;
+    static auto Get() { return hwreg::RegisterAddr<FaultEventAddress>(kAddr); }
+
+    DEF_RSVDZ_FIELD(1, 0);
+    DEF_FIELD(31, 2, message_address);
+};
+
+class FaultEventUpperAddress : public hwreg::RegisterBase<uint32_t> {
+public:
+    static constexpr uint32_t kAddr = 0x44;
+    static auto Get() { return hwreg::RegisterAddr<FaultEventUpperAddress>(kAddr); }
+
+    DEF_FIELD(31, 0, message_upper_address);
+};
+
+class FaultRecordLow : public hwreg::RegisterBase<uint64_t> {
+public:
+    static constexpr uint32_t kInstanceOffset = 0x0;
+    static auto Get(uint32_t fault_record_base, uint32_t index) {
+        return hwreg::RegisterAddr<FaultRecordLow>(fault_record_base + 16 * index +
+                                                   kInstanceOffset);
+    }
+
+    DEF_RSVDZ_FIELD(11, 0);
+    DEF_FIELD(63, 12, fault_info);
+};
+
+class FaultRecordHigh : public hwreg::RegisterBase<uint64_t> {
+public:
+    static constexpr uint32_t kInstanceOffset = 0x8;
+    static auto Get(uint32_t fault_record_base, uint32_t index) {
+        return hwreg::RegisterAddr<FaultRecordHigh>(fault_record_base + 16 * index +
+                                                    kInstanceOffset);
+    }
+
+    DEF_FIELD(15, 0, source_id);
+    DEF_RSVDZ_FIELD(28, 16);
+    DEF_BIT(29, supervisor_mode_requested);
+    DEF_BIT(30, execute_permission_requested);
+    DEF_BIT(31, pasid_present);
+    DEF_FIELD(39, 32, fault_reason);
+    DEF_FIELD(59, 40, pasid_value);
+    DEF_FIELD(61, 60, address_type);
+    DEF_BIT(62, request_type);
+    DEF_BIT(63, fault);
+};
+
+} // namespace reg
+
+namespace ds {
+
+struct RootEntrySubentry {
+    uint64_t raw;
+
+    DEF_SUBBIT(raw, 0, present);
+    DEF_SUBFIELD(raw, 63, 12, context_table);
+
+    void ReadFrom(volatile RootEntrySubentry* dst) {
+        raw = dst->raw;
+    }
+    void WriteTo(volatile RootEntrySubentry* dst) {
+        dst->raw = raw;
+    }
+};
+
+struct RootEntry {
+    RootEntrySubentry lower;
+    RootEntrySubentry upper;
+};
+static_assert(mxtl::is_pod<RootEntry>::value, "not POD");
+static_assert(sizeof(RootEntry) == 16, "wrong size");
+
+struct RootTable {
+    static constexpr size_t kNumEntries = 256;
+    RootEntry entry[kNumEntries];
+};
+static_assert(mxtl::is_pod<RootTable>::value, "not POD");
+static_assert(sizeof(RootTable) == 4096, "wrong size");
+
+struct ContextEntry {
+    uint64_t raw[2];
+
+    DEF_SUBBIT(raw[0], 0, present);
+    DEF_SUBBIT(raw[0], 1, fault_processing_disable);
+    DEF_SUBFIELD(raw[0], 3, 2, translation_type);
+    DEF_SUBFIELD(raw[0], 63, 12, second_level_pt_ptr);
+    DEF_SUBFIELD(raw[1], 2, 0, address_width);
+    DEF_SUBFIELD(raw[1], 6, 3, hw_ignored);
+    DEF_SUBFIELD(raw[1], 23, 8, domain_id);
+
+    void ReadFrom(volatile ContextEntry* dst) {
+        raw[0] = dst->raw[0];
+        raw[1] = dst->raw[1];
+    }
+
+    void WriteTo(volatile ContextEntry* dst) {
+        // Write word with present bit last
+        dst->raw[1] = raw[1];
+        dst->raw[0] = raw[0];
+    }
+
+    enum TranslationType {
+        kDeviceTlbDisabled  = 0b00,
+        kDeviceTlbEnabled   = 0b01,
+        kPassThrough        = 0b10,
+    };
+
+    enum AddressWidth {
+        k30Bit = 0b000,
+        k39Bit = 0b001,
+        k48Bit = 0b010,
+        k57Bit = 0b011,
+        k64Bit = 0b100,
+    };
+};
+static_assert(mxtl::is_pod<ContextEntry>::value, "not POD");
+static_assert(sizeof(ContextEntry) == 16, "wrong size");
+
+struct ContextTable {
+    static constexpr size_t kNumEntries = 256;
+    ContextEntry entry[kNumEntries];
+};
+static_assert(mxtl::is_pod<ContextTable>::value, "not POD");
+static_assert(sizeof(ContextTable) == 4096, "wrong size");
+
+struct ExtendedContextEntry {
+    uint64_t raw[4];
+
+    DEF_SUBBIT(raw[0], 0, present);
+    DEF_SUBBIT(raw[0], 1, fault_processing_disable);
+    DEF_SUBFIELD(raw[0], 4, 2, translation_type);
+    DEF_SUBFIELD(raw[0], 7, 5, extended_mem_type);
+    DEF_SUBBIT(raw[0], 8, deferred_invld_enable);
+    DEF_SUBBIT(raw[0], 9, page_request_enable);
+    DEF_SUBBIT(raw[0], 10, nested_translation_enable);
+    DEF_SUBBIT(raw[0], 11, pasid_enable);
+    DEF_SUBFIELD(raw[0], 63, 12, second_level_pt_ptr);
+
+    DEF_SUBFIELD(raw[1], 2, 0, address_width);
+    DEF_SUBBIT(raw[1], 3, global_page_enable);
+    DEF_SUBBIT(raw[1], 4, no_exec_enable);
+    DEF_SUBBIT(raw[1], 5, write_protect_enable);
+    DEF_SUBBIT(raw[1], 6, cache_disable);
+    DEF_SUBBIT(raw[1], 7, extended_mem_type_enable);
+    DEF_SUBFIELD(raw[1], 23, 8, domain_id);
+    DEF_SUBBIT(raw[1], 24, smep_enable);
+    DEF_SUBBIT(raw[1], 25, extended_accessed_flag_enable);
+    DEF_SUBBIT(raw[1], 26, execute_requests_enable);
+    DEF_SUBBIT(raw[1], 27, second_level_execute_bit_enable);
+    DEF_SUBFIELD(raw[1], 63, 32, page_attribute_table);
+
+    DEF_SUBFIELD(raw[2], 3, 0, pasid_table_size);
+    DEF_SUBFIELD(raw[2], 63, 12, pasid_table_ptr);
+
+    DEF_SUBFIELD(raw[3], 63, 12, pasid_state_table_ptr);
+
+    void ReadFrom(volatile ExtendedContextEntry* dst) {
+        raw[0] = dst->raw[0];
+        raw[1] = dst->raw[1];
+        raw[2] = dst->raw[2];
+        raw[3] = dst->raw[3];
+    }
+
+    void WriteTo(volatile ExtendedContextEntry* dst) {
+        dst->raw[1] = raw[1];
+        dst->raw[2] = raw[2];
+        dst->raw[3] = raw[3];
+        // Write word with present bit last
+        dst->raw[0] = raw[0];
+    }
+
+    enum TranslationType {
+        kHostModeWithDeviceTlbDisabled  = 0b000,
+        kHostModeWithDeviceTlbEnabled   = 0b001,
+        kPassThrough                    = 0b010,
+        kGuestModeWithDeviceTlbDisabled = 0b100,
+        kGuestModeWithDeviceTlbEnabled  = 0b101,
+    };
+
+    enum AddressWidth {
+        k30Bit = 0b000,
+        k39Bit = 0b001,
+        k48Bit = 0b010,
+        k57Bit = 0b011,
+        k64Bit = 0b100,
+    };
+};
+static_assert(mxtl::is_pod<ExtendedContextEntry>::value, "not POD");
+static_assert(sizeof(ExtendedContextEntry) == 32, "wrong size");
+
+struct ExtendedContextTable {
+    static constexpr size_t kNumEntries = 128;
+    ExtendedContextEntry entry[kNumEntries];
+};
+static_assert(mxtl::is_pod<ExtendedContextTable>::value, "not POD");
+static_assert(sizeof(ExtendedContextTable) == 4096, "wrong size");
+
+struct PasidEntry {
+    uint64_t raw;
+
+    DEF_SUBBIT(raw, 0, present);
+    DEF_SUBBIT(raw, 3, page_level_write_through);
+    DEF_SUBBIT(raw, 4, page_level_cache_disable);
+    DEF_SUBBIT(raw, 11, supervisor_requests_enable);
+    DEF_SUBFIELD(raw, 63, 12, first_level_pt_ptr);
+
+    void WriteTo(volatile PasidEntry* dst) {
+        dst->raw = raw;
+    }
+};
+static_assert(mxtl::is_pod<PasidEntry>::value, "not POD");
+static_assert(sizeof(PasidEntry) == 8, "wrong size");
+
+struct PasidState {
+    volatile uint64_t raw;
+
+    //DEF_SUBFIELD(raw, 47, 32, active_ref_count);
+    //DEF_SUBBIT(raw, 63, deferred_invld);
+
+    uint64_t active_ref_count() {
+        return (atomic_load_u64(&raw) >> 32) & 0xffff;
+    }
+
+    uint64_t deferred_invld() {
+        return atomic_load_u64(&raw) >> 63;
+    }
+    void set_deferred_invld() {
+        // The specification is unclear as to how to update this field.  This is
+        // an in-memory data structure, and the active_ref_count field is specified
+        // as being updated atomically by hardware.  Reading that "atomically"
+        // to be an atomic memory access, this atomic_or should be the right
+        // thing.
+        atomic_or_u64(&raw, 1ull << 63);
+    }
+};
+static_assert(mxtl::is_pod<PasidState>::value, "not POD");
+static_assert(sizeof(PasidState) == 8, "wrong size");
+
+} // namespace ds
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/include/dev/iommu/intel.h

@@ -0,0 +1,17 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <dev/iommu.h>
+#include <magenta/syscalls/iommu.h>
+#include <mxtl/ref_ptr.h>
+
+class IntelIommu {
+public:
+    static status_t Create(mxtl::unique_ptr<const uint8_t[]> desc, uint32_t desc_len,
+                           mxtl::RefPtr<Iommu>* out);
+};

kernel/dev/iommu/intel/intel_iommu.cpp

@@ -0,0 +1,13 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#include <dev/iommu/intel.h>
+#include "iommu_impl.h"
+
+status_t IntelIommu::Create(mxtl::unique_ptr<const uint8_t[]> desc, uint32_t desc_len,
+                            mxtl::RefPtr<Iommu>* out) {
+    return intel_iommu::IommuImpl::Create(mxtl::move(desc), desc_len, out);
+}

kernel/dev/iommu/intel/iommu_impl.cpp

@@ -0,0 +1,706 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#include "iommu_impl.h"
+
+#include <err.h>
+#include <kernel/vm.h>
+#include <kernel/vm/vm_aspace.h>
+#include <kernel/vm/vm_object_paged.h>
+#include <mxcpp/new.h>
+#include <mxtl/algorithm.h>
+#include <mxtl/auto_lock.h>
+#include <mxtl/limits.h>
+#include <mxtl/ref_ptr.h>
+#include <mxtl/unique_ptr.h>
+#include <platform.h>
+#include <safeint/safe_math.h>
+#include <trace.h>
+
+#include "context_table_state.h"
+#include "device_context.h"
+#include "hw.h"
+
+#define LOCAL_TRACE 1
+
+namespace intel_iommu {
+
+IommuImpl::IommuImpl(volatile void* register_base,
+                     mxtl::unique_ptr<const uint8_t[]> desc, uint32_t desc_len)
+    : desc_(mxtl::move(desc)), desc_len_(desc_len), mmio_(register_base) {
+    memset(&irq_block_, 0, sizeof(irq_block_));
+}
+
+status_t IommuImpl::Create(mxtl::unique_ptr<const uint8_t[]> desc_bytes, uint32_t desc_len,
+                           mxtl::RefPtr<Iommu>* out) {
+    status_t status = ValidateIommuDesc(desc_bytes, desc_len);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    auto desc = reinterpret_cast<const mx_iommu_desc_intel_t*>(desc_bytes.get());
+    const uint64_t register_base = desc->register_base;
+
+    auto kernel_aspace = VmAspace::kernel_aspace();
+    void *vaddr;
+    status = kernel_aspace->AllocPhysical(
+            "iommu",
+            PAGE_SIZE,
+            &vaddr,
+            PAGE_SIZE_SHIFT,
+            register_base,
+            0,
+            ARCH_MMU_FLAG_PERM_READ | ARCH_MMU_FLAG_PERM_WRITE | ARCH_MMU_FLAG_UNCACHED);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    mxtl::AllocChecker ac;
+    auto instance = mxtl::AdoptRef<IommuImpl>(new (&ac) IommuImpl(vaddr, mxtl::move(desc_bytes),
+                                                                  desc_len));
+    if (!ac.check()) {
+        kernel_aspace->FreeRegion(reinterpret_cast<vaddr_t>(vaddr));
+        return MX_ERR_NO_MEMORY;
+    }
+
+    status = instance->Initialize();
+    if (status != MX_OK) {
+        return status;
+    }
+
+    *out = mxtl::move(instance);
+    return MX_OK;
+}
+
+IommuImpl::~IommuImpl() {
+    mxtl::AutoLock guard(&lock_);
+
+    // We cannot unpin memory until translation is disabled
+    status_t status = SetTranslationEnableLocked(false, INFINITE_TIME);
+    ASSERT(status == MX_OK);
+
+    DisableFaultsLocked();
+    auto& pcie_platform = PcieBusDriver::GetDriver()->platform();
+    pcie_platform.FreeMsiBlock(&irq_block_);
+
+    VmAspace::kernel_aspace()->FreeRegion(mmio_.base());
+}
+
+status_t IommuImpl::ValidateIommuDesc(const mxtl::unique_ptr<const uint8_t[]>& desc_bytes,
+                                      uint32_t desc_len) {
+    auto desc = reinterpret_cast<const mx_iommu_desc_intel_t*>(desc_bytes.get());
+
+    // Validate the size
+    if (desc_len < sizeof(*desc)) {
+        LTRACEF("desc too short: %u < %zu\n", desc_len, sizeof(*desc));
+        return MX_ERR_INVALID_ARGS;
+    }
+    static_assert(sizeof(desc->scope_bytes) < sizeof(size_t),
+                  "if this changes, need to check for overflow");
+    safeint::CheckedNumeric<size_t> actual_size = sizeof(*desc);
+    actual_size += desc->scope_bytes;
+    actual_size += desc->reserved_memory_bytes;
+    if (!actual_size.IsValid() || desc_len != actual_size.ValueOrDie()) {
+        LTRACEF("desc size mismatch: %u != %zu\n", desc_len, actual_size.ValueOrDefault(SIZE_MAX));
+        return MX_ERR_INVALID_ARGS;
+    }
+
+    // Validate scopes
+    if (desc->scope_bytes == 0 && !desc->whole_segment) {
+        LTRACEF("desc has no scopes\n");
+        return MX_ERR_INVALID_ARGS;
+    }
+    const size_t num_scopes = desc->scope_bytes / sizeof(mx_iommu_desc_intel_scope_t);
+    safeint::CheckedNumeric<size_t> scope_bytes = num_scopes;
+    scope_bytes *= sizeof(mx_iommu_desc_intel_scope_t);
+    if (!scope_bytes.IsValid() || scope_bytes.ValueOrDie() != desc->scope_bytes) {
+        LTRACEF("desc has invalid scope_bytes field\n");
+        return MX_ERR_INVALID_ARGS;
+    }
+
+    auto scopes = reinterpret_cast<mx_iommu_desc_intel_scope_t*>(
+            reinterpret_cast<uintptr_t>(desc) + sizeof(*desc));
+    for (size_t i = 0; i < num_scopes; ++i) {
+        if (scopes[i].num_hops == 0) {
+            LTRACEF("desc scope %zu has no hops\n", i);
+            return MX_ERR_INVALID_ARGS;
+        }
+        if (scopes[i].num_hops > countof(scopes[0].dev_func)) {
+            LTRACEF("desc scope %zu has too many hops\n", i);
+            return MX_ERR_INVALID_ARGS;
+        }
+    }
+
+    // Validate reserved memory regions
+    size_t cursor_bytes = sizeof(*desc) + desc->scope_bytes;
+    while (cursor_bytes + sizeof(mx_iommu_desc_intel_reserved_memory_t) < desc_len) {
+        auto mem = reinterpret_cast<mx_iommu_desc_intel_reserved_memory_t*>(
+                reinterpret_cast<uintptr_t>(desc) + cursor_bytes);
+
+        safeint::CheckedNumeric<size_t> next_entry = cursor_bytes;
+        next_entry += sizeof(mx_iommu_desc_intel_reserved_memory_t);
+        next_entry += mem->scope_bytes;
+        if (!next_entry.IsValid() || next_entry.ValueOrDie() > desc_len) {
+            LTRACEF("desc reserved memory entry has invalid scope_bytes\n");
+            return MX_ERR_INVALID_ARGS;
+        }
+
+        // TODO(teisenbe): Make sure that the reserved memory regions are not in our
+        // allocatable RAM pools
+
+        // Validate scopes
+        if (mem->scope_bytes == 0) {
+            LTRACEF("desc reserved memory entry has no scopes\n");
+            return MX_ERR_INVALID_ARGS;
+        }
+        const size_t num_scopes = mem->scope_bytes / sizeof(mx_iommu_desc_intel_scope_t);
+        safeint::CheckedNumeric<size_t> scope_bytes = num_scopes;
+        scope_bytes *= sizeof(mx_iommu_desc_intel_scope_t);
+        if (!scope_bytes.IsValid() || scope_bytes.ValueOrDie() != desc->scope_bytes) {
+            LTRACEF("desc reserved memory entry has invalid scope_bytes field\n");
+            return MX_ERR_INVALID_ARGS;
+        }
+
+        auto scopes = reinterpret_cast<mx_iommu_desc_intel_scope_t*>(
+                reinterpret_cast<uintptr_t>(mem) + sizeof(*mem));
+        for (size_t i = 0; i < num_scopes; ++i) {
+            if (scopes[i].num_hops == 0) {
+                LTRACEF("desc reserved memory entry scope %zu has no hops\n", i);
+                return MX_ERR_INVALID_ARGS;
+            }
+            if (scopes[i].num_hops > countof(scopes[0].dev_func)) {
+                LTRACEF("desc reserved memory entry scope %zu has too many hops\n", i);
+                return MX_ERR_INVALID_ARGS;
+            }
+        }
+
+        cursor_bytes = next_entry.ValueOrDie();
+    }
+    if (cursor_bytes != desc_len) {
+        LTRACEF("desc has invalid reserved_memory_bytes field\n");
+        return MX_ERR_INVALID_ARGS;
+    }
+
+    LTRACEF("validated desc\n");
+    return MX_OK;
+}
+
+bool IommuImpl::IsValidBusTxnId(uint64_t bus_txn_id) const {
+    if (bus_txn_id > UINT16_MAX) {
+        return false;
+    }
+
+    uint8_t bus;
+    uint8_t dev_func;
+    decode_bus_txn_id(bus_txn_id, &bus, &dev_func);
+
+    auto desc = reinterpret_cast<const mx_iommu_desc_intel_t*>(desc_.get());
+    const size_t num_scopes = desc->scope_bytes / sizeof(mx_iommu_desc_intel_scope_t);
+    auto scopes = reinterpret_cast<mx_iommu_desc_intel_scope_t*>(
+            reinterpret_cast<uintptr_t>(desc) + sizeof(*desc));
+
+    // Search for this BDF in the scopes we have
+    for (size_t i = 0; i < num_scopes; ++i) {
+        if (scopes[i].num_hops != 1) {
+            // TODO(teisenbe): Implement
+            continue;
+        }
+
+        if (scopes[i].start_bus == bus && scopes[i].dev_func[0] == dev_func) {
+            return !desc->whole_segment;
+        }
+    }
+
+    if (desc->whole_segment) {
+        // Since we only support single segment currently, just return true
+        // here.  To support more segments, we need to make sure the segment
+        // matches, too.
+        return true;
+    }
+
+    return false;
+}
+
+status_t IommuImpl::Map(uint64_t bus_txn_id, paddr_t paddr, size_t size, uint32_t perms,
+                        dev_vaddr_t* vaddr) {
+    DEBUG_ASSERT(vaddr);
+    if (!IS_PAGE_ALIGNED(paddr) || !IS_PAGE_ALIGNED(size)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+    if (perms & ~(IOMMU_FLAG_PERM_READ | IOMMU_FLAG_PERM_WRITE | IOMMU_FLAG_PERM_EXECUTE)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+    if (perms == 0) {
+        return MX_ERR_INVALID_ARGS;
+    }
+    if (!IsValidBusTxnId(bus_txn_id)) {
+        return MX_ERR_NOT_FOUND;
+    }
+
+    uint8_t bus;
+    uint8_t dev_func;
+    decode_bus_txn_id(bus_txn_id, &bus, &dev_func);
+
+    mxtl::AutoLock guard(&lock_);
+    DeviceContext* dev;
+    status_t status = GetOrCreateDeviceContextLocked(bus, dev_func, &dev);
+    if (status != MX_OK) {
+        return status;
+    }
+    status = dev->SecondLevelMap(paddr, size, perms, vaddr);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    ASSERT(!caps_.required_write_buf_flushing());
+    // TODO(teisenbe): Integrate finer-grained cache flushing inside of the page
+    // table management
+    __asm__ volatile("wbinvd" : : : "memory");
+
+    return MX_OK;
+}
+
+status_t IommuImpl::Unmap(uint64_t bus_txn_id, dev_vaddr_t vaddr, size_t size) {
+    if (!IS_PAGE_ALIGNED(vaddr) || !IS_PAGE_ALIGNED(size)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+    if (!IsValidBusTxnId(bus_txn_id)) {
+        return MX_ERR_NOT_FOUND;
+    }
+
+    uint8_t bus;
+    uint8_t dev_func;
+    decode_bus_txn_id(bus_txn_id, &bus, &dev_func);
+
+    mxtl::AutoLock guard(&lock_);
+    DeviceContext* dev;
+    status_t status = GetOrCreateDeviceContextLocked(bus, dev_func, &dev);
+    if (status != MX_OK) {
+        return status;
+    }
+    status = dev->SecondLevelUnmap(vaddr, size);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    __asm__ volatile("wbinvd" : : : "memory");
+
+    // TODO: Is this the right flush?
+    // TODO: Do finer grained flushing
+    status = InvalidateContextCacheGlobalLocked();
+    if (status != MX_OK) {
+        return status;
+    }
+    status = InvalidateIotlbGlobalLocked();
+    if (status != MX_OK) {
+        return status;
+    }
+
+    return MX_OK;
+}
+
+status_t IommuImpl::ClearMappingsForBusTxnId(uint64_t bus_txn_id) {
+    PANIC_UNIMPLEMENTED;
+    return MX_ERR_NOT_SUPPORTED;
+}
+
+status_t IommuImpl::Initialize() {
+    mxtl::AutoLock guard(&lock_);
+
+    // Ensure we support this device version
+    auto version = reg::Version::Get().ReadFrom(&mmio_);
+    if (version.major() != 1 && version.minor() != 0) {
+        LTRACEF("Unsupported IOMMU version: %u.%u\n", version.major(), version.minor());
+        return MX_ERR_NOT_SUPPORTED;
+    }
+
+    // Cache useful capability info
+    caps_ = reg::Capability::Get().ReadFrom(&mmio_);
+    extended_caps_ = reg::ExtendedCapability::Get().ReadFrom(&mmio_);
+
+    max_guest_addr_mask_ = (1ULL << (caps_.max_guest_addr_width() + 1)) - 1;
+    fault_recording_reg_offset_ = static_cast<uint32_t>(
+            caps_.fault_recording_register_offset() * 16);
+    num_fault_recording_reg_ = static_cast<uint32_t>(caps_.num_fault_recording_reg() + 1);
+    iotlb_reg_offset_ = static_cast<uint32_t>(extended_caps_.iotlb_register_offset() * 16);
+    if (iotlb_reg_offset_ > PAGE_SIZE - 16) {
+        LTRACEF("Unsupported IOMMU: IOTLB offset runs past the register page\n");
+        return MX_ERR_NOT_SUPPORTED;
+    }
+    supports_extended_context_ = extended_caps_.supports_extended_context();
+    if (extended_caps_.supports_pasid()) {
+        valid_pasid_mask_ = static_cast<uint32_t>((1ULL << (extended_caps_.pasid_size() + 1)) - 1);
+    }
+
+    const uint64_t num_domains_raw = caps_.num_domains();
+    if (num_domains_raw > 0x6) {
+        LTRACEF("Unknown num_domains value\n");
+        return MX_ERR_NOT_SUPPORTED;
+    }
+    const uint32_t num_supported_domains = static_cast<uint32_t>(1ul << (4 + 2 * num_domains_raw));
+    domain_allocator_.set_num_domains(num_supported_domains);
+
+    // Sanity check initial configuration
+    auto global_ctl = reg::GlobalControl::Get().ReadFrom(&mmio_);
+    if (global_ctl.translation_enable()) {
+        LTRACEF("DMA remapping already enabled?!\n");
+        return MX_ERR_BAD_STATE;
+    }
+    if (global_ctl.interrupt_remap_enable()) {
+        LTRACEF("IRQ remapping already enabled?!\n");
+        return MX_ERR_BAD_STATE;
+    }
+
+    // Allocate and setup the root table
+    status_t status = IommuPage::AllocatePage(&root_table_page_);
+    if (status != MX_OK) {
+        LTRACEF("alloc root table failed\n");
+        return status;
+    }
+    status = SetRootTablePointerLocked(root_table_page_.paddr());
+    if (status != MX_OK) {
+        LTRACEF("set root table failed\n");
+        return status;
+    }
+
+    // Enable interrupts before we enable translation
+    status = ConfigureFaultEventInterruptLocked();
+    if (status != MX_OK) {
+        LTRACEF("configuring fault event irq failed\n");
+        return status;
+    }
+
+    status = EnableBiosReservedMappingsLocked();
+    if (status != MX_OK) {
+        LTRACEF("enable bios reserved mappings failed\n");
+        return status;
+    }
+
+    status = SetTranslationEnableLocked(true, current_time() + LK_SEC(1));
+    if (status != MX_OK) {
+        LTRACEF("set translation enable failed\n");
+        return status;
+    }
+
+    return MX_OK;
+}
+
+status_t IommuImpl::EnableBiosReservedMappingsLocked() {
+    auto desc = reinterpret_cast<const mx_iommu_desc_intel_t*>(desc_.get());
+
+    size_t cursor_bytes = 0;
+    while (cursor_bytes + sizeof(mx_iommu_desc_intel_reserved_memory_t) < desc->reserved_memory_bytes) {
+        // The descriptor has already been validated, so no need to check again.
+        auto mem = reinterpret_cast<mx_iommu_desc_intel_reserved_memory_t*>(
+                reinterpret_cast<uintptr_t>(desc) + sizeof(*desc) + desc->scope_bytes +
+                cursor_bytes);
+
+        const size_t num_scopes = mem->scope_bytes / sizeof(mx_iommu_desc_intel_scope_t);
+        auto scopes = reinterpret_cast<mx_iommu_desc_intel_scope_t*>(
+                reinterpret_cast<uintptr_t>(mem) + sizeof(*mem));
+        for (size_t i = 0; i < num_scopes; ++i) {
+            if (scopes[i].num_hops != 1) {
+                // TODO(teisenbe): Implement
+                return MX_ERR_NOT_SUPPORTED;
+            }
+
+            DeviceContext* dev;
+            status_t status = GetOrCreateDeviceContextLocked(scopes[i].start_bus,
+                                                             scopes[i].dev_func[0], &dev);
+            if (status != MX_OK) {
+                return status;
+            }
+
+            LTRACEF("Enabling region [%lx, %lx) for %02x:%02x.%02x\n", mem->base_addr, mem->base_addr + mem->len, scopes[i].start_bus, (uint8_t)(scopes[i].dev_func[0] >> 3), scopes[i].dev_func[0] & 0x7u);
+            dev_vaddr_t vaddr;
+            const uint32_t perms = IOMMU_FLAG_PERM_READ | IOMMU_FLAG_PERM_WRITE;
+            status = dev->SecondLevelMap(mem->base_addr, mem->len, perms, &vaddr);
+            if (status != MX_OK) {
+                return status;
+            }
+            ASSERT(mem->base_addr == vaddr);
+        }
+
+        cursor_bytes += sizeof(*mem) + mem->scope_bytes;
+    }
+
+    ASSERT(!caps_.required_write_buf_flushing());
+    // TODO(teisenbe): Integrate finer-grained cache flushing inside of the page
+    // table management
+    __asm__ volatile("wbinvd" : : : "memory");
+    status_t status = InvalidateContextCacheGlobalLocked();
+    if (status != MX_OK) {
+        return status;
+    }
+
+    status = InvalidateIotlbGlobalLocked();
+    if (status != MX_OK) {
+        return status;
+    }
+
+    return MX_OK;
+}
+
+// Sets the root table pointer and invalidates the context-cache and IOTLB.
+status_t IommuImpl::SetRootTablePointerLocked(paddr_t pa) {
+    DEBUG_ASSERT(IS_PAGE_ALIGNED(pa));
+
+    auto root_table_addr = reg::RootTableAddress::Get().FromValue(0);
+    // If we support extended contexts, use it.
+    root_table_addr.set_root_table_type(supports_extended_context_);
+    root_table_addr.set_root_table_address(pa >> PAGE_SIZE_SHIFT);
+    root_table_addr.WriteTo(&mmio_);
+
+    auto global_ctl = reg::GlobalControl::Get().ReadFrom(&mmio_);
+    DEBUG_ASSERT(!global_ctl.translation_enable());
+    global_ctl.set_root_table_ptr(1);
+    global_ctl.WriteTo(&mmio_);
+    status_t status = WaitForValueLocked(&global_ctl, &decltype(global_ctl)::root_table_ptr,
+                                         1, current_time() + LK_SEC(1));
+    if (status != MX_OK) {
+        LTRACEF("Timed out waiting for root_table_ptr bit to take\n");
+        return status;
+    }
+
+    status = InvalidateContextCacheGlobalLocked();
+    if (status != MX_OK) {
+        return status;
+    }
+
+    status = InvalidateIotlbGlobalLocked();
+    if (status != MX_OK) {
+        return status;
+    }
+
+    return MX_OK;
+}
+
+status_t IommuImpl::SetTranslationEnableLocked(bool enabled, lk_time_t deadline) {
+    auto global_ctl = reg::GlobalControl::Get().ReadFrom(&mmio_);
+    global_ctl.set_translation_enable(enabled);
+    global_ctl.WriteTo(&mmio_);
+
+    return WaitForValueLocked(&global_ctl, &decltype(global_ctl)::translation_enable,
+                              enabled, deadline);
+}
+
+status_t IommuImpl::InvalidateContextCacheGlobalLocked() {
+    DEBUG_ASSERT(lock_.IsHeld());
+
+    auto context_cmd = reg::ContextCommand::Get().FromValue(0);
+    context_cmd.set_invld_context_cache(1);
+    context_cmd.set_invld_request_granularity(reg::ContextCommand::kGlobalInvld);
+    context_cmd.WriteTo(&mmio_);
+
+    return WaitForValueLocked(&context_cmd, &decltype(context_cmd)::invld_context_cache, 0,
+                              INFINITE_TIME);
+}
+
+status_t IommuImpl::InvalidateIotlbGlobalLocked() {
+    DEBUG_ASSERT(lock_.IsHeld());
+
+    auto iotlb_invld = reg::IotlbInvalidate::Get(iotlb_reg_offset_).ReadFrom(&mmio_);
+    iotlb_invld.set_invld_iotlb(1);
+    iotlb_invld.set_invld_request_granularity(reg::IotlbInvalidate::kGlobalInvld);
+    iotlb_invld.WriteTo(&mmio_);
+
+    return WaitForValueLocked(&iotlb_invld, &decltype(iotlb_invld)::invld_iotlb, 0,
+                              INFINITE_TIME);
+}
+
+template <class RegType>
+status_t IommuImpl::WaitForValueLocked(RegType* reg,
+                                       typename RegType::ValueType (RegType::*getter)(),
+                                       typename RegType::ValueType value,
+                                       lk_time_t deadline) {
+    DEBUG_ASSERT(lock_.IsHeld());
+
+    const lk_time_t kMaxSleepDuration = LK_USEC(10);
+
+    while (true) {
+        reg->ReadFrom(&mmio_);
+        if ((reg->*getter)() == value) {
+            return MX_OK;
+        }
+
+        const lk_time_t now = current_time();
+        if (now > deadline) {
+            break;
+        }
+
+        lk_time_t sleep_deadline = mxtl::min(now + kMaxSleepDuration, deadline);
+        thread_sleep(sleep_deadline);
+    }
+    return MX_ERR_TIMED_OUT;
+}
+
+enum handler_return IommuImpl::FaultHandler(void* ctx) {
+    auto self = static_cast<IommuImpl*>(ctx);
+
+    auto status = reg::FaultStatus::Get().ReadFrom(&self->mmio_);
+
+    if (!status.primary_pending_fault()) {
+        TRACEF("Non primary fault\n");
+        return INT_NO_RESCHEDULE;
+    }
+
+    auto caps = reg::Capability::Get().ReadFrom(&self->mmio_);
+    const uint32_t num_regs = static_cast<uint32_t>(caps.num_fault_recording_reg() + 1);
+    const uint32_t reg_offset = static_cast<uint32_t>(caps.fault_recording_register_offset() * 16);
+
+    uint32_t index = status.fault_record_index();
+    while (1) {
+        auto rec_high = reg::FaultRecordHigh::Get(reg_offset, index).ReadFrom(&self->mmio_);
+        if (!rec_high.fault()) {
+            break;
+        }
+        auto rec_low = reg::FaultRecordLow::Get(reg_offset, index).ReadFrom(&self->mmio_);
+        uint64_t source = rec_high.source_id();
+        TRACEF("IOMMU Fault: access %c, PASID (%c) %#04lx, reason %#02lx, source %02lx:%02lx.%lx, info: %lx\n",
+               rec_high.request_type() ? 'R' : 'W',
+               rec_high.pasid_present() ? 'V' : '-',
+               rec_high.pasid_value(),
+               rec_high.fault_reason(),
+               source >> 8, (source >> 3) & 0x1f, source & 0x7,
+               rec_low.fault_info() << 12);
+
+        // Clear this fault (RW1CS)
+        rec_high.WriteTo(&self->mmio_);
+
+        ++index;
+        if (index >= num_regs) {
+            index -= num_regs;
+        }
+    }
+
+    status.set_reg_value(0);
+    // Clear the primary fault overflow condition (RW1CS)
+    // TODO(teisenbe): How do we guarantee we get an interrupt on the next fault/if we left a fault unprocessed?
+    status.set_primary_fault_overflow(1);
+    status.WriteTo(&self->mmio_);
+
+    return INT_NO_RESCHEDULE;
+}
+
+status_t IommuImpl::ConfigureFaultEventInterruptLocked() {
+    DEBUG_ASSERT(lock_.IsHeld());
+
+    auto& pcie_platform = PcieBusDriver::GetDriver()->platform();
+    if (!pcie_platform.supports_msi()) {
+        return MX_ERR_NOT_SUPPORTED;
+    }
+    status_t status = pcie_platform.AllocMsiBlock(1, false/* can_target_64bit */,
+                                                  false /* msi x */, &irq_block_);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    auto event_data = reg::FaultEventData::Get().FromValue(irq_block_.tgt_data);
+    auto event_addr = reg::FaultEventAddress::Get().FromValue(
+            static_cast<uint32_t>(irq_block_.tgt_addr));
+    auto event_upper_addr = reg::FaultEventUpperAddress::Get().FromValue(
+            static_cast<uint32_t>(irq_block_.tgt_addr >> 32));
+
+    event_data.WriteTo(&mmio_);
+    event_addr.WriteTo(&mmio_);
+    event_upper_addr.WriteTo(&mmio_);
+
+    // Clear all primary fault records
+    for (uint32_t i = 0; i < num_fault_recording_reg_; ++i) {
+        const uint32_t offset = fault_recording_reg_offset_;
+        auto record_high = reg::FaultRecordHigh::Get(offset, i).ReadFrom(&mmio_);
+        record_high.WriteTo(&mmio_);
+    }
+
+    // Clear all pending faults
+    auto fault_status_ctl = reg::FaultStatus::Get().ReadFrom(&mmio_);
+    fault_status_ctl.WriteTo(&mmio_);
+
+    pcie_platform.RegisterMsiHandler(&irq_block_, 0, FaultHandler, this);
+
+    // Unmask interrupts
+    auto fault_event_ctl = reg::FaultEventControl::Get().ReadFrom(&mmio_);
+    fault_event_ctl.set_interrupt_mask(0);
+    fault_event_ctl.WriteTo(&mmio_);
+
+    return MX_OK;
+}
+
+void IommuImpl::DisableFaultsLocked() {
+    auto fault_event_ctl = reg::FaultEventControl::Get().ReadFrom(&mmio_);
+    fault_event_ctl.set_interrupt_mask(1);
+    fault_event_ctl.WriteTo(&mmio_);
+}
+
+status_t IommuImpl::GetOrCreateContextTableLocked(uint8_t bus, uint8_t dev_func,
+                                                  ContextTableState** tbl) {
+    DEBUG_ASSERT(lock_.IsHeld());
+
+    volatile ds::RootTable* root_table = this->root_table();
+    DEBUG_ASSERT(root_table);
+
+    volatile ds::RootEntrySubentry* target_entry = &root_table->entry[bus].lower;
+    if (supports_extended_context_ && dev_func >= 0x80) {
+        // If this is an extended root table and the device is in the upper half
+        // of the bus address space, use the upper pointer.
+        target_entry = &root_table->entry[bus].upper;
+    }
+
+    ds::RootEntrySubentry entry;
+    entry.ReadFrom(target_entry);
+    if (entry.present()) {
+        // We know the entry exists, so search our list of tables for it.
+        for (ContextTableState& context_table : context_tables_) {
+            if (context_table.includes_bdf(bus, dev_func)) {
+                *tbl = &context_table;
+                return MX_OK;
+            }
+        }
+    }
+
+    // Couldn't find the ContextTable, so create it.
+    mxtl::unique_ptr<ContextTableState> table;
+    status_t status = ContextTableState::Create(bus, supports_extended_context_,
+                                                dev_func >= 0x80 /* upper */,
+                                                this,
+                                                target_entry,
+                                                &table);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    *tbl = table.get();
+    context_tables_.push_back(mxtl::move(table));
+
+    return MX_OK;
+}
+
+status_t IommuImpl::GetOrCreateDeviceContextLocked(uint8_t bus, uint8_t dev_func,
+                                                   DeviceContext** context) {
+
+    DEBUG_ASSERT(lock_.IsHeld());
+
+    ContextTableState* ctx_table_state;
+    status_t status = GetOrCreateContextTableLocked(bus, dev_func, &ctx_table_state);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    status = ctx_table_state->GetDeviceContext(bus, dev_func, context);
+    if (status != MX_ERR_NOT_FOUND) {
+        // Either status was MX_OK and we're done, or some error occurred.
+        return status;
+    }
+
+    uint32_t domain_id;
+    status = domain_allocator_.Allocate(&domain_id);
+    if (status != MX_OK) {
+        return status;
+    }
+    return ctx_table_state->CreateDeviceContext(bus, dev_func, domain_id, context);
+}
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/iommu_impl.h

@@ -0,0 +1,129 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <bits.h>
+#include <dev/pcie_bus_driver.h>
+#include <dev/iommu.h>
+#include <hwreg/mmio.h>
+#include <magenta/syscalls/iommu.h>
+#include <mxtl/intrusive_double_list.h>
+#include <mxtl/macros.h>
+#include <mxtl/mutex.h>
+
+#include "domain_allocator.h"
+#include "hw.h"
+#include "iommu_page.h"
+
+class VmMapping;
+class VmObject;
+
+namespace intel_iommu {
+
+class ContextTableState;
+class DeviceContext;
+
+class IommuImpl final : public Iommu {
+public:
+
+    static status_t Create(mxtl::unique_ptr<const uint8_t[]> desc, uint32_t desc_len,
+                           mxtl::RefPtr<Iommu>* out);
+
+    bool IsValidBusTxnId(uint64_t bus_txn_id) const final;
+
+    status_t Map(uint64_t bus_txn_id, paddr_t paddr, size_t size, uint32_t perms,
+                 dev_vaddr_t* vaddr) final;
+    status_t Unmap(uint64_t bus_txn_id, dev_vaddr_t vaddr, size_t size) final;
+
+    status_t ClearMappingsForBusTxnId(uint64_t bus_txn_id) final;
+
+    ~IommuImpl() final;
+
+    reg::Capability* caps() TA_NO_THREAD_SAFETY_ANALYSIS { return &caps_; }
+    reg::ExtendedCapability* extended_caps() TA_NO_THREAD_SAFETY_ANALYSIS {
+        return &extended_caps_;
+    }
+private:
+    DISALLOW_COPY_ASSIGN_AND_MOVE(IommuImpl);
+    IommuImpl(volatile void* register_base, mxtl::unique_ptr<const uint8_t[]> desc,
+              uint32_t desc_len);
+
+    static void decode_bus_txn_id(uint64_t bus_txn_id, uint8_t* bus, uint8_t* dev_func) {
+        *bus = static_cast<uint8_t>(BITS_SHIFT(bus_txn_id, 15, 8));
+        *dev_func = static_cast<uint8_t>(BITS_SHIFT(bus_txn_id, 7, 0));
+    }
+
+    static status_t ValidateIommuDesc(const mxtl::unique_ptr<const uint8_t[]>& desc,
+                                      uint32_t desc_len);
+
+    // Set up initial root structures and enable translation
+    status_t Initialize();
+
+    status_t InvalidateContextCacheGlobalLocked() TA_REQ(lock_);
+    status_t InvalidateIotlbGlobalLocked() TA_REQ(lock_);
+
+    status_t SetRootTablePointerLocked(paddr_t pa) TA_REQ(lock_);
+    status_t SetTranslationEnableLocked(bool enabled, lk_time_t deadline) TA_REQ(lock_);
+    status_t ConfigureFaultEventInterruptLocked() TA_REQ(lock_);
+
+    // Process Reserved Memory Mapping Regions and set them up as pass-through.
+    status_t EnableBiosReservedMappingsLocked() TA_REQ(lock_);
+
+    void DisableFaultsLocked() TA_REQ(lock_);
+    static enum handler_return FaultHandler(void* ctx);
+    status_t GetOrCreateContextTableLocked(uint8_t bus, uint8_t dev_func,
+                                           ContextTableState** tbl) TA_REQ(lock_);
+    status_t GetOrCreateDeviceContextLocked(uint8_t bus, uint8_t dev_func,
+                                            DeviceContext** context) TA_REQ(lock_);
+
+    // Utility for waiting until a register field changes to a value, timing out
+    // if the deadline elapses.  If deadline is INFINITE_TIME, then will never time
+    // out.  Can only return NO_ERROR and ERR_TIMED_OUT.
+    template <class RegType>
+    status_t WaitForValueLocked(RegType* reg,
+                                typename RegType::ValueType (RegType::*getter)(),
+                                typename RegType::ValueType value,
+                                lk_time_t deadline) TA_REQ(lock_);
+
+    volatile ds::RootTable* root_table() const TA_REQ(lock_) {
+        return reinterpret_cast<volatile ds::RootTable*>(root_table_page_.vaddr());
+    }
+
+    mxtl::Mutex lock_;
+
+    // Descriptor of this hardware unit
+    mxtl::unique_ptr<const uint8_t[]> desc_;
+    uint32_t desc_len_;
+
+    // Location of the memory-mapped hardware register bank.
+    hwreg::RegisterIo mmio_ TA_GUARDED(lock_);
+
+    // Interrupt allocation
+    pcie_msi_block_t irq_block_ TA_GUARDED(lock_);
+
+    // In-memory root table
+    IommuPage root_table_page_ TA_GUARDED(lock_);
+    // List of allocated context tables
+    mxtl::DoublyLinkedList<mxtl::unique_ptr<ContextTableState>> context_tables_ TA_GUARDED(lock_);
+
+    DomainAllocator domain_allocator_ TA_GUARDED(lock_);
+
+    // A mask with bits set for each usable bit in an address with the largest allowed
+    // address width.  E.g., if the largest allowed width is 48-bit,
+    // max_guest_addr_mask will be 0xffff_ffff_ffff.
+    uint64_t max_guest_addr_mask_ TA_GUARDED(lock_) = 0;
+    uint32_t valid_pasid_mask_ TA_GUARDED(lock_) = 0;
+    uint32_t iotlb_reg_offset_ TA_GUARDED(lock_) = 0;
+    uint32_t fault_recording_reg_offset_ TA_GUARDED(lock_) = 0;
+    uint32_t num_fault_recording_reg_ TA_GUARDED(lock_) = 0;
+    bool supports_extended_context_ TA_GUARDED(lock_) = 0;
+
+    reg::Capability caps_ TA_GUARDED(lock_);
+    reg::ExtendedCapability extended_caps_ TA_GUARDED(lock_);
+};
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/iommu_page.cpp

@@ -0,0 +1,53 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#include "iommu_page.h"
+
+#include <kernel/vm.h>
+#include <kernel/vm/vm_aspace.h>
+
+namespace intel_iommu {
+
+IommuPage::IommuPage(vm_page_t* page, uintptr_t virt) : page_(page), virt_(virt) {
+}
+
+IommuPage::~IommuPage() {
+    if (page_) {
+        VmAspace::kernel_aspace()->FreeRegion(reinterpret_cast<vaddr_t>(virt_));
+        pmm_free_page(page_);
+    }
+}
+
+status_t IommuPage::AllocatePage(IommuPage* out) {
+    paddr_t paddr;
+    vm_page_t* page = pmm_alloc_page(0, &paddr);
+    if (!page) {
+        return MX_ERR_NO_MEMORY;
+    }
+    page->state = VM_PAGE_STATE_IOMMU;
+
+    void* vaddr;
+    auto kernel_aspace = VmAspace::kernel_aspace();
+    status_t status = kernel_aspace->AllocPhysical(
+            "iommu_ctx_tbl",
+            PAGE_SIZE,
+            &vaddr,
+            PAGE_SIZE_SHIFT,
+            paddr,
+            0,
+            ARCH_MMU_FLAG_PERM_READ | ARCH_MMU_FLAG_PERM_WRITE);
+    if (status != MX_OK) {
+        pmm_free_page(page);
+        return status;
+    }
+
+    arch_zero_page(vaddr);
+
+    *out = IommuPage(page, reinterpret_cast<uintptr_t>(vaddr));
+    return MX_OK;
+}
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/iommu_page.h

@@ -0,0 +1,52 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <err.h>
+#include <kernel/vm/pmm.h>
+#include <mxtl/macros.h>
+#include <stdint.h>
+
+namespace intel_iommu {
+
+// RAII object for managing the lifetime of the memory that backs hardware
+// datastructures.
+class IommuPage {
+public:
+    IommuPage() : page_(nullptr), virt_(0) { }
+    ~IommuPage();
+
+    IommuPage(IommuPage&& p) : page_(p.page_), virt_(p.virt_) {
+        p.page_ = nullptr;
+        p.virt_ = 0;
+    }
+    IommuPage& operator=(IommuPage&& p) {
+        page_ = p.page_;
+        virt_ = p.virt_;
+        p.page_ = nullptr;
+        p.virt_ = 0;
+        return *this;
+    }
+
+    static status_t AllocatePage(IommuPage* out);
+
+    uintptr_t vaddr() const {
+        return virt_;
+    }
+    paddr_t paddr() const {
+        return likely(page_) ? vm_page_to_paddr(page_) : UINT64_MAX;
+    }
+
+private:
+    IommuPage(vm_page_t* page, uintptr_t virt);
+    DISALLOW_COPY_AND_ASSIGN_ALLOW_MOVE(IommuPage);
+
+    vm_page_t* page_;
+    uintptr_t virt_;
+};
+
+} // namespace intel_iommu

kernel/dev/iommu/intel/rules.mk

@@ -0,0 +1,25 @@
+# Copyright 2017 The Fuchsia Authors
+#
+# Use of this source code is governed by a MIT-style
+# license that can be found in the LICENSE file or at
+# https://opensource.org/licenses/MIT
+
+LOCAL_DIR := $(GET_LOCAL_DIR)
+MODULE := $(LOCAL_DIR)
+
+MODULE_SRCS := \
+    $(LOCAL_DIR)/context_table_state.cpp \
+    $(LOCAL_DIR)/device_context.cpp \
+    $(LOCAL_DIR)/domain_allocator.cpp \
+    $(LOCAL_DIR)/intel_iommu.cpp \
+    $(LOCAL_DIR)/iommu_impl.cpp \
+    $(LOCAL_DIR)/iommu_page.cpp \
+
+MODULE_DEPS := \
+	kernel/dev/pcie \
+	kernel/lib/bitmap \
+	kernel/lib/hwreg \
+	kernel/lib/mxtl \
+	third_party/lib/safeint \
+
+include make/module.mk

kernel/include/dev/iommu.h

@@ -0,0 +1,61 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <kernel/mutex.h>
+#include <magenta/thread_annotations.h>
+#include <mxtl/intrusive_double_list.h>
+#include <mxtl/ref_counted.h>
+#include <mxtl/ref_ptr.h>
+
+#include <stdbool.h>
+#include <sys/types.h>
+#include <inttypes.h>
+
+#define IOMMU_FLAG_PERM_READ    (1<<0)
+#define IOMMU_FLAG_PERM_WRITE   (1<<1)
+#define IOMMU_FLAG_PERM_EXECUTE (1<<2)
+
+// Type used to refer to virtual addresses presented to a device by the IOMMU.
+typedef uint64_t dev_vaddr_t;
+
+class Iommu : public mxtl::RefCounted<Iommu>,
+              public mxtl::DoublyLinkedListable<mxtl::RefPtr<Iommu>> {
+public:
+    // Check if |bus_txn_id| is valid for this IOMMU (i.e. could be used
+    // to configure a device).
+    virtual bool IsValidBusTxnId(uint64_t bus_txn_id) const = 0;
+
+    // Grant the device identified by |bus_txn_id| access to the range of
+    // physical addresses given by [paddr, paddr + size).  The base of the
+    // mapped range is returned via |vaddr|.  |vaddr| must not be NULL.
+    //
+    // |perms| defines the access permissions, using the IOMMU_FLAG_PERM_*
+    // flags.
+    //
+    // Returns MX_ERR_INVALID_ARGS if:
+    //  |size| is not a multiple of PAGE_SIZE
+    //  |paddr| is not aligned to PAGE_SIZE
+    // Returns MX_ERR_NOT_FOUND if |bus_txn_id| is not valid.
+    virtual status_t Map(uint64_t bus_txn_id, paddr_t paddr, size_t size, uint32_t perms,
+                         dev_vaddr_t* vaddr) = 0;
+
+    // Revoke access to the range of addresses [vaddr, vaddr + size) for the
+    // device identified by |bus_txn_id|.
+    //
+    // Returns MX_ERR_INVALID_ARGS if:
+    //  |size| is not a multiple of PAGE_SIZE
+    //  |vaddr| is not aligned to PAGE_SIZE
+    // Returns MX_ERR_NOT_FOUND if |bus_txn_id| is not valid.
+    virtual status_t Unmap(uint64_t bus_txn_id, dev_vaddr_t vaddr, size_t size) = 0;
+
+    // Remove all mappings for |bus_txn_id|.
+    // Returns MX_ERR_NOT_FOUND if |bus_txn_id| is not valid.
+    virtual status_t ClearMappingsForBusTxnId(uint64_t bus_txn_id) = 0;
+
+    virtual ~Iommu() { }
+};

kernel/include/kernel/vm/page.h

@@ -56,6 +56,7 @@ enum vm_page_state {
     VM_PAGE_STATE_HEAP,
     VM_PAGE_STATE_OBJECT,
     VM_PAGE_STATE_MMU, /* allocated to serve arch-specific mmu purposes */
+    VM_PAGE_STATE_IOMMU, /* allocated for platform-specific iommu structures */
 
     _VM_PAGE_STATE_COUNT
 };

kernel/lib/hwreg/rules.mk

@@ -0,0 +1,17 @@
+# Copyright 2017 The Fuchsia Authors
+#
+# Use of this source code is governed by a MIT-style
+# license that can be found in the LICENSE file or at
+# https://opensource.org/licenses/MIT
+
+SRC_DIR := system/ulib/hwreg
+LOCAL_DIR := $(GET_LOCAL_DIR)
+
+MODULE := $(LOCAL_DIR)
+
+KERNEL_INCLUDES += $(SRC_DIR)/include
+
+MODULE_SRCS := $(LOCAL_DIR)/empty.cpp
+MODULE_DEPS := kernel/lib/mxtl
+
+include make/module.mk

kernel/lib/magenta/bus_transaction_initiator_dispatcher.cpp

@@ -0,0 +1,123 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#include <magenta/bus_transaction_initiator_dispatcher.h>
+
+#include <dev/iommu.h>
+#include <err.h>
+#include <kernel/vm/vm_object.h>
+#include <magenta/rights.h>
+#include <mxcpp/new.h>
+#include <mxtl/auto_lock.h>
+
+status_t BusTransactionInitiatorDispatcher::Create(mxtl::RefPtr<Iommu> iommu, uint64_t bti_id,
+                                                   mxtl::RefPtr<Dispatcher>* dispatcher,
+                                                   mx_rights_t* rights) {
+
+    if (!iommu->IsValidBusTxnId(bti_id)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+
+    mxtl::AllocChecker ac;
+    auto disp = new (&ac) BusTransactionInitiatorDispatcher(mxtl::move(iommu), bti_id);
+    if (!ac.check()) {
+        return MX_ERR_NO_MEMORY;
+    }
+
+    *rights = MX_DEFAULT_BTI_RIGHTS;
+    *dispatcher = mxtl::AdoptRef<Dispatcher>(disp);
+    return MX_OK;
+}
+
+BusTransactionInitiatorDispatcher::BusTransactionInitiatorDispatcher(mxtl::RefPtr<Iommu> iommu,
+                                                                     uint64_t bti_id)
+        : iommu_(mxtl::move(iommu)), bti_id_(bti_id), state_tracker_(0u), zero_handles_(false) {}
+
+BusTransactionInitiatorDispatcher::~BusTransactionInitiatorDispatcher() {
+    DEBUG_ASSERT(pinned_memory_.is_empty());
+}
+
+status_t BusTransactionInitiatorDispatcher::Pin(mxtl::RefPtr<VmObject> vmo, uint64_t offset,
+                                                uint64_t size, uint32_t perms,
+                                                uint64_t* mapped_extents,
+                                                size_t mapped_extents_len,
+                                                size_t* actual_mapped_extents_len) {
+
+    DEBUG_ASSERT(mapped_extents);
+    DEBUG_ASSERT(IS_PAGE_ALIGNED(offset));
+    DEBUG_ASSERT(actual_mapped_extents_len);
+    if (!IS_PAGE_ALIGNED(offset)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+
+    mxtl::AutoLock guard(&lock_);
+
+    if (zero_handles_) {
+        return MX_ERR_BAD_STATE;
+    }
+
+    mxtl::unique_ptr<PinnedMemoryObject> pmo;
+    status_t status = PinnedMemoryObject::Create(*this, mxtl::move(vmo),
+                                                 offset, size, perms, &pmo);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    const auto& pmo_addrs = pmo->mapped_extents();
+    const size_t found_extents = pmo->mapped_extents_len();
+    if (mapped_extents_len < found_extents)  {
+        return MX_ERR_BUFFER_TOO_SMALL;
+    }
+
+    // Copy out addrs
+    DEBUG_ASSERT(pmo->mapped_extents_len() <= ROUNDUP(size, PAGE_SIZE) / PAGE_SIZE);
+    for (size_t i = 0; i < found_extents; ++i) {
+        mapped_extents[i] = pmo_addrs[i];
+    }
+
+    *actual_mapped_extents_len = found_extents;
+    pinned_memory_.push_back(mxtl::move(pmo));
+    return MX_OK;
+}
+
+status_t BusTransactionInitiatorDispatcher::Unpin(const uint64_t* mapped_extents,
+                                                  size_t mapped_extents_len) {
+    mxtl::AutoLock guard(&lock_);
+
+    if (zero_handles_) {
+        return MX_ERR_BAD_STATE;
+    }
+
+    for (auto& pmo : pinned_memory_) {
+        if (pmo.mapped_extents_len() != mapped_extents_len) {
+            continue;
+        }
+
+        const auto& pmo_extents = pmo.mapped_extents();
+        bool match = true;
+        for (size_t i = 0; i < mapped_extents_len ; ++i) {
+            if (mapped_extents[i] != pmo_extents[i]) {
+                match = false;
+                break;
+            }
+        }
+        if (match) {
+            // The PMO dtor will take care of the actual unpinning.
+            pinned_memory_.erase(pmo);
+            return MX_OK;
+        }
+    }
+
+    return MX_ERR_INVALID_ARGS;
+}
+
+void BusTransactionInitiatorDispatcher::on_zero_handles() {
+    mxtl::AutoLock guard(&lock_);
+    while (!pinned_memory_.is_empty()) {
+        pinned_memory_.pop_front();
+    }
+    zero_handles_ = true;
+}

kernel/lib/magenta/diagnostics.cpp

@@ -59,7 +59,7 @@ static void DumpProcessListKeyMap() {
 }
 
 static const char* ObjectTypeToString(mx_obj_type_t type) {
-    static_assert(MX_OBJ_TYPE_LAST == 23, "need to update switch below");
+    static_assert(MX_OBJ_TYPE_LAST == 25, "need to update switch below");
 
     switch (type) {
         case MX_OBJ_TYPE_PROCESS: return "process";
@@ -80,6 +80,8 @@ static const char* ObjectTypeToString(mx_obj_type_t type) {
         case MX_OBJ_TYPE_GUEST: return "guest";
         case MX_OBJ_TYPE_VCPU: return "vcpu";
         case MX_OBJ_TYPE_TIMER: return "timer";
+        case MX_OBJ_TYPE_IOMMU: return "iommu";
+        case MX_OBJ_TYPE_BTI: return "bti";
         default: return "???";
     }
 }

kernel/lib/magenta/include/magenta/bus_transaction_initiator_dispatcher.h

@@ -0,0 +1,62 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <dev/iommu.h>
+#include <magenta/dispatcher.h>
+#include <magenta/pinned_memory_object.h>
+#include <magenta/state_tracker.h>
+#include <mxtl/canary.h>
+#include <mxtl/mutex.h>
+
+#include <sys/types.h>
+
+class Iommu;
+
+class BusTransactionInitiatorDispatcher final : public Dispatcher {
+public:
+    static status_t Create(mxtl::RefPtr<Iommu> iommu, uint64_t bti_id,
+                           mxtl::RefPtr<Dispatcher>* dispatcher, mx_rights_t* rights);
+
+    ~BusTransactionInitiatorDispatcher() final;
+    mx_obj_type_t get_type() const final { return MX_OBJ_TYPE_BTI; }
+    StateTracker* get_state_tracker() final { return &state_tracker_; }
+
+    // Pins the given VMO range and writes the addresses into |mapped_extents|.  The
+    // number of addresses returned is given in |actual_mapped_extents_len|.
+    //
+    // Returns MX_ERR_INVALID_ARGS if |offset| or |size| are not PAGE_SIZE aligned.
+    // Returns MX_ERR_INVALID_ARGS if |perms| is not suitable to pass to the Iommu::Map() interface.
+    // Returns MX_ERR_BUFFER_TOO_SMALL if |mapped_extents_len| is not at least |size|/PAGE_SIZE.
+    status_t Pin(mxtl::RefPtr<VmObject> vmo, uint64_t offset, uint64_t size, uint32_t perms,
+                 uint64_t* mapped_extents, size_t mapped_extents_len,
+                 size_t* actual_mapped_extents_len);
+
+    // Unpins the given list of extents.  Returns an error if the described
+    // list of extents do not correspond to the exact set created in a
+    // previous call to Pin().
+    status_t Unpin(const uint64_t* mapped_extents, size_t mapped_extents_len);
+
+    void on_zero_handles() final;
+
+    mxtl::RefPtr<Iommu> iommu() const { return iommu_; }
+    uint64_t bti_id() const { return bti_id_; }
+
+private:
+    BusTransactionInitiatorDispatcher(mxtl::RefPtr<Iommu> iommu, uint64_t bti_id);
+
+    mxtl::Canary<mxtl::magic("BTID")> canary_;
+
+    mxtl::Mutex lock_;
+    const mxtl::RefPtr<Iommu> iommu_;
+    const uint64_t bti_id_;
+
+    StateTracker state_tracker_;
+
+    mxtl::DoublyLinkedList<mxtl::unique_ptr<PinnedMemoryObject>> pinned_memory_ TA_GUARDED(lock_);
+    bool zero_handles_ TA_GUARDED(lock_);
+};

kernel/lib/magenta/include/magenta/dispatcher.h

@@ -44,6 +44,8 @@ DECLARE_DISPTAG(FifoDispatcher, MX_OBJ_TYPE_FIFO)
 DECLARE_DISPTAG(GuestDispatcher, MX_OBJ_TYPE_GUEST)
 DECLARE_DISPTAG(VcpuDispatcher, MX_OBJ_TYPE_VCPU)
 DECLARE_DISPTAG(TimerDispatcher, MX_OBJ_TYPE_TIMER)
+DECLARE_DISPTAG(IommuDispatcher, MX_OBJ_TYPE_IOMMU)
+DECLARE_DISPTAG(BusTransactionInitiatorDispatcher, MX_OBJ_TYPE_BTI)
 
 #undef DECLARE_DISPTAG
 

kernel/lib/magenta/include/magenta/iommu_dispatcher.h

@@ -0,0 +1,32 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <dev/iommu.h>
+#include <magenta/dispatcher.h>
+#include <magenta/state_tracker.h>
+#include <mxtl/canary.h>
+
+#include <sys/types.h>
+
+class IommuDispatcher final : public Dispatcher {
+public:
+    static mx_status_t Create(uint32_t type, mxtl::unique_ptr<const uint8_t[]> desc,
+                              uint32_t desc_len, mxtl::RefPtr<Dispatcher>* dispatcher,
+                              mx_rights_t* rights);
+
+    ~IommuDispatcher() final;
+    mx_obj_type_t get_type() const final { return MX_OBJ_TYPE_IOMMU; }
+
+    mxtl::RefPtr<Iommu> iommu() const { return iommu_; }
+
+private:
+    explicit IommuDispatcher(mxtl::RefPtr<Iommu> iommu);
+
+    mxtl::Canary<mxtl::magic("IOMD")> canary_;
+    const mxtl::RefPtr<Iommu> iommu_;
+};

kernel/lib/magenta/include/magenta/pinned_memory_object.h

@@ -0,0 +1,74 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#pragma once
+
+#include <dev/iommu.h>
+#include <err.h>
+#include <mxtl/canary.h>
+#include <mxtl/intrusive_double_list.h>
+#include <mxtl/unique_ptr.h>
+
+#include <sys/types.h>
+
+class BusTransactionInitiatorDispatcher;
+class VmObject;
+
+class PinnedMemoryObject final : public mxtl::DoublyLinkedListable<mxtl::unique_ptr<PinnedMemoryObject>> {
+public:
+    // Pin memory in |vmo|'s range [offset, offset+size) on behalf of |bti|,
+    // with permissions specified by |perms|.  |perms| should be flags suitable
+    // for the Iommu::Map() interface.
+    static status_t Create(const BusTransactionInitiatorDispatcher& bti,
+                           mxtl::RefPtr<VmObject> vmo, size_t offset,
+                           size_t size, uint32_t perms,
+                           mxtl::unique_ptr<PinnedMemoryObject>* out);
+    ~PinnedMemoryObject();
+
+    class Extent {
+     public:
+        // Implicit conversion to uint64_t
+        operator uint64_t() const { return val_; }
+
+        Extent() : val_(0) { }
+        Extent(uint64_t base, size_t pages) : val_(base | (pages - 1)) {
+            ASSERT(pages > 0 && pages <= PAGE_SIZE);
+        }
+        uint64_t base() const { return val_ & ~(PAGE_SIZE - 1); }
+        size_t pages() const { return (val_ & (PAGE_SIZE - 1)) + 1; }
+        status_t extend(size_t num_pages) {
+            if (pages() + num_pages > PAGE_SIZE) return MX_ERR_OUT_OF_RANGE;
+            val_ += num_pages;
+            return MX_OK;
+        }
+     private:
+        uint64_t val_;
+    };
+
+    // Returns an array of the extents usable by the given device
+    const mxtl::unique_ptr<Extent[]>& mapped_extents() const { return mapped_extents_; }
+    size_t mapped_extents_len() const { return mapped_extents_len_; }
+private:
+    PinnedMemoryObject(const BusTransactionInitiatorDispatcher& bti,
+                       mxtl::RefPtr<VmObject> vmo, size_t offset, size_t size,
+                       bool is_contiguous,
+                       mxtl::unique_ptr<Extent[]> mapped_extents);
+    DISALLOW_COPY_ASSIGN_AND_MOVE(PinnedMemoryObject);
+
+    status_t MapIntoIommu(uint32_t perms);
+    status_t UnmapFromIommu();
+
+    mxtl::Canary<mxtl::magic("PMO_")> canary_;
+
+    const mxtl::RefPtr<VmObject> vmo_;
+    const uint64_t offset_;
+    const uint64_t size_;
+    const bool is_contiguous_;
+
+    const BusTransactionInitiatorDispatcher& bti_;
+    const mxtl::unique_ptr<Extent[]> mapped_extents_;
+    size_t mapped_extents_len_;
+};

kernel/lib/magenta/iommu_dispatcher.cpp

@@ -0,0 +1,67 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+#include <magenta/iommu_dispatcher.h>
+
+#include <dev/iommu.h>
+#include <magenta/rights.h>
+#include <magenta/syscalls/iommu.h>
+#include <mxcpp/new.h>
+
+#include <assert.h>
+#include <err.h>
+#include <inttypes.h>
+#include <trace.h>
+
+#if WITH_DEV_IOMMU_DUMMY
+#include <dev/iommu/dummy.h>
+#endif // WITH_DEV_IOMMU_DUMMY
+#if WITH_DEV_IOMMU_INTEL
+#include <dev/iommu/intel.h>
+#endif // WITH_DEV_IOMMU_INTEL
+
+#define LOCAL_TRACE 0
+
+mx_status_t IommuDispatcher::Create(uint32_t type, mxtl::unique_ptr<const uint8_t[]> desc,
+                                    uint32_t desc_len, mxtl::RefPtr<Dispatcher>* dispatcher,
+                                    mx_rights_t* rights) {
+
+    mxtl::RefPtr<Iommu> iommu;
+    mx_status_t status;
+    switch (type) {
+#if WITH_DEV_IOMMU_DUMMY
+        case MX_IOMMU_TYPE_DUMMY:
+            status = DummyIommu::Create(mxtl::move(desc), desc_len, &iommu);
+            break;
+#endif // WITH_DEV_IOMMU_DUMMY
+#if WITH_DEV_IOMMU_INTEL
+        case MX_IOMMU_TYPE_INTEL:
+            status = IntelIommu::Create(mxtl::move(desc), desc_len, &iommu);
+            break;
+#endif // WITH_DEV_IOMMU_INTEL
+        default:
+            return MX_ERR_NOT_SUPPORTED;
+    }
+    if (status != MX_OK) {
+        return status;
+    }
+
+    mxtl::AllocChecker ac;
+    auto disp = new (&ac) IommuDispatcher(mxtl::move(iommu));
+    if (!ac.check())
+        return MX_ERR_NO_MEMORY;
+
+    *rights = MX_DEFAULT_IOMMU_RIGHTS;
+    *dispatcher = mxtl::AdoptRef<Dispatcher>(disp);
+    return MX_OK;
+}
+
+IommuDispatcher::IommuDispatcher(mxtl::RefPtr<Iommu> iommu)
+    : iommu_(iommu) {}
+
+IommuDispatcher::~IommuDispatcher() {
+}
+

kernel/lib/magenta/pinned_memory_object.cpp

@@ -0,0 +1,216 @@
+// Copyright 2017 The Fuchsia Authors
+//
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file or at
+// https://opensource.org/licenses/MIT
+
+
+#include <magenta/pinned_memory_object.h>
+
+#include <assert.h>
+#include <err.h>
+#include <kernel/vm.h>
+#include <kernel/vm/vm_object.h>
+#include <magenta/bus_transaction_initiator_dispatcher.h>
+#include <mxcpp/new.h>
+#include <mxtl/algorithm.h>
+#include <mxtl/auto_call.h>
+#include <mxtl/auto_lock.h>
+#include <trace.h>
+
+#define LOCAL_TRACE 0
+
+namespace {
+
+struct IommuMapPageContext {
+    mxtl::RefPtr<Iommu> iommu;
+    uint64_t bus_txn_id;
+    PinnedMemoryObject::Extent* page_array;
+    size_t num_entries;
+    uint32_t perms;
+};
+
+// Callback for VmObject::Lookup that handles mapping individual pages into the IOMMU.
+status_t IommuMapPage(void* context, size_t offset, size_t index, paddr_t pa) {
+    IommuMapPageContext* ctx = static_cast<IommuMapPageContext*>(context);
+
+    dev_vaddr_t vaddr;
+    status_t status = ctx->iommu->Map(ctx->bus_txn_id, pa, PAGE_SIZE, ctx->perms, &vaddr);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    DEBUG_ASSERT(IS_PAGE_ALIGNED(vaddr));
+    // TODO(teisenbe): Re-enable this to get run-encoding.
+    /*
+    if (ctx->num_entries == 0) {
+        ctx->page_array[0] = PinnedMemoryObject::Extent(vaddr, 1);
+        ctx->num_entries++;
+        return MX_OK;
+    }
+
+    PinnedMemoryObject::Extent* prev_extent = &ctx->page_array[ctx->num_entries - 1];
+    if (prev_extent->base() + prev_extent->pages() * PAGE_SIZE == vaddr &&
+        prev_extent->extend(1) == MX_OK) {
+
+        return MX_OK;
+    }
+    ctx->page_array[ctx->num_entries] = PinnedMemoryObject::Extent(vaddr, 1);
+    ctx->num_entries++;
+    */
+    ctx->page_array[ctx->num_entries] = PinnedMemoryObject::Extent(vaddr, 1);
+    ctx->num_entries++;
+    return MX_OK;
+}
+
+} // namespace {}
+
+status_t PinnedMemoryObject::Create(const BusTransactionInitiatorDispatcher& bti,
+                                    mxtl::RefPtr<VmObject> vmo, size_t offset,
+                                    size_t size, uint32_t perms,
+                                    mxtl::unique_ptr<PinnedMemoryObject>* out) {
+    LTRACE_ENTRY;
+    DEBUG_ASSERT(IS_PAGE_ALIGNED(offset));
+
+    // Pin the memory to make sure it doesn't change from underneath us for the
+    // lifetime of the created PMO.
+    status_t status = vmo->Pin(offset, size);
+    if (status != MX_OK) {
+        LTRACEF("vmo->Pin failed: %d\n", status);
+        return status;
+    }
+
+    uint64_t expected_addr = 0;
+    auto check_contiguous = [](void* context, size_t offset, size_t index, paddr_t pa) {
+        auto expected_addr = static_cast<uint64_t*>(context);
+        if (index != 0 && pa != *expected_addr) {
+            return MX_ERR_NOT_FOUND;
+        }
+        *expected_addr = pa + PAGE_SIZE;
+        return MX_OK;
+    };
+    status = vmo->Lookup(offset, size, 0, check_contiguous, &expected_addr);
+    bool is_contiguous = (status == MX_OK);
+
+    // Set up a cleanup function to undo the pin if we need to fail this
+    // operation.
+    auto unpin_vmo = mxtl::MakeAutoCall([vmo, offset, size]() {
+        vmo->Unpin(offset, size);
+    });
+
+    // TODO(teisenbe): Be more intelligent about allocating this, since if this
+    // is backed by a real IOMMU, we will likely compress the page array greatly
+    // using extents.
+    mxtl::AllocChecker ac;
+    const size_t num_pages = is_contiguous ? 1 : ROUNDUP(size, PAGE_SIZE) / PAGE_SIZE;
+    mxtl::unique_ptr<Extent[]> page_array(new (&ac) Extent[num_pages]);
+    if (!ac.check()) {
+        return MX_ERR_NO_MEMORY;
+    }
+
+    mxtl::unique_ptr<PinnedMemoryObject> pmo(
+            new (&ac) PinnedMemoryObject(bti, mxtl::move(vmo), offset, size, is_contiguous,
+                                         mxtl::move(page_array)));
+    if (!ac.check()) {
+        return MX_ERR_NO_MEMORY;
+    }
+
+    // Now that the pmo object has been created, it is responsible for
+    // unpinning.
+    unpin_vmo.cancel();
+
+    status = pmo->MapIntoIommu(perms);
+    if (status != MX_OK) {
+        LTRACEF("MapIntoIommu failed: %d\n", status);
+        return status;
+    }
+
+    *out = mxtl::move(pmo);
+    return MX_OK;
+}
+
+// Used during initialization to set up the IOMMU state for this PMO.
+status_t PinnedMemoryObject::MapIntoIommu(uint32_t perms) {
+    if (is_contiguous_) {
+        paddr_t paddr = 1;
+        auto get_paddr = [](void* context, size_t offset, size_t index, paddr_t pa) {
+            *static_cast<paddr_t*>(context) = pa;
+            return MX_OK;
+        };
+        const size_t lookup_size = mxtl::min<size_t>(size_, PAGE_SIZE);
+        status_t status = vmo_->Lookup(offset_, lookup_size, 0, get_paddr, &paddr);
+        if (status != MX_OK) {
+            return status;
+        }
+        ASSERT(paddr != 1);
+
+        dev_vaddr_t vaddr;
+        status = bti_.iommu()->Map(bti_.bti_id(), paddr, ROUNDUP(size_, PAGE_SIZE), perms, &vaddr);
+        if (status != MX_OK) {
+            return status;
+        }
+        mapped_extents_[0] = PinnedMemoryObject::Extent(vaddr, 1);
+        mapped_extents_len_ = 1;
+        return MX_OK;
+    }
+
+    IommuMapPageContext context = {
+        .iommu = bti_.iommu(),
+        .bus_txn_id = bti_.bti_id(),
+        .page_array = mapped_extents_.get(),
+        .num_entries = 0,
+        .perms = perms,
+    };
+    status_t status = vmo_->Lookup(offset_, size_, 0, IommuMapPage, static_cast<void*>(&context));
+    mapped_extents_len_ = context.num_entries;
+    if (status != MX_OK) {
+        status_t err = UnmapFromIommu();
+        ASSERT(err == MX_OK);
+        return status;
+    }
+
+    return MX_OK;
+}
+
+status_t PinnedMemoryObject::UnmapFromIommu() {
+    auto iommu = bti_.iommu();
+    const uint64_t bus_txn_id = bti_.bti_id();
+
+    if (unlikely(mapped_extents_len_ == 0)) {
+        return MX_OK;
+    }
+
+    status_t status = MX_OK;
+    if (is_contiguous_) {
+        status = iommu->Unmap(bus_txn_id, mapped_extents_[0].base(), ROUNDUP(size_, PAGE_SIZE));
+    } else {
+        for (size_t i = 0; i < mapped_extents_len_; ++i) {
+            // Try to unmap all pages even if we get an error, and return the
+            // first error encountered.
+            status_t err = iommu->Unmap(bus_txn_id, mapped_extents_[i].base(),
+                                        mapped_extents_[i].pages() * PAGE_SIZE);
+            DEBUG_ASSERT(err == MX_OK);
+            if (err != MX_OK && status == MX_OK) {
+                status = err;
+            }
+        }
+    }
+    // Clear this so we won't try again if this gets called again in the
+    // destructor.
+    mapped_extents_len_ = 0;
+    return status;
+}
+
+PinnedMemoryObject::~PinnedMemoryObject() {
+    status_t status = UnmapFromIommu();
+    ASSERT(status == MX_OK);
+    vmo_->Unpin(offset_, size_);
+}
+
+PinnedMemoryObject::PinnedMemoryObject(const BusTransactionInitiatorDispatcher& bti,
+                                       mxtl::RefPtr<VmObject> vmo, size_t offset, size_t size,
+                                       bool is_contiguous,
+                                       mxtl::unique_ptr<Extent[]> mapped_extents)
+    : vmo_(mxtl::move(vmo)), offset_(offset), size_(size), is_contiguous_(is_contiguous), bti_(bti),
+      mapped_extents_(mxtl::move(mapped_extents)), mapped_extents_len_(0) {
+}

kernel/lib/magenta/rules.mk

@@ -10,6 +10,7 @@ LOCAL_DIR := $(GET_LOCAL_DIR)
 MODULE := $(LOCAL_DIR)
 
 MODULE_SRCS := \
+	$(LOCAL_DIR)/bus_transaction_initiator_dispatcher.cpp \
     $(LOCAL_DIR)/channel_dispatcher.cpp \
     $(LOCAL_DIR)/diagnostics.cpp \
     $(LOCAL_DIR)/dispatcher.cpp \
@@ -24,6 +25,7 @@ MODULE_SRCS := \
     $(LOCAL_DIR)/handle.cpp \
     $(LOCAL_DIR)/handle_reaper.cpp \
     $(LOCAL_DIR)/interrupt_event_dispatcher.cpp \
+    $(LOCAL_DIR)/iommu_dispatcher.cpp \
     $(LOCAL_DIR)/job_dispatcher.cpp \
     $(LOCAL_DIR)/log_dispatcher.cpp \
     $(LOCAL_DIR)/magenta.cpp \
@@ -31,6 +33,7 @@ MODULE_SRCS := \
     $(LOCAL_DIR)/message_packet.cpp \
     $(LOCAL_DIR)/pci_device_dispatcher.cpp \
     $(LOCAL_DIR)/pci_interrupt_dispatcher.cpp \
+    $(LOCAL_DIR)/pinned_memory_object.cpp \
     $(LOCAL_DIR)/policy_manager.cpp \
     $(LOCAL_DIR)/port_dispatcher.cpp \
     $(LOCAL_DIR)/process_dispatcher.cpp \

kernel/lib/syscalls/syscalls_ddk.cpp

@@ -13,6 +13,7 @@
 #include <trace.h>
 
 #include <dev/interrupt.h>
+#include <dev/iommu.h>
 #include <dev/udisplay.h>
 #include <kernel/vm.h>
 #include <kernel/vm/vm_object_paged.h>
@@ -24,14 +25,20 @@
 #include <platform/pc/bootloader.h>
 #endif
 
+#include <magenta/bus_transaction_initiator_dispatcher.h>
 #include <magenta/handle_owner.h>
 #include <magenta/interrupt_dispatcher.h>
 #include <magenta/interrupt_event_dispatcher.h>
+#include <magenta/iommu_dispatcher.h>
 #include <magenta/magenta.h>
 #include <magenta/process_dispatcher.h>
+#include <magenta/syscalls/iommu.h>
 #include <magenta/syscalls/pci.h>
 #include <magenta/user_copy.h>
 #include <magenta/vm_object_dispatcher.h>
+#include <mxcpp/new.h>
+#include <mxtl/auto_call.h>
+#include <mxtl/inline_array.h>
 
 #include "syscalls_priv.h"
 
@@ -276,6 +283,71 @@ mx_status_t sys_set_framebuffer_vmo(mx_handle_t hrsrc, mx_handle_t vmo_handle, u
     return MX_OK;
 }
 
+// TODO: Write docs for this syscall
+mx_status_t sys_iommu_create(mx_handle_t rsrc_handle, uint32_t type, user_ptr<const void> desc,
+                             uint32_t desc_len, user_ptr<mx_handle_t> out) {
+    // TODO: finer grained validation
+    mx_status_t status;
+    if ((status = validate_resource(rsrc_handle, MX_RSRC_KIND_ROOT)) < 0) {
+        return status;
+    }
+
+    TRACEF("IOMMU Create\n");
+    static mxtl::RefPtr<Dispatcher> main_iommu = nullptr;
+    static mxtl::Mutex m;
+    static mx_rights_t main_iommu_rights;
+    mxtl::AutoLock guard(&m);
+    if (type == MX_IOMMU_TYPE_DUMMY && main_iommu) {
+        TRACEF("Using stashed IOMMU\n");
+        HandleOwner handle(MakeHandle(main_iommu, main_iommu_rights));
+
+        auto up = ProcessDispatcher::GetCurrent();
+        if (out.copy_to_user(up->MapHandleToValue(handle)) != MX_OK)
+            return MX_ERR_INVALID_ARGS;
+
+        up->AddHandle(mxtl::move(handle));
+        return MX_OK;
+    }
+
+    if (desc_len > MX_IOMMU_MAX_DESC_LEN) {
+        return MX_ERR_INVALID_ARGS;
+    }
+
+    mxtl::RefPtr<Dispatcher> dispatcher;
+    mx_rights_t rights;
+
+    {
+        // Copy the descriptor into the kernel and try to create the dispatcher
+        // using it.
+        mxtl::AllocChecker ac;
+        mxtl::unique_ptr<uint8_t[]> copied_desc(new (&ac) uint8_t[desc_len]);
+        if (!ac.check()) {
+            return MX_ERR_NO_MEMORY;
+        }
+        if ((status = desc.copy_array_from_user(copied_desc.get(), desc_len)) != MX_OK) {
+            return status;
+        }
+        status = IommuDispatcher::Create(type,
+                                         mxtl::unique_ptr<const uint8_t[]>(copied_desc.release()),
+                                         desc_len, &dispatcher, &rights);
+        if (status != MX_OK) {
+            return status;
+        }
+    }
+
+    main_iommu = dispatcher;
+    main_iommu_rights = rights;
+
+    HandleOwner handle(MakeHandle(mxtl::move(dispatcher), rights));
+
+    auto up = ProcessDispatcher::GetCurrent();
+    if (out.copy_to_user(up->MapHandleToValue(handle)) != MX_OK)
+        return MX_ERR_INVALID_ARGS;
+
+    up->AddHandle(mxtl::move(handle));
+    return MX_OK;
+}
+
 #if ARCH_X86
 #include <arch/x86/descriptor.h>
 #include <arch/x86/ioport.h>
@@ -327,3 +399,137 @@ mx_status_t sys_acpi_cache_flush(mx_handle_t hrsrc) {
     return MX_ERR_NOT_SUPPORTED;
 #endif
 }
+
+mx_status_t sys_bti_create(mx_handle_t iommu, uint64_t bti_id, user_ptr<mx_handle_t> out) {
+    auto up = ProcessDispatcher::GetCurrent();
+
+    mxtl::RefPtr<IommuDispatcher> iommu_dispatcher;
+    // TODO(teisenbe): This should probably have a right on it.
+    mx_status_t status = up->GetDispatcherWithRights(iommu, MX_RIGHT_NONE, &iommu_dispatcher);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    mxtl::RefPtr<Dispatcher> dispatcher;
+    mx_rights_t rights;
+    // TODO(teisenbe): Migrate BusTransactionInitiatorDispatcher::Create to
+    // taking the iommu_dispatcher
+    status = BusTransactionInitiatorDispatcher::Create(iommu_dispatcher->iommu(), bti_id,
+                                                       &dispatcher, &rights);
+    if (status != MX_OK) {
+        return status;
+    }
+    HandleOwner handle(MakeHandle(mxtl::move(dispatcher), rights));
+
+    mx_handle_t hv = up->MapHandleToValue(handle);
+    if ((status = out.copy_to_user(hv)) != MX_OK) {
+        return status;
+    }
+
+    up->AddHandle(mxtl::move(handle));
+    return MX_OK;
+}
+
+mx_status_t sys_bti_pin(mx_handle_t bti, mx_handle_t vmo, uint64_t offset, uint64_t size,
+        uint32_t perms, user_ptr<uint64_t> extents, uint32_t extents_len,
+        user_ptr<uint32_t> actual_extents_len) {
+    auto up = ProcessDispatcher::GetCurrent();
+
+    if (!IS_PAGE_ALIGNED(offset)) {
+        return MX_ERR_INVALID_ARGS;
+    }
+
+    mxtl::RefPtr<BusTransactionInitiatorDispatcher> bti_dispatcher;
+    mx_status_t status = up->GetDispatcherWithRights(bti, MX_RIGHT_MAP, &bti_dispatcher);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    mxtl::RefPtr<VmObjectDispatcher> vmo_dispatcher;
+    mx_rights_t vmo_rights;
+    status = up->GetDispatcherAndRights(vmo, &vmo_dispatcher, &vmo_rights);
+    if (status != MX_OK) {
+        return status;
+    }
+    if (!(vmo_rights & MX_RIGHT_MAP)) {
+        return MX_ERR_ACCESS_DENIED;
+    }
+
+    // Convert requested permissions and check against VMO rights
+    uint32_t iommu_perms = 0;
+    if (perms & MX_VM_FLAG_PERM_READ) {
+        if (!(vmo_rights & MX_RIGHT_READ)) {
+            return MX_ERR_ACCESS_DENIED;
+        }
+        iommu_perms |= IOMMU_FLAG_PERM_READ;
+        perms &= ~MX_VM_FLAG_PERM_READ;
+    }
+    if (perms & MX_VM_FLAG_PERM_WRITE) {
+        if (!(vmo_rights & MX_RIGHT_WRITE)) {
+            return MX_ERR_ACCESS_DENIED;
+        }
+        iommu_perms |= IOMMU_FLAG_PERM_WRITE;
+        perms &= ~MX_VM_FLAG_PERM_WRITE;
+    }
+    if (perms & MX_VM_FLAG_PERM_EXECUTE) {
+        if (!(vmo_rights & MX_RIGHT_EXECUTE)) {
+            return MX_ERR_ACCESS_DENIED;
+        }
+        iommu_perms |= IOMMU_FLAG_PERM_EXECUTE;
+        perms &= ~MX_VM_FLAG_PERM_EXECUTE;
+    }
+    if (perms) {
+        return MX_ERR_INVALID_ARGS;
+    }
+
+    mxtl::AllocChecker ac;
+    mxtl::InlineArray<dev_vaddr_t, 4u> mapped_extents(&ac, extents_len);
+    if (!ac.check()) {
+        return MX_ERR_NO_MEMORY;
+    }
+
+    size_t actual_len;
+    status = bti_dispatcher->Pin(vmo_dispatcher->vmo(), offset, size, iommu_perms,
+                                 mapped_extents.get(), extents_len, &actual_len);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    auto pin_cleanup = mxtl::MakeAutoCall([&bti_dispatcher, &mapped_extents, actual_len]() {
+        bti_dispatcher->Unpin(mapped_extents.get(), actual_len);
+    });
+
+    static_assert(sizeof(dev_vaddr_t) == sizeof(uint64_t), "mismatched types");
+    if ((status = extents.copy_array_to_user(mapped_extents.get(), actual_len)) != MX_OK) {
+        return status;
+    }
+    if ((status = actual_extents_len.copy_to_user(static_cast<uint32_t>(actual_len))) != MX_OK) {
+        return status;
+    }
+
+    pin_cleanup.cancel();
+    return MX_OK;
+}
+
+mx_status_t sys_bti_unpin(mx_handle_t bti, user_ptr<const uint64_t> extents, uint32_t extents_len) {
+    auto up = ProcessDispatcher::GetCurrent();
+
+    mxtl::RefPtr<BusTransactionInitiatorDispatcher> bti_dispatcher;
+    mx_status_t status = up->GetDispatcherWithRights(bti, MX_RIGHT_MAP, &bti_dispatcher);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    mxtl::AllocChecker ac;
+    mxtl::InlineArray<dev_vaddr_t, 4u> mapped_extents(&ac, extents_len);
+    if (!ac.check()) {
+        return MX_ERR_NO_MEMORY;
+    }
+    static_assert(sizeof(dev_vaddr_t) == sizeof(uint64_t), "mismatched types");
+    if ((status = extents.copy_array_from_user(mapped_extents.get(), extents_len)) != MX_OK) {
+        return status;
+    }
+
+    return bti_dispatcher->Unpin(mapped_extents.get(), extents_len);
+
+}

system/dev/block/ahci/ahci.c

@@ -710,13 +710,20 @@ static mx_status_t ahci_bind(void* ctx, mx_device_t* dev, void** cookie) {
         return MX_ERR_NOT_SUPPORTED;
     }
 
+    mx_handle_t bti;
+    mx_status_t status = device->pci.ops->get_bti(device->pci.ctx, &bti);
+    if (status != MX_OK) {
+        goto fail;
+    }
+    iotxn_set_default_bti(bti);
+
     // map register window
-    mx_status_t status = pci_map_resource(&device->pci,
-                                          PCI_RESOURCE_BAR_5,
-                                          MX_CACHE_POLICY_UNCACHED_DEVICE,
-                                          (void**)&device->regs,
-                                          &device->regs_size,
-                                          &device->regs_handle);
+    status = pci_map_resource(&device->pci,
+                              PCI_RESOURCE_BAR_5,
+                              MX_CACHE_POLICY_UNCACHED_DEVICE,
+                              (void**)&device->regs,
+                              &device->regs_size,
+                              &device->regs_handle);
     if (status != MX_OK) {
         xprintf("ahci: error %d mapping register window\n", status);
         goto fail;

system/dev/bus/acpi/init.c

@@ -2,12 +2,17 @@
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
+#include <assert.h>
+#include <stdbool.h>
+#include <magenta/syscalls/iommu.h>
 #include <stdio.h>
 
 #include "init.h"
 
 #define ACPI_MAX_INIT_TABLES 32
 
+static mx_status_t find_iommus(void);
+
 /* @brief Switch interrupts to APIC model (controls IRQ routing) */
 static ACPI_STATUS set_apic_irq_mode(void) {
     ACPI_OBJECT selector = {
@@ -60,6 +65,11 @@ ACPI_STATUS init(void) {
         return status;
     }
 
+    mx_status_t mx_status = find_iommus();
+    if (mx_status != MX_OK) {
+        printf("Failed to publish iommus\n");
+    }
+
     status = set_apic_irq_mode();
     if (status == AE_NOT_FOUND) {
         printf("WARNING: Could not find ACPI IRQ mode switch\n");
@@ -74,3 +84,466 @@ ACPI_STATUS init(void) {
     return AE_OK;
 }
 
+static mx_status_t acpi_scope_to_desc(ACPI_DMAR_DEVICE_SCOPE* acpi_scope,
+                                      mx_iommu_desc_intel_scope_t* desc_scope) {
+    switch (acpi_scope->EntryType) {
+        case ACPI_DMAR_SCOPE_TYPE_ENDPOINT:
+            desc_scope->type = MX_IOMMU_INTEL_SCOPE_ENDPOINT;
+            break;
+        case ACPI_DMAR_SCOPE_TYPE_BRIDGE:
+            desc_scope->type = MX_IOMMU_INTEL_SCOPE_BRIDGE;
+            break;
+        default:
+            // Skip this scope, since it's not a type we care about.
+            return MX_ERR_WRONG_TYPE;
+    }
+    desc_scope->start_bus = acpi_scope->Bus;
+    // TODO(teisenbe): Check for overflow
+    desc_scope->num_hops = (acpi_scope->Length - 6) / 2;
+    if (countof(desc_scope->dev_func) < desc_scope->num_hops) {
+        return MX_ERR_NOT_SUPPORTED;
+    }
+    for (ssize_t i = 0; i < desc_scope->num_hops; ++i) {
+        uint16_t v = *(uint16_t*)((uintptr_t)acpi_scope + 6 + 2 * i);
+        const uint8_t dev = v & 0x1f;
+        const uint8_t func = (v >> 8) & 0x7;
+        desc_scope->dev_func[i] = (dev << 3) | func;
+    }
+    return MX_OK;
+}
+
+// Walks the given unit's scopes and appends them to the given descriptor.
+// |max_scopes| is the number of scopes |scopes| can hold.  |num_scopes_found|
+// is the number of scopes found on |unit|, even if they wouldn't all fit in |scopes|.
+static mx_status_t append_scopes(ACPI_DMAR_HARDWARE_UNIT* unit,
+                                 size_t max_scopes,
+                                 mx_iommu_desc_intel_scope_t* scopes,
+                                 size_t* num_scopes_found) {
+    size_t num_scopes = 0;
+    uintptr_t scope;
+    const uintptr_t addr = (uintptr_t)unit;
+    for (scope = addr + 16; scope < addr + unit->Header.Length; ) {
+        ACPI_DMAR_DEVICE_SCOPE* s = (ACPI_DMAR_DEVICE_SCOPE*)scope;
+        printf("  DMAR Scope: %u, bus %u\n", s->EntryType, s->Bus);
+        for (ssize_t i = 0; i < (s->Length - 6) / 2; ++i) {
+            uint16_t v = *(uint16_t*)(scope + 6 + 2 * i);
+            printf("    Path %ld: %02x.%02x\n", i, v & 0xffu, (uint16_t)(v >> 8));
+        }
+        scope += s->Length;
+
+        // Count the scopes we care about
+        switch (s->EntryType) {
+            case ACPI_DMAR_SCOPE_TYPE_ENDPOINT:
+            case ACPI_DMAR_SCOPE_TYPE_BRIDGE:
+                num_scopes++;
+                break;
+        }
+    }
+
+    if (num_scopes_found) {
+        *num_scopes_found = num_scopes;
+    }
+    if (!scopes) {
+        return MX_OK;
+    }
+
+    if (num_scopes > max_scopes) {
+        return MX_ERR_BUFFER_TOO_SMALL;
+    }
+
+    size_t cur_num_scopes = 0;
+    for (scope = addr + 16; scope < addr + unit->Header.Length && cur_num_scopes < max_scopes;) {
+        ACPI_DMAR_DEVICE_SCOPE* s = (ACPI_DMAR_DEVICE_SCOPE*)scope;
+
+        mx_status_t status = acpi_scope_to_desc(s, &scopes[cur_num_scopes]);
+        if (status != MX_OK && status != MX_ERR_WRONG_TYPE) {
+            return status;
+        }
+        if (status == MX_OK) {
+            cur_num_scopes++;
+        }
+
+        scope += s->Length;
+    }
+    assert(cur_num_scopes == num_scopes);
+    return MX_OK;
+}
+
+static bool scope_eq(mx_iommu_desc_intel_scope_t* scope,
+                     ACPI_DMAR_DEVICE_SCOPE* acpi_scope) {
+
+    mx_iommu_desc_intel_scope_t other_scope;
+    mx_status_t status = acpi_scope_to_desc(acpi_scope, &other_scope);
+    if (status != MX_OK) {
+        return false;
+    }
+
+    if (scope->type != other_scope.type || scope->start_bus != other_scope.start_bus ||
+        scope->num_hops != other_scope.num_hops) {
+
+        return false;
+    }
+
+    for (size_t i = 0; i < scope->num_hops; ++i) {
+        if (scope->dev_func[i] != other_scope.dev_func[i]) {
+            return false;
+        }
+    }
+
+    return true;
+}
+
+// Appends to desc any reserved memory regions that match its scopes.  If
+// |desc_len| is not large enough to include the reserved memory descriptors, this
+// function will not append all of the found entries.  |bytes_needed| will
+// always return the number of bytes needed to represent all of the reserved
+// descriptors.  This function does not modify desc->reserved_mem_bytes.
+static mx_status_t append_reserved_mem(ACPI_TABLE_DMAR* table,
+                                       mx_iommu_desc_intel_t* desc,
+                                       size_t desc_len,
+                                       size_t* bytes_needed) {
+
+    const uintptr_t records_start = (uintptr_t)table + sizeof(*table);
+    const uintptr_t records_end = (uintptr_t)table + table->Header.Length;
+
+    mx_iommu_desc_intel_scope_t* desc_scopes = (mx_iommu_desc_intel_scope_t*)(
+            (uintptr_t)desc + sizeof(*desc));
+    const size_t num_desc_scopes = desc->scope_bytes / sizeof(mx_iommu_desc_intel_scope_t);
+
+    uintptr_t next_reserved_mem_desc_base = (uintptr_t)desc + sizeof(mx_iommu_desc_intel_t) +
+            desc->scope_bytes + desc->reserved_memory_bytes;
+
+    *bytes_needed = 0;
+    for (uintptr_t addr = records_start; addr < records_end;) {
+        ACPI_DMAR_HEADER* record_hdr = (ACPI_DMAR_HEADER*)addr;
+        switch (record_hdr->Type) {
+            case ACPI_DMAR_TYPE_RESERVED_MEMORY: {
+                ACPI_DMAR_RESERVED_MEMORY* rec = (ACPI_DMAR_RESERVED_MEMORY*)record_hdr;
+
+                if (desc->pci_segment != rec->Segment) {
+                    break;
+                }
+
+                mx_iommu_desc_intel_reserved_memory_t* mem_desc =
+                        (mx_iommu_desc_intel_reserved_memory_t*)next_reserved_mem_desc_base;
+                size_t mem_desc_size = sizeof(*mem_desc);
+
+                // Search for scopes that match
+                for (uintptr_t scope = addr + 24; scope < addr + rec->Header.Length; ) {
+                    ACPI_DMAR_DEVICE_SCOPE* s = (ACPI_DMAR_DEVICE_SCOPE*)scope;
+                    // TODO(teisenbe): We should skip scope types we don't
+                    // care about here
+
+                    // Search for a scope in the descriptor that matches this
+                    // ACPI scope.
+                    bool no_matches = true;
+                    for (size_t i = 0; i < num_desc_scopes; ++i) {
+                        mx_iommu_desc_intel_scope_t* scope_desc = &desc_scopes[i];
+                        const bool scope_matches = scope_eq(scope_desc, s);
+
+                        no_matches &= !scope_matches;
+
+                        // If this is a whole segment descriptor, then a match
+                        // corresponds to an entry we should ignore.
+                        if (scope_matches && !desc->whole_segment) {
+                            mx_iommu_desc_intel_scope_t* new_scope_desc =
+                                    (mx_iommu_desc_intel_scope_t*)(next_reserved_mem_desc_base +
+                                                                   mem_desc_size);
+                            mem_desc_size += sizeof(mx_iommu_desc_intel_scope_t);
+
+                            if (next_reserved_mem_desc_base + mem_desc_size <=
+                                (uintptr_t)desc + desc_len) {
+
+                                memcpy(new_scope_desc, scope_desc, sizeof(*scope_desc));
+                            }
+                            break;
+                        }
+                    }
+
+                    if (no_matches && desc->whole_segment) {
+                        mx_iommu_desc_intel_scope_t other_scope;
+                        mx_status_t status = acpi_scope_to_desc(s, &other_scope);
+                        if (status != MX_ERR_WRONG_TYPE && status != MX_OK) {
+                            return status;
+                        }
+                        if (status == MX_OK) {
+                            mx_iommu_desc_intel_scope_t* new_scope_desc =
+                                    (mx_iommu_desc_intel_scope_t*)(next_reserved_mem_desc_base +
+                                                                   mem_desc_size);
+                            mem_desc_size += sizeof(mx_iommu_desc_intel_scope_t);
+
+                            if (next_reserved_mem_desc_base + mem_desc_size <=
+                                (uintptr_t)desc + desc_len) {
+
+                                memcpy(new_scope_desc, &other_scope, sizeof(other_scope));
+                            }
+                        }
+                    }
+
+                    scope += s->Length;
+                }
+
+                // If this descriptor does not have any scopes, ignore it
+                if (mem_desc_size == sizeof(*mem_desc)) {
+                    break;
+                }
+
+                if (next_reserved_mem_desc_base + mem_desc_size <= (uintptr_t)desc + desc_len) {
+                    mem_desc->base_addr = rec->BaseAddress;
+                    mem_desc->len = rec->EndAddress - rec->BaseAddress + 1;
+                    if (mem_desc->base_addr == 0x8ab7a000 && mem_desc->len == 0x20000) {
+                        // TODO(teisenbe): Make this significantly less hacky
+                        printf("applying acer12 quirk\n");
+                        mem_desc->len += 0x68000;
+                    }
+
+                    mem_desc->scope_bytes = mem_desc_size - sizeof(*mem_desc);
+                    next_reserved_mem_desc_base += mem_desc_size;
+                }
+                *bytes_needed += mem_desc_size;
+
+                break;
+            }
+        }
+
+        addr += record_hdr->Length;
+    }
+    if (*bytes_needed + sizeof(mx_iommu_desc_intel_t) +
+        desc->scope_bytes + desc->reserved_memory_bytes > desc_len) {
+        return MX_ERR_BUFFER_TOO_SMALL;
+    }
+
+    return MX_OK;
+}
+
+static mx_status_t create_whole_segment_iommu_desc(ACPI_TABLE_DMAR* table,
+                                                   ACPI_DMAR_HARDWARE_UNIT* unit,
+                                                   mx_iommu_desc_intel_t** desc_out,
+                                                   size_t* desc_len_out) {
+    assert(unit->Flags & ACPI_DMAR_INCLUDE_ALL);
+
+    // The VT-d spec requires that whole-segment hardware units appear in the
+    // DMAR table after all other hardware units on their segment.  Search those
+    // entries for scopes to specify as excluded from this descriptor.
+
+    size_t num_scopes = 0;
+    size_t num_scopes_on_unit;
+
+    const uintptr_t records_start = ((uintptr_t)table) + sizeof(*table);
+    const uintptr_t records_end = (uintptr_t)unit + unit->Header.Length;
+
+    // TODO: Check scopes on self
+
+    uintptr_t addr;
+    for (addr = records_start; addr < records_end;) {
+        ACPI_DMAR_HEADER* record_hdr = (ACPI_DMAR_HEADER*)addr;
+        switch (record_hdr->Type) {
+            case ACPI_DMAR_TYPE_HARDWARE_UNIT: {
+                ACPI_DMAR_HARDWARE_UNIT* rec = (ACPI_DMAR_HARDWARE_UNIT*)record_hdr;
+                if (rec->Segment != unit->Segment) {
+                    break;
+                }
+                mx_status_t status = append_scopes(rec, 0, NULL, &num_scopes_on_unit);
+                if (status != MX_OK) {
+                    return status;
+                }
+                num_scopes += num_scopes_on_unit;
+            }
+        }
+        addr += record_hdr->Length;
+    }
+
+    size_t desc_len = sizeof(mx_iommu_desc_intel_t) +
+            sizeof(mx_iommu_desc_intel_scope_t) * num_scopes;
+    mx_iommu_desc_intel_t* desc = malloc(desc_len);
+    if (!desc) {
+        return MX_ERR_NO_MEMORY;
+    }
+    desc->register_base = unit->Address;
+    desc->pci_segment = unit->Segment;
+    desc->whole_segment = true;
+    desc->scope_bytes = 0;
+    desc->reserved_memory_bytes = 0;
+
+    for (addr = records_start; addr < records_end;) {
+        ACPI_DMAR_HEADER* record_hdr = (ACPI_DMAR_HEADER*)addr;
+        switch (record_hdr->Type) {
+            case ACPI_DMAR_TYPE_HARDWARE_UNIT: {
+                ACPI_DMAR_HARDWARE_UNIT* rec = (ACPI_DMAR_HARDWARE_UNIT*)record_hdr;
+                if (rec->Segment != unit->Segment) {
+                    break;
+                }
+                size_t scopes_found = 0;
+                mx_iommu_desc_intel_scope_t* scopes = (mx_iommu_desc_intel_scope_t*)(
+                        (uintptr_t)desc + sizeof(*desc) + desc->scope_bytes);
+                mx_status_t status = append_scopes(rec, num_scopes, scopes, &scopes_found);
+                if (status != MX_OK) {
+                    free(desc);
+                    return status;
+                }
+                desc->scope_bytes += scopes_found * sizeof(mx_iommu_desc_intel_scope_t);
+                num_scopes -= scopes_found;
+            }
+        }
+        addr += record_hdr->Length;
+    }
+
+    size_t reserved_mem_bytes = 0;
+    mx_status_t status = append_reserved_mem(table, desc, desc_len, &reserved_mem_bytes);
+    if (status == MX_ERR_BUFFER_TOO_SMALL) {
+        mx_iommu_desc_intel_t* new_desc = realloc(desc, desc_len + reserved_mem_bytes);
+        if (new_desc == NULL) {
+            free(desc);
+            return MX_ERR_NO_MEMORY;
+        }
+        desc = new_desc;
+        desc_len += reserved_mem_bytes;
+        status = append_reserved_mem(table, desc, desc_len, &reserved_mem_bytes);
+    }
+    if (status != MX_OK) {
+        free(desc);
+        return status;
+    }
+    desc->reserved_memory_bytes += reserved_mem_bytes;
+
+    *desc_out = desc;
+    *desc_len_out = desc_len;
+    return MX_OK;
+}
+
+static mx_status_t create_non_whole_segment_iommu_desc(ACPI_TABLE_DMAR* table,
+                                                       ACPI_DMAR_HARDWARE_UNIT* unit,
+                                                       mx_iommu_desc_intel_t** desc_out,
+                                                       size_t* desc_len_out) {
+    assert((unit->Flags & ACPI_DMAR_INCLUDE_ALL) == 0);
+
+    size_t num_scopes;
+    mx_status_t status = append_scopes(unit, 0, NULL, &num_scopes);
+    if (status != MX_OK) {
+        return status;
+    }
+
+    size_t desc_len = sizeof(mx_iommu_desc_intel_t) +
+            sizeof(mx_iommu_desc_intel_scope_t) * num_scopes;
+    mx_iommu_desc_intel_t* desc = malloc(desc_len);
+    if (!desc) {
+        return MX_ERR_NO_MEMORY;
+    }
+    desc->register_base = unit->Address;
+    desc->pci_segment = unit->Segment;
+    desc->whole_segment = false;
+    desc->scope_bytes = 0;
+    desc->reserved_memory_bytes = 0;
+    mx_iommu_desc_intel_scope_t* scopes = (mx_iommu_desc_intel_scope_t*)(
+            (uintptr_t)desc + sizeof(*desc));
+    size_t actual_num_scopes;
+    status = append_scopes(unit, num_scopes, scopes, &actual_num_scopes);
+    if (status != MX_OK) {
+        free(desc);
+        return status;
+    }
+    desc->scope_bytes = actual_num_scopes * sizeof(mx_iommu_desc_intel_scope_t);
+
+    size_t reserved_mem_bytes = 0;
+    status = append_reserved_mem(table, desc, desc_len, &reserved_mem_bytes);
+    if (status == MX_ERR_BUFFER_TOO_SMALL) {
+        mx_iommu_desc_intel_t* new_desc = realloc(desc, desc_len + reserved_mem_bytes);
+        if (new_desc == NULL) {
+            free(desc);
+            return MX_ERR_NO_MEMORY;
+        }
+        desc = new_desc;
+        desc_len += reserved_mem_bytes;
+        status = append_reserved_mem(table, desc, desc_len, &reserved_mem_bytes);
+    }
+    if (status != MX_OK) {
+        free(desc);
+        return status;
+    }
+    desc->reserved_memory_bytes += reserved_mem_bytes;
+
+    *desc_out = desc;
+    *desc_len_out = desc_len;
+    return MX_OK;
+}
+
+static mx_status_t find_iommus(void) {
+    ACPI_TABLE_HEADER* table = NULL;
+    ACPI_STATUS status = AcpiGetTable((char*)ACPI_SIG_DMAR, 1, &table);
+    if (status != AE_OK) {
+        printf("could not find DMAR\n");
+        return MX_ERR_NOT_FOUND;
+    }
+    ACPI_TABLE_DMAR* dmar = (ACPI_TABLE_DMAR*)table;
+    const uintptr_t records_start = ((uintptr_t)dmar) + sizeof(*dmar);
+    const uintptr_t records_end = ((uintptr_t)dmar) + dmar->Header.Length;
+    if (records_start >= records_end) {
+        printf("DMAR wraps around address space\n");
+        return MX_ERR_IO_DATA_INTEGRITY;
+    }
+    // Shouldn't be too many records
+    if (dmar->Header.Length > 4096) {
+        printf("DMAR suspiciously long: %u\n", dmar->Header.Length);
+        return MX_ERR_IO_DATA_INTEGRITY;
+    }
+
+    uintptr_t addr;
+    mx_handle_t iommu_handle = MX_HANDLE_INVALID;
+    for (addr = records_start; addr < records_end;) {
+        ACPI_DMAR_HEADER* record_hdr = (ACPI_DMAR_HEADER*)addr;
+        printf("DMAR record: %d\n", record_hdr->Type);
+        switch (record_hdr->Type) {
+            case ACPI_DMAR_TYPE_HARDWARE_UNIT: {
+                ACPI_DMAR_HARDWARE_UNIT* rec = (ACPI_DMAR_HARDWARE_UNIT*)record_hdr;
+
+                printf("DMAR Hardware Unit: %u %#llx %#x\n", rec->Segment, rec->Address, rec->Flags);
+                const bool whole_segment = rec->Flags & ACPI_DMAR_INCLUDE_ALL;
+
+                mx_iommu_desc_intel_t* desc = NULL;
+                size_t desc_len = 0;
+                mx_status_t mx_status;
+                if (whole_segment) {
+                    mx_status = create_whole_segment_iommu_desc(dmar, rec, &desc, &desc_len);
+                } else {
+                    mx_status = create_non_whole_segment_iommu_desc(dmar, rec, &desc, &desc_len);
+                }
+                if (mx_status != MX_OK) {
+                    printf("Failed to create iommu desc: %d\n", mx_status);
+                    return mx_status;
+                }
+
+                mx_status = mx_iommu_create(root_resource_handle, MX_IOMMU_TYPE_INTEL,
+                                            desc, desc_len, &iommu_handle);
+                free(desc);
+                if (mx_status != MX_OK) {
+                    printf("Failed to create iommu: %d\n", mx_status);
+                    return mx_status;
+                }
+                // TODO(teisenbe): Do something with these handles
+                //mx_handle_close(iommu_handle);
+                break;
+            }
+            case ACPI_DMAR_TYPE_RESERVED_MEMORY: {
+                ACPI_DMAR_RESERVED_MEMORY* rec = (ACPI_DMAR_RESERVED_MEMORY*)record_hdr;
+                printf("DMAR Reserved Memory: %u %#llx %#llx\n", rec->Segment, rec->BaseAddress, rec->EndAddress);
+                for (uintptr_t scope = addr + 24; scope < addr + rec->Header.Length; ) {
+                    ACPI_DMAR_DEVICE_SCOPE* s = (ACPI_DMAR_DEVICE_SCOPE*)scope;
+                    printf("  DMAR Scope: %u, bus %u\n", s->EntryType, s->Bus);
+                    for (ssize_t i = 0; i < (s->Length - 6) / 2; ++i) {
+                        uint16_t v = *(uint16_t*)(scope + 6 + 2 * i);
+                        printf("    Path %ld: %02x.%02x\n", i, v & 0xffu, (uint16_t)(v >> 8));
+                    }
+                    scope += s->Length;
+                }
+                break;
+            }
+        }
+
+        addr += record_hdr->Length;
+    }
+    if (addr != records_end) {
+        return MX_ERR_IO_DATA_INTEGRITY;
+    }
+
+    return MX_OK;
+}

system/dev/bus/pci/protocol.c

@@ -7,6 +7,7 @@
 #include <assert.h>
 #include <limits.h>
 #include <magenta/process.h>
+#include <magenta/syscalls/iommu.h>
 
 #include "kpci-private.h"
 
@@ -15,6 +16,27 @@ static mx_status_t kpci_enable_bus_master(void* ctx, bool enable) {
     return mx_pci_enable_bus_master(device->handle, enable);
 }
 
+static mx_status_t kpci_get_bti(void* ctx, mx_handle_t* out_handle) {
+    kpci_device_t* device = ctx;
+
+    printf("get_bti: [%04x:%04x] %x:%x.%u\n", device->info.vendor_id, device->info.device_id, device->info.bus_id, device->info.dev_id, device->info.func_id);
+    const uint32_t bdf = ((uint32_t)device->info.bus_id << 8) +
+            ((uint32_t)device->info.dev_id << 3) +
+            device->info.func_id;
+    mx_handle_t iommu_handle;
+    mx_iommu_desc_dummy_t dummy;
+    mx_status_t status = mx_iommu_create(get_root_resource(), MX_IOMMU_TYPE_DUMMY, &dummy,
+                                         1, &iommu_handle);
+    if (status != MX_OK) {
+        printf("iommu created failed\n");
+        return status;
+    }
+    status = mx_bti_create(iommu_handle, bdf, out_handle);
+    printf("bti create(%x): %d\n", bdf, status);
+    mx_handle_close(iommu_handle);
+    return status;
+}
+
 static mx_status_t kpci_enable_pio(void* ctx, bool enable) {
     kpci_device_t* device = ctx;
     return mx_pci_enable_pio(device->handle, enable);
@@ -187,6 +209,7 @@ static mx_status_t kpci_get_device_info(void* ctx, mx_pcie_device_info_t* out_in
 static pci_protocol_ops_t _pci_protocol = {
     .enable_bus_master = kpci_enable_bus_master,
     .enable_pio = kpci_enable_pio,
+    .get_bti = kpci_get_bti,
     .reset_device = kpci_reset_device,
     .map_resource = kpci_map_resource,
     .map_interrupt = kpci_map_interrupt,

system/dev/ethernet/intel-ethernet/ethernet.c

@@ -160,6 +160,13 @@ static mx_status_t eth_bind(void* ctx, mx_device_t* dev, void** cookie) {
         goto fail;
     }
 
+    mx_handle_t bti;
+    mx_status_t status = edev->pci.ops->get_bti(edev->pci.ctx, &bti);
+    if (status != MX_OK) {
+        goto fail;
+    }
+    iotxn_set_default_bti(bti);
+
     // Query whether we have MSI or Legacy interrupts.
     uint32_t irq_cnt = 0;
     if ((pci_query_irq_mode_caps(&edev->pci, MX_PCIE_IRQ_MODE_MSI, &irq_cnt) == MX_OK) &&

system/dev/usb/xhci/usb-xhci.c

@@ -233,6 +233,13 @@ static mx_status_t usb_xhci_bind(void* ctx, mx_device_t* dev, void** cookie) {
         goto error_return;
     }
 
+    mx_handle_t bti;
+    status = pci.ops->get_bti(pci.ctx, &bti);
+    if (status != MX_OK) {
+        goto error_return;
+    }
+    iotxn_set_default_bti(bti);
+
     xhci = calloc(1, sizeof(xhci_t));
     if (!xhci) {
         status = MX_ERR_NO_MEMORY;
@@ -284,7 +291,7 @@ static mx_status_t usb_xhci_bind(void* ctx, mx_device_t* dev, void** cookie) {
     // used for enabling bus mastering
     memcpy(&xhci->pci, &pci, sizeof(pci_protocol_t));
 
-    status = xhci_init(xhci, mmio);
+    status = xhci_init(xhci, mmio, bti);
     if (status != MX_OK) {
         goto error_return;
     }

system/dev/usb/xhci/xhci.c

@@ -161,7 +161,7 @@ static mx_status_t xhci_vmo_init(size_t size, mx_handle_t* out_handle, mx_vaddr_
     }
 
     if (!contiguous) {
-        // needs to be done before MX_VMO_OP_LOOKUP for non-contiguous VMOs
+        // needs to be done before mx_bti_pin for non-contiguous VMOs
         status = mx_vmo_op_range(handle, MX_VMO_OP_COMMIT, 0, size, NULL, 0);
         if (status != MX_OK) {
             printf("xhci_vmo_init: mx_vmo_op_range(MX_VMO_OP_COMMIT) failed %d\n", status);
@@ -189,7 +189,7 @@ static void xhci_vmo_release(mx_handle_t handle, mx_vaddr_t virt) {
     mx_handle_close(handle);
 }
 
-mx_status_t xhci_init(xhci_t* xhci, void* mmio) {
+mx_status_t xhci_init(xhci_t* xhci, void* mmio, mx_handle_t bti) {
     mx_status_t result = MX_OK;
     mx_paddr_t* phys_addrs = NULL;
 
@@ -284,15 +284,24 @@ mx_status_t xhci_init(xhci_t* xhci, void* mmio) {
 
     // set up DCBAA, ERST array and input context
     xhci->dcbaa = (uint64_t *)xhci->dcbaa_erst_virt;
-    result = mx_vmo_op_range(xhci->dcbaa_erst_handle, MX_VMO_OP_LOOKUP, 0, PAGE_SIZE,
-                             &xhci->dcbaa_phys, sizeof(xhci->dcbaa_phys));
+    uint32_t expected_num_addrs;
+    result = mx_bti_pin(bti, xhci->dcbaa_erst_handle, 0, PAGE_SIZE,
+                        MX_VM_FLAG_PERM_READ | MX_VM_FLAG_PERM_WRITE,
+                        &xhci->dcbaa_phys, 1, &expected_num_addrs);
+    if (result == MX_OK && expected_num_addrs != 1) {
+        result = MX_ERR_BAD_STATE;
+    }
     if (result != MX_OK) {
         printf("mx_vmo_op_range failed for xhci->dcbaa_erst_handle\n");
         goto fail;
     }
     xhci->input_context = (uint8_t *)xhci->input_context_virt;
-    result = mx_vmo_op_range(xhci->input_context_handle, MX_VMO_OP_LOOKUP, 0, PAGE_SIZE,
-                             &xhci->input_context_phys, sizeof(xhci->input_context_phys));
+    result = mx_bti_pin(bti, xhci->input_context_handle, 0, PAGE_SIZE,
+                        MX_VM_FLAG_PERM_READ | MX_VM_FLAG_PERM_WRITE,
+                        &xhci->input_context_phys, 1, &expected_num_addrs);
+    if (result == MX_OK && expected_num_addrs != 1) {
+        result = MX_ERR_BAD_STATE;
+    }
     if (result != MX_OK) {
         printf("mx_vmo_op_range failed for xhci->input_context_handle\n");
         goto fail;
@@ -308,8 +317,12 @@ mx_status_t xhci_init(xhci_t* xhci, void* mmio) {
         off_t offset = 0;
         for (uint32_t i = 0; i < scratch_pad_bufs; i++) {
             mx_paddr_t scratch_pad_phys;
-            result = mx_vmo_op_range(xhci->scratch_pad_pages_handle, MX_VMO_OP_LOOKUP, offset,
-                                     PAGE_SIZE, &scratch_pad_phys, sizeof(scratch_pad_phys));
+            result = mx_bti_pin(bti, xhci->scratch_pad_pages_handle, offset, xhci->page_size,
+                                MX_VM_FLAG_PERM_READ | MX_VM_FLAG_PERM_WRITE,
+                                &scratch_pad_phys, 1, &expected_num_addrs);
+            if (result == MX_OK && expected_num_addrs != 1) {
+                result = MX_ERR_BAD_STATE;
+            }
             if (result != MX_OK) {
                 printf("mx_vmo_op_range failed for xhci->scratch_pad_pages_handle\n");
                 goto fail;
@@ -319,8 +332,13 @@ mx_status_t xhci_init(xhci_t* xhci, void* mmio) {
         }
 
         mx_paddr_t scratch_pad_index_phys;
-        result = mx_vmo_op_range(xhci->scratch_pad_index_handle, MX_VMO_OP_LOOKUP, 0, PAGE_SIZE,
-                                  &scratch_pad_index_phys, sizeof(scratch_pad_index_phys));
+        const size_t scratch_pad_index_size = PAGE_ROUNDUP(scratch_pad_bufs * sizeof(uint64_t));
+        result = mx_bti_pin(bti, xhci->scratch_pad_index_handle, 0, scratch_pad_index_size,
+                            MX_VM_FLAG_PERM_READ | MX_VM_FLAG_PERM_WRITE,
+                            &scratch_pad_index_phys, 1, &expected_num_addrs);
+        if (result == MX_OK && expected_num_addrs != 1) {
+            result = MX_ERR_BAD_STATE;
+        }
         if (result != MX_OK) {
             printf("mx_vmo_op_range failed for xhci->scratch_pad_index_handle\n");
             goto fail;

system/dev/usb/xhci/xhci.h

@@ -177,7 +177,7 @@ struct xhci {
     mx_vaddr_t scratch_pad_index_virt;
 };
 
-mx_status_t xhci_init(xhci_t* xhci, void* mmio);
+mx_status_t xhci_init(xhci_t* xhci, void* mmio, mx_handle_t bti);
 int xhci_get_ep_ctx_state(xhci_endpoint_t* ep);
 mx_status_t xhci_start(xhci_t* xhci);
 void xhci_handle_interrupt(xhci_t* xhci, bool legacy);

system/public/magenta/rights.h

@@ -80,3 +80,9 @@
   (MX_RIGHT_DUPLICATE | MX_RIGHT_TRANSFER | MX_RIGHT_READ | MX_RIGHT_WRITE | \
    MX_RIGHT_EXECUTE | MX_RIGHT_MAP | MX_RIGHT_GET_PROPERTY |                 \
    MX_RIGHT_SET_PROPERTY | MX_RIGHT_SIGNAL)
+
+#define MX_DEFAULT_IOMMU_RIGHTS \
+  (MX_RIGHT_DUPLICATE | MX_RIGHT_TRANSFER)
+
+#define MX_DEFAULT_BTI_RIGHTS \
+  (MX_RIGHT_DUPLICATE | MX_RIGHT_TRANSFER | MX_RIGHT_READ | MX_RIGHT_MAP)

system/public/magenta/syscalls.sysgen

@@ -532,6 +532,25 @@ syscall vmo_create_physical
     (rsrc_handle: mx_handle_t, paddr: mx_paddr_t, size: size_t)
     returns (mx_status_t, out: mx_handle_t);
 
+# DDK Syscalls: Device Memory Access
+
+syscall iommu_create
+    (rsrc_handle: mx_handle_t, type: uint32_t, desc: any[desc_len] IN, desc_len: uint32_t)
+    returns (mx_status_t, out: mx_handle_t);
+
+syscall bti_create
+    (iommu: mx_handle_t, bti_id: uint64_t)
+    returns (mx_status_t, out: mx_handle_t);
+
+syscall bti_pin
+    (bti: mx_handle_t, vmo: mx_handle_t, offset: uint64_t, size: uint64_t,
+        perms: uint32_t, extents: uint64_t[extents_len] OUT, extents_len: uint32_t)
+    returns (mx_status_t, actual: uint32_t);
+
+syscall bti_unpin
+    (bti: mx_handle_t, extents: uint64_t[extents_len] IN, extents_len: uint32_t)
+    returns (mx_status_t);
+
 # DDK Syscalls: Misc Info
 
 syscall bootloader_fb_get_info

system/public/magenta/syscalls/iommu.h

@@ -0,0 +1,93 @@
+// Copyright 2017 The Fuchsia Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#pragma once
+
+#include <assert.h>
+#include <magenta/compiler.h>
+#include <stdbool.h>
+#include <stdint.h>
+
+__BEGIN_CDECLS
+
+#define MX_IOMMU_MAX_DESC_LEN 4096
+
+// Values for the |type| argument of the mx_iommu_create() syscall.
+#define MX_IOMMU_TYPE_DUMMY 0
+#define MX_IOMMU_TYPE_INTEL 1
+
+// Data structures for creating a dummy IOMMU instance
+typedef struct mx_iommu_desc_dummy {
+    uint8_t reserved;
+} mx_iommu_desc_dummy_t;
+
+
+// Data structures for creating an Intel IOMMU instance
+
+// This scope represents a single PCI endpoint device
+#define MX_IOMMU_INTEL_SCOPE_ENDPOINT 0
+// This scope represents a PCI-PCI bridge.  The bridge and all of its downstream
+// devices will be included in this scope.
+#define MX_IOMMU_INTEL_SCOPE_BRIDGE   1
+
+typedef struct mx_iommu_desc_intel_scope {
+    uint8_t type;
+    // The bus number of the first bus decoded by the host bridge this scope is attached to.
+    uint8_t start_bus;
+    // Number of bridges (including the host bridge) between host bridge and the
+    // device.
+    uint8_t num_hops;
+    // The device number and function numbers of the bridges along the way,
+    // ending with the device itself.
+    // |dev_func[0]| is the address on |start_bus| of the first bridge in the
+    // path (excluding the host bridge).  |dev_func[num_hops-1]| is the address
+    // of the the device itself.
+    uint8_t dev_func[5];
+} mx_iommu_desc_intel_scope_t;
+
+typedef struct mx_iommu_desc_intel_reserved_memory {
+    uint64_t base_addr; // Physical address of the base of reserved memory.
+    uint64_t len; // Number of bytes of reserved memory.
+
+    // The number of bytes of mx_iommu_desc_intel_scope_t's that follow this descriptor.
+    uint8_t scope_bytes;
+
+    uint8_t _reserved[7]; // Padding
+
+    // This is a list of all devices that need access to this memory range.
+    //
+    // mx_iommu_desc_intel_scope_t scopes[num_scopes];
+} mx_iommu_desc_intel_reserved_memory_t;
+
+typedef struct mx_iommu_desc_intel {
+    uint64_t register_base; // Physical address of registers
+    uint16_t pci_segment; // The PCI segment associated with this IOMMU
+
+    // If true, this IOMMU has all PCI devices in its segment under its scope.
+    // In this case, the list of scopes acts as a blacklist.
+    bool whole_segment;
+
+    // The number of bytes of mx_iommu_desc_intel_scope_t's that follow this descriptor.
+    uint8_t scope_bytes;
+
+    // The number of bytes of mx_iommu_desc_intel_reserved_memory_t's that follow the scope
+    // list.
+    uint16_t reserved_memory_bytes;
+
+    uint8_t _reserved[2]; // Padding
+
+    // If |whole_segment| is false, this is a list of all devices managed by
+    // this IOMMU.  If |whole_segment| is true, this is a list of all devices on
+    // this segment *not* managed by this IOMMU.  It has a total length in bytes of
+    // |scope_bytes|.
+    //
+    // mx_iommu_desc_intel_scope_t scopes[];
+
+    // A list of all BIOS-reserved memory regions this IOMMU needs to translate.
+    // It has a total length in bytes of |reserved_memory_bytes|.
+    //
+    // mx_iommu_desc_intel_reserved_memory_t reserved_mem[];
+} mx_iommu_desc_intel_t;
+
+__END_CDECLS

system/public/magenta/syscalls/object.h

@@ -53,6 +53,8 @@ typedef enum {
     MX_OBJ_TYPE_GUEST               = 20,
     MX_OBJ_TYPE_VCPU                = 21,
     MX_OBJ_TYPE_TIMER               = 22,
+    MX_OBJ_TYPE_IOMMU               = 23,
+    MX_OBJ_TYPE_BTI                 = 24,
     MX_OBJ_TYPE_LAST
 } mx_obj_type_t;
 

system/ulib/ddk/include/ddk/iotxn.h

@@ -186,6 +186,9 @@ ssize_t iotxn_copyfrom(iotxn_t* txn, void* data, size_t length, size_t offset);
 // Out of range operations are ignored.
 ssize_t iotxn_copyto(iotxn_t* txn, const void* data, size_t length, size_t offset);
 
+// set the bti to be used when performing physmap
+void iotxn_set_default_bti(mx_handle_t bti);
+
 // iotxn_physmap() looks up the physical pages backing this iotxn's vm object.
 // the 'phys' and 'phys_count' fields are set if this function succeeds.
 mx_status_t iotxn_physmap(iotxn_t* txn);

system/ulib/ddk/include/ddk/protocol/pci.h

@@ -32,6 +32,8 @@ typedef struct pci_protocol_ops {
     mx_status_t (*map_resource)(void* ctx, uint32_t res_id, uint32_t cache_policy,
                                 void** vaddr, size_t* size, mx_handle_t* out_handle);
     mx_status_t (*enable_bus_master)(void* ctx, bool enable);
+    // TODO(teisenbe): Merge this into enable_bus_master
+    mx_status_t (*get_bti)(void* ctx, mx_handle_t* out_handle);
     mx_status_t (*enable_pio)(void* ctx, bool enable);
     mx_status_t (*reset_device)(void* ctx);
     mx_status_t (*map_interrupt)(void* ctx, int which_irq, mx_handle_t* out_handle);

system/ulib/ddk/internal.h

@@ -0,0 +1,7 @@
+// Copyright 2017 The Fuchsia Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#pragma once
+
+extern mx_handle_t io_default_bti;

system/ulib/ddk/io-buffer.c

@@ -4,11 +4,14 @@
 
 #include <ddk/io-buffer.h>
 #include <ddk/driver.h>
+#include <magenta/assert.h>
 #include <magenta/process.h>
 #include <magenta/syscalls.h>
 #include <limits.h>
 #include <stdio.h>
 
+#include "internal.h"
+
 static mx_status_t io_buffer_init_common(io_buffer_t* buffer, mx_handle_t vmo_handle, size_t size,
                                          mx_off_t offset, uint32_t flags) {
     mx_vaddr_t virt;
@@ -21,8 +24,18 @@ static mx_status_t io_buffer_init_common(io_buffer_t* buffer, mx_handle_t vmo_ha
     }
 
     mx_paddr_t phys;
-    size_t lookup_size = size < PAGE_SIZE ? size : PAGE_SIZE;
-    status = mx_vmo_op_range(vmo_handle, MX_VMO_OP_LOOKUP, 0, lookup_size, &phys, sizeof(phys));
+    if (io_default_bti == MX_HANDLE_INVALID) {
+        size_t lookup_size = size < PAGE_SIZE ? size : PAGE_SIZE;
+        status = mx_vmo_op_range(vmo_handle, MX_VMO_OP_LOOKUP, 0, lookup_size, &phys, sizeof(phys));
+    } else {
+        uint32_t actual_extents;
+        status = mx_bti_pin(io_default_bti, vmo_handle, 0, ROUNDUP(size, PAGE_SIZE),
+                            MX_VM_FLAG_PERM_READ | MX_VM_FLAG_PERM_WRITE,
+                            &phys, 1, &actual_extents);
+        // We should remove this assert by tracking the length of this array
+        // explicitly in buffer.
+        MX_DEBUG_ASSERT(status != MX_OK || actual_extents == 1);
+    }
     if (status != MX_OK) {
         printf("io_buffer: mx_vmo_op_range failed %d size: %zu\n", status, size);
         mx_vmar_unmap(mx_vmar_root_self(), virt, size);
@@ -116,6 +129,11 @@ mx_status_t io_buffer_init_physical(io_buffer_t* buffer, mx_paddr_t addr, size_t
 
 void io_buffer_release(io_buffer_t* buffer) {
     if (buffer->vmo_handle) {
+        if (io_default_bti != MX_HANDLE_INVALID) {
+            mx_status_t status = mx_bti_unpin(io_default_bti, &buffer->phys, 1);
+            MX_DEBUG_ASSERT(status == MX_OK);
+        }
+
         mx_vmar_unmap(mx_vmar_root_self(), (uintptr_t)buffer->virt, buffer->size);
         mx_handle_close(buffer->vmo_handle);
         buffer->vmo_handle = MX_HANDLE_INVALID;

system/ulib/ddk/iotxn.c

@@ -16,6 +16,8 @@
 #include <inttypes.h>
 #include <threads.h>
 
+#include "internal.h"
+
 #define TRACE 0
 
 #if TRACE
@@ -35,6 +37,7 @@
 #define IOTXN_PFLAG_MMAP       (1 << 3)   // we performed mmap() on this vmo
 #define IOTXN_PFLAG_FREE       (1 << 4)   // this txn has been released
 #define IOTXN_PFLAG_QUEUED     (1 << 5)   // transaction has been queued and not yet released
+#define IOTXN_PFLAG_BTI        (1 << 6)   // this data was pinned via the BTI interface
 
 #define IOTXN_STATE_MASK       (IOTXN_PFLAG_FREE | IOTXN_PFLAG_QUEUED)
 
@@ -45,6 +48,12 @@ static size_t free_list_length = 0;
 static size_t free_list_monitor_warned = 0;
 #endif
 
+mx_handle_t io_default_bti = MX_HANDLE_INVALID;
+void iotxn_set_default_bti(mx_handle_t bti) {
+    // TODO: locking
+    io_default_bti = bti;
+}
+
 // This assert will fail if we attempt to access the buffer of a cloned txn after it has been completed
 #define ASSERT_BUFFER_VALID(priv) MX_DEBUG_ASSERT(!(priv->flags & IOTXN_FLAG_DEAD))
 
@@ -89,6 +98,7 @@ static iotxn_t* find_in_free_list(uint32_t pflags, uint64_t data_size) {
 
 // return the iotxn into the free list
 static void iotxn_release_free_list(iotxn_t* txn) {
+    xprintf("iotxn_release_free_list txn %p\n", txn);
     mx_handle_t vmo_handle = txn->vmo_handle;
     uint64_t vmo_offset = txn->vmo_offset;
     uint64_t vmo_length = txn->vmo_length;
@@ -114,6 +124,12 @@ static void iotxn_release_free_list(iotxn_t* txn) {
     } else {
         if (do_free_phys(pflags)) {
             if (phys != NULL) {
+                if (pflags & IOTXN_PFLAG_BTI) {
+                    mx_status_t status = mx_bti_unpin(io_default_bti, phys, phys_count);
+                    if (status != MX_OK) {
+                        printf("iotxn_release_free_list: bti unpin failed %d\n", status);
+                    }
+                }
                 free(phys);
             }
         }
@@ -138,15 +154,25 @@ static void iotxn_release_free_list(iotxn_t* txn) {
         free_list_monitor_warned = free_list_length;
     }
 #endif
+    xprintf("iotxn_release_free_list released txn %p old %#x new %#x\n", txn, pflags, txn->pflags);
     mtx_unlock(&free_list_mutex);
+}
 
-    xprintf("iotxn_release_free_list released txn %p\n", txn);
+static bool is_clone(iotxn_t* txn) {
+    return txn->release_cb == iotxn_release_free_list;
 }
 
 // free the iotxn
 static void iotxn_release_free(iotxn_t* txn) {
+    xprintf("iotxn_release_free txn %p\n", txn);
     if (do_free_phys(txn->pflags)) {
         if (txn->phys != NULL) {
+            if (txn->pflags & IOTXN_PFLAG_BTI) {
+                mx_status_t status = mx_bti_unpin(io_default_bti, txn->phys, txn->phys_count);
+                if (status != MX_OK) {
+                    printf("iotxn_release_free: bti unpin failed %d\n", status);
+                }
+            }
             free(txn->phys);
         }
     }
@@ -163,17 +189,22 @@ static void iotxn_release_free(iotxn_t* txn) {
 
 // releases data for a statically allocated iotxn
 static void iotxn_release_static(iotxn_t* txn) {
-    uint32_t pflags = txn->pflags;
-
+    xprintf("iotxn_release_static txn %p\n", txn);
     if (do_free_phys(txn->pflags)) {
         // only free the scatter list if we called physmap()
         if (txn->phys != NULL) {
+            if (txn->pflags & IOTXN_PFLAG_BTI) {
+                mx_status_t status = mx_bti_unpin(io_default_bti, txn->phys, txn->phys_count);
+                if (status != MX_OK) {
+                    printf("iotxn_release_static: bti unpin failed %d\n", status);
+                }
+            }
             free(txn->phys);
             txn->phys = NULL;
             txn->phys_count = 0;
         }
     }
-    if (pflags & IOTXN_PFLAG_MMAP) {
+    if (txn->pflags & IOTXN_PFLAG_MMAP) {
         // only unmap if we called mmap()
         if (txn->virt) {
             mx_vmar_unmap(mx_vmar_root_self(), (uintptr_t)txn->virt, txn->vmo_length);
@@ -188,6 +219,22 @@ void iotxn_complete(iotxn_t* txn, mx_status_t status, mx_off_t actual) {
     MX_DEBUG_ASSERT((txn->pflags & IOTXN_STATE_MASK) == IOTXN_PFLAG_QUEUED);
     txn->pflags &= ~IOTXN_PFLAG_QUEUED;
 
+    // If this transaction is a clone, and it pinned memory, release it before
+    // we call the complete_cb, in case the parent txn tries to pin the memory
+    // afterwards.
+    if (is_clone(txn) && do_free_phys(txn->pflags) && (txn->pflags & IOTXN_PFLAG_BTI)) {
+        MX_DEBUG_ASSERT(txn->phys);
+
+        mx_status_t status = mx_bti_unpin(io_default_bti, txn->phys, txn->phys_count);
+        if (status != MX_OK) {
+            printf("iotxn_complete: bti unpin failed %d\n", status);
+        }
+        free(txn->phys);
+        txn->phys = NULL;
+        txn->phys_count = 0;
+        txn->pflags &= ~(IOTXN_PFLAG_BTI | IOTXN_PFLAG_PHYSMAP);
+    }
+
     txn->actual = actual;
     txn->status = status;
     if (txn->complete_cb) {
@@ -214,17 +261,30 @@ ssize_t iotxn_copyto(iotxn_t* txn, const void* data, size_t length, size_t offse
 static mx_status_t iotxn_physmap_contiguous(iotxn_t* txn) {
     txn->phys = txn->phys_inline;
 
-    // for contiguous buffers, commit the whole range but just map the first
-    // page
+    // for contiguous buffers, commit the whole range
     uint64_t page_offset = ROUNDDOWN(txn->vmo_offset, PAGE_SIZE);
     mx_status_t status = mx_vmo_op_range(txn->vmo_handle, MX_VMO_OP_COMMIT, page_offset, txn->vmo_length, NULL, 0);
     if (status != MX_OK) {
         goto fail;
     }
 
-    status = mx_vmo_op_range(txn->vmo_handle, MX_VMO_OP_LOOKUP, page_offset, PAGE_SIZE, txn->phys, sizeof(mx_paddr_t));
-    if (status != MX_OK) {
-        goto fail;
+    if (io_default_bti == MX_HANDLE_INVALID) {
+        status = mx_vmo_op_range(txn->vmo_handle, MX_VMO_OP_LOOKUP, page_offset, PAGE_SIZE, txn->phys, sizeof(mx_paddr_t));
+        if (status != MX_OK) {
+            xprintf("iotxn_physmap_contiguous: error %d in lookup\n", status);
+            goto fail;
+        }
+    } else {
+        uint32_t actual_extents;
+        status = mx_bti_pin(io_default_bti, txn->vmo_handle, page_offset, txn->vmo_length,
+                            MX_VM_FLAG_PERM_READ | MX_VM_FLAG_PERM_WRITE,
+                            txn->phys, 1, &actual_extents);
+        if (status != MX_OK) {
+            xprintf("iotxn_physmap_contiguous: error %d in bti_pin\n", status);
+            goto fail;
+        }
+        MX_DEBUG_ASSERT(actual_extents == 1);
+        txn->pflags |= IOTXN_PFLAG_BTI;
     }
 
     txn->phys_count = 1;
@@ -253,27 +313,40 @@ static mx_status_t iotxn_physmap_paged(iotxn_t* txn) {
     mx_status_t status = mx_vmo_op_range(txn->vmo_handle, MX_VMO_OP_COMMIT, txn->vmo_offset, txn->vmo_length, NULL, 0);
     if (status != MX_OK) {
         xprintf("iotxn_physmap_paged: error %d in commit\n", status);
-        if (!use_inline) {
-            free(paddrs);
-        }
-        return status;
+        goto fail;
     }
 
-    status = mx_vmo_op_range(txn->vmo_handle, MX_VMO_OP_LOOKUP, page_offset, page_length, paddrs, sizeof(mx_paddr_t) * pages);
-    if (status != MX_OK) {
-        xprintf("iotxn_physmap_paged: error %d in lookup\n", status);
-        if (!use_inline) {
-            free(paddrs);
+    if (io_default_bti == MX_HANDLE_INVALID) {
+        status = mx_vmo_op_range(txn->vmo_handle, MX_VMO_OP_LOOKUP, page_offset, page_length, paddrs, sizeof(mx_paddr_t) * pages);
+        if (status != MX_OK) {
+            xprintf("iotxn_physmap_paged: error %d in lookup\n", status);
+            goto fail;
         }
-        return status;
+        txn->phys_count = pages;
+    } else {
+        uint32_t actual_extents;
+        status = mx_bti_pin(io_default_bti, txn->vmo_handle, page_offset, page_length,
+                            MX_VM_FLAG_PERM_READ | MX_VM_FLAG_PERM_WRITE,
+                            paddrs, pages, &actual_extents);
+        if (status != MX_OK) {
+            xprintf("iotxn_physmap_paged: error %d in bti_pin\n", status);
+            goto fail;
+        }
+        txn->phys_count = actual_extents;
+        txn->pflags |= IOTXN_PFLAG_BTI;
     }
 
     if (!use_inline) {
         txn->pflags |= IOTXN_PFLAG_PHYSMAP;
     }
     txn->phys = paddrs;
-    txn->phys_count = pages;
     return MX_OK;
+fail:
+    if (!use_inline) {
+        free(paddrs);
+    }
+    txn->phys = NULL;
+    return status;
 }
 
 mx_status_t iotxn_physmap(iotxn_t* txn) {
@@ -289,6 +362,7 @@ mx_status_t iotxn_physmap(iotxn_t* txn) {
     } else {
         status = iotxn_physmap_paged(txn);
     }
+    xprintf("iotxn_physmap: txn %p, vmo %d, offset %#lx, len %#lx, status %d\n", txn, txn->vmo_handle, txn->vmo_offset, txn->vmo_length, status);
     return status;
 }
 
@@ -328,6 +402,8 @@ mx_status_t iotxn_clone(iotxn_t* txn, iotxn_t** out) {
     // clones are always freelisted on release
     clone->release_cb = iotxn_release_free_list;
 
+    xprintf("iotxn_clone txn %p: new %p\n", txn, clone);
+
     *out = clone;
     return MX_OK;
 }
@@ -384,6 +460,8 @@ void iotxn_release(iotxn_t* txn) {
     // should not release a queued transaction
     MX_DEBUG_ASSERT((txn->pflags & IOTXN_STATE_MASK) == 0);
 
+    xprintf("iotxn_release: txn %p, vmo %d, offset %#lx, len %#lx\n", txn, txn->vmo_handle, txn->vmo_offset, txn->vmo_length);
+
     if (txn->release_cb) {
         txn->release_cb(txn);
     }

system/ulib/hwreg/include/hwreg/bitfields.h

@@ -0,0 +1,250 @@
+// Copyright 2017 The Fuchsia Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#pragma once
+
+#include <hwreg/mmio.h>
+
+#include <magenta/assert.h>
+#include <mxtl/type_support.h>
+#include <stdint.h>
+
+// This file provides some helpers for accessing bitfields in registers.
+//
+// Example usage:
+//
+//   // Define bitfields for an "AuxControl" register.
+//   class AuxControl : public RegisterBase<uint32_t> {
+//   public:
+//       // Define a single-bit field.
+//       DEF_BIT(31, enabled);
+//       // Define a 5-bit field, from bits 20-24 (inclusive).
+//       DEF_FIELD(24, 20, message_size);
+//
+//       // Returns an object representing the register's type and address.
+//       static auto Get() { return RegisterAddr<AuxControl>(0x64010); }
+//   };
+//
+//   void Example1(RegisterIo* reg_io)
+//   {
+//       // Read the register's value from MMIO.  "reg" is a snapshot of the
+//       // register's value which also knows the register's address.
+//       auto reg = AuxControl::Get().ReadFrom(reg_io);
+//
+//       // Read this register's "message_size" field.
+//       uint32_t size = reg.message_size();
+//
+//       // Change this field's value.  This modifies the snapshot.
+//       reg.set_message_size(1234);
+//
+//       // Write the modified register value to MMIO.
+//       reg.WriteTo(reg_io);
+//   }
+//
+//   // It is also possible to write a register without having to read it
+//   // first:
+//   void Example2(RegisterIo* reg_io)
+//   {
+//       // Start off with a value that is initialized to zero.
+//       auto reg = AuxControl::Get().FromValue(0);
+//       // Fill out fields.
+//       reg.set_message_size(2345);
+//       // Write the register value to MMIO.
+//       reg.WriteTo(reg_io);
+//   }
+//
+// Note that this produces efficient code with GCC and Clang, which are
+// capable of optimizing away the intermediate objects.
+//
+// The arguments to DEF_FIELD() are organized to match up with Intel's
+// documentation.  For example, if the docs specify a field as:
+//   23:0  Data M value
+// then that translates to:
+//   DEF_FIELD(23, 0, data_m_value)
+// To match up, we put the upper bit first and use an inclusive bit range.
+
+namespace hwreg {
+
+// An instance of RegisterBase represents a staging copy of a register,
+// which can be written to the register itself.  It knows the register's
+// address and stores a value for the register.
+//
+// Normal usage is to create classes that derive from RegisterBase and
+// provide methods for accessing bitfields of the register.  RegisterBase
+// does not provide a constructor because constructors are not inherited by
+// derived classes by default, and we don't want the derived classes to
+// have to declare constructors.
+template <class IntType> class RegisterBase {
+public:
+    using ValueType = IntType;
+    uint32_t reg_addr() { return reg_addr_; }
+    void set_reg_addr(uint32_t addr) { reg_addr_ = addr; }
+
+    ValueType reg_value() { return reg_value_; }
+    ValueType* reg_value_ptr() { return &reg_value_; }
+    void set_reg_value(IntType value) { reg_value_ = value; }
+
+    void ReadFrom(RegisterIo* reg_io) { reg_value_ = reg_io->Read<ValueType>(reg_addr_); }
+    void WriteTo(RegisterIo* reg_io) { reg_io->Write(reg_addr_, reg_value_ & ~rsvdz_mask_); }
+
+protected:
+    ValueType rsvdz_mask_ = 0;
+private:
+    uint32_t reg_addr_ = 0;
+    ValueType reg_value_ = 0;
+};
+
+// An instance of RegisterAddr represents a typed register address: It
+// knows the address of the register (within the MMIO address space) and
+// the type of its contents, RegType.  RegType represents the register's
+// bitfields.  RegType should be a subclass of RegisterBase.
+template <class RegType> class RegisterAddr {
+public:
+    RegisterAddr(uint32_t reg_addr) : reg_addr_(reg_addr) {}
+
+    static_assert(mxtl::is_base_of<RegisterBase<typename RegType::ValueType>, RegType>::value,
+                  "Parameter of RegisterAddr<> should derive from RegisterBase");
+
+    // Instantiate a RegisterBase using the value of the register read from
+    // MMIO.
+    RegType ReadFrom(RegisterIo* reg_io)
+    {
+        RegType reg;
+        reg.set_reg_addr(reg_addr_);
+        reg.ReadFrom(reg_io);
+        return reg;
+    }
+
+    // Instantiate a RegisterBase using the given value for the register.
+    RegType FromValue(typename RegType::ValueType value)
+    {
+        RegType reg;
+        reg.set_reg_addr(reg_addr_);
+        reg.set_reg_value(value);
+        return reg;
+    }
+
+    uint32_t addr() { return reg_addr_; }
+
+private:
+    uint32_t reg_addr_;
+};
+
+namespace internal {
+
+template <class IntType> constexpr IntType ComputeMask(uint32_t num_bits) {
+    return static_cast<IntType>((static_cast<IntType>(1) << num_bits) - 1);
+}
+
+template <class IntType> class RsvdZField {
+public:
+    RsvdZField(IntType* mask, uint32_t bit_high_incl, uint32_t bit_low) {
+        *mask |= internal::ComputeMask<IntType>(bit_high_incl - bit_low + 1) << bit_low;
+    }
+};
+
+} // namespace internal
+
+// TODO(teisenbe): Maybe get rid of this class and turn it into utility
+// functions?
+template <class IntType> class BitfieldRef {
+public:
+    BitfieldRef(IntType* value_ptr, uint32_t bit_high_incl, uint32_t bit_low)
+        : value_ptr_(value_ptr), shift_(bit_low),
+          mask_(internal::ComputeMask<IntType>(bit_high_incl - bit_low + 1))
+    {
+    }
+
+    IntType get() { return (*value_ptr_ >> shift_) & mask_; }
+
+    void set(IntType field_val)
+    {
+        MX_DEBUG_ASSERT((field_val & ~mask_) == 0);
+        *value_ptr_ &= ~(mask_ << shift_);
+        *value_ptr_ |= (field_val << shift_);
+    }
+
+private:
+    IntType* const value_ptr_;
+    const uint32_t shift_;
+    const IntType mask_;
+};
+
+// Declares multi-bit fields in a derived class of RegisterBase<T>.  This
+// produces functions "T NAME()" and "void set_NAME(T)".  Both bit indices
+// are inclusive.
+#define DEF_FIELD(BIT_HIGH, BIT_LOW, NAME)                                                         \
+    static_assert((BIT_HIGH) > (BIT_LOW), "Upper bit goes before lower bit");                      \
+    static_assert((BIT_HIGH) < sizeof(ValueType) * CHAR_BIT, "Upper bit is out of range");         \
+    ValueType NAME()                                                                               \
+    {                                                                                              \
+        return hwreg::BitfieldRef<ValueType>(reg_value_ptr(), (BIT_HIGH), (BIT_LOW)).get();        \
+    }                                                                                              \
+    void set_ ## NAME(ValueType val)                                                               \
+    {                                                                                              \
+        hwreg::BitfieldRef<ValueType>(reg_value_ptr(), (BIT_HIGH), (BIT_LOW)).set(val);            \
+    }
+
+// Declares single-bit fields in a derived class of RegisterBase<T>.  This
+// produces functions "T NAME()" and "void set_NAME(T)".
+#define DEF_BIT(BIT, NAME)                                                                         \
+    static_assert((BIT) < sizeof(ValueType) * CHAR_BIT, "Bit is out of range");                    \
+    ValueType NAME()                                                                               \
+    {                                                                                              \
+        return hwreg::BitfieldRef<ValueType>(reg_value_ptr(), (BIT), (BIT)).get();                 \
+    }                                                                                              \
+    void set_ ## NAME(ValueType val)                                                               \
+    {                                                                                              \
+        hwreg::BitfieldRef<ValueType>(reg_value_ptr(), (BIT), (BIT)).set(val);                     \
+    }
+
+// Declares multi-bit reserved-zero fields in a derived class of RegisterBase<T>.
+// This will ensure that on RegisterBase<T>::WriteTo(), reserved-zero bits are
+// zeroed.  Both bit indices are inclusive.
+#define DEF_RSVDZ_FIELD(BIT_HIGH, BIT_LOW)                                                         \
+    static_assert((BIT_HIGH) > (BIT_LOW), "Upper bit goes before lower bit");                      \
+    static_assert((BIT_HIGH) < sizeof(ValueType) * CHAR_BIT, "Upper bit is out of range");         \
+    hwreg::internal::RsvdZField<ValueType> RsvdZ ## BIT_HIGH ## _ ## BIT_LOW =                     \
+        hwreg::internal::RsvdZField<ValueType>(&this->rsvdz_mask_, (BIT_HIGH), (BIT_LOW));
+
+// Declares single-bit reserved-zero fields in a derived class of RegisterBase<T>.
+// This will ensure that on RegisterBase<T>::WriteTo(), reserved-zero bits are
+// zeroed.
+#define DEF_RSVDZ_BIT(BIT)                                                                         \
+    static_assert((BIT) < sizeof(ValueType) * CHAR_BIT, "Bit is out of range");                    \
+    hwreg::internal::RsvdZField<ValueType> RsvdZ ## BIT =                                          \
+        hwreg::internal::RsvdZField<ValueType>(&this->rsvdz_mask_, (BIT), (BIT));
+
+// Declares "decltype(FIELD) NAME()" and "void set_NAME(decltype(FIELD))" that
+// reads/modifies the declared bitrange.  Both bit indices are inclusive.
+#define DEF_SUBFIELD(FIELD, BIT_HIGH, BIT_LOW, NAME)                                               \
+    typename mxtl::remove_reference<decltype(FIELD)>::type NAME()                                  \
+    {                                                                                              \
+        return hwreg::BitfieldRef<typename mxtl::remove_reference<decltype(FIELD)>::type>(         \
+            &FIELD, (BIT_HIGH), (BIT_LOW)).get();                                                  \
+    }                                                                                              \
+    void set_ ## NAME(typename mxtl::remove_reference<decltype(FIELD)>::type val)                  \
+    {                                                                                              \
+        hwreg::BitfieldRef<typename mxtl::remove_reference<decltype(FIELD)>::type>(                \
+                &FIELD, (BIT_HIGH), (BIT_LOW)).set(val);                                           \
+    }
+
+// Declares "decltype(FIELD) NAME()" and "void set_NAME(decltype(FIELD))" that
+// reads/modifies the declared bit.
+#define DEF_SUBBIT(FIELD, BIT, NAME)                                                               \
+    typename mxtl::remove_reference<decltype(FIELD)>::type NAME()                                  \
+    {                                                                                              \
+        static_assert((BIT) <                                                                      \
+                      sizeof(typename mxtl::remove_reference<decltype(FIELD)>::type) * CHAR_BIT,   \
+                      "Bit is out of range");                                                      \
+        return hwreg::BitfieldRef<typename mxtl::remove_reference<decltype(FIELD)>::type>(         \
+                &FIELD, (BIT), (BIT)).get();                                                       \
+    }                                                                                              \
+    void set_ ## NAME(typename mxtl::remove_reference<decltype(FIELD)>::type val)                  \
+    {                                                                                              \
+        hwreg::BitfieldRef<typename mxtl::remove_reference<decltype(FIELD)>::type>(                \
+                &FIELD, (BIT), (BIT)).set(val);                                                    \
+    }
+
+} // namespace hwreg

system/ulib/hwreg/include/hwreg/mmio.h

@@ -0,0 +1,33 @@
+// Copyright 2017 The Fuchsia Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#pragma once
+
+namespace hwreg {
+
+// Wrap MMIO for easier testing of device drivers
+
+class RegisterIo {
+public:
+    RegisterIo(volatile void* mmio) : mmio_(reinterpret_cast<uintptr_t>(mmio)) {
+    }
+
+    template <class IntType> void Write(uint32_t offset, IntType val) {
+        auto ptr = reinterpret_cast<volatile IntType*>(mmio_ + offset);
+        *ptr = val;
+    }
+
+    template <class IntType> IntType Read(uint32_t offset)
+    {
+        auto ptr = reinterpret_cast<volatile IntType*>(mmio_ + offset);
+        return *ptr;
+    }
+
+    uintptr_t base() const { return mmio_; }
+
+private:
+    const uintptr_t mmio_;
+};
+
+} // namespace hwreg

system/ulib/hwreg/rules.mk

@@ -0,0 +1,14 @@
+# Copyright 2017 The Fuchsia Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+LOCAL_DIR := $(GET_LOCAL_DIR)
+
+MODULE := $(LOCAL_DIR)
+
+MODULE_TYPE := userlib
+
+MODULE_SRCS += $(LOCAL_DIR)/empty.cpp
+MODULE_LIBS += system/ulib/mxtl
+
+include make/module.mk