ChangeSet 1.946.3.18, 2002/12/27 09:39:46-08:00, david-b@pacbell.net [PATCH] usbcore dma updates (and doc) Attached is a patch leveraging some of the new generic dma stuff: - Replaces dma mapping calls in usbcore with generic equivalents. This is a minor code shrink (which we'd hoped could happen). - Pass dma mask along, so net drivers can notice it'd be good to set NETIF_F_HIGHDMA; or scsi ones can set highmem_io. (Some Intel EHCI setups are able to support this kind of DMA.) - Updates one net driver (usbnet) to set NETIF_F_HIGHDMA when appropriate, mostly as an example (since I can't test this). - Provides Documentation/usb/dma.txt, describing current APIs. (Unchanged by this patch, except dma mask visibility.) - Converted another info() to dev_info(), and likewise a couple dbg() to dev_dbg() conversions in the modified routine. The number of FIXMEs was conserved: the generic API doesn't yet fix the error reporting bugs in the PCI-specific mapping API. diff -Nru a/Documentation/usb/dma.txt b/Documentation/usb/dma.txt --- /dev/null Wed Dec 31 16:00:00 1969 +++ b/Documentation/usb/dma.txt Fri Dec 27 23:56:55 2002 @@ -0,0 +1,104 @@ +In Linux 2.5 kernels (and later), USB device drivers have additional control +over how DMA may be used to perform I/O operations. The APIs are detailed +in the kernel usb programming guide (kerneldoc, from the source code). + + +API OVERVIEW + +The big picture is that USB drivers can continue to ignore most DMA issues, +though they still must provide DMA-ready buffers (see DMA-mapping.txt). +That's how they've worked through the 2.4 (and earlier) kernels. + +OR: they can now be DMA-aware. + +- New calls enable DMA-aware drivers, letting them allocate dma buffers and + manage dma mappings for existing dma-ready buffers (see below). + +- URBs have an additional "transfer_dma" field, as well as a transfer_flags + bit saying if it's valid. (Control requests also needed "setup_dma".) + +- "usbcore" will map those DMA addresses, if a DMA-aware driver didn't do it + first and set URB_NO_DMA_MAP. HCDs don't manage dma mappings for urbs. + +- There's a new "generic DMA API", parts of which are usable by USB device + drivers. Never use dma_set_mask() on any USB interface or device; that + would potentially break all devices sharing that bus. + + +ELIMINATING COPIES + +It's good to avoid making CPUs copy data needlessly. The costs can add up, +and effects like cache-trashing can impose subtle penalties. + +- When you're allocating a buffer for DMA purposes anyway, use the buffer + primitives. Think of them as kmalloc and kfree that give you the right + kind of addresses to store in urb->transfer_buffer and urb->transfer_dma, + while guaranteeing that hidden copies through DMA "bounce" buffers won't + slow things down. You'd also set URB_NO_DMA_MAP in urb->transfer_flags: + + void *usb_buffer_alloc (struct usb_device *dev, size_t size, + int mem_flags, dma_addr_t *dma); + + void usb_buffer_free (struct usb_device *dev, size_t size, + void *addr, dma_addr_t dma); + + The memory buffer returned is "dma-coherent"; sometimes you might need to + force a consistent memory access ordering by using memory barriers. It's + not using a streaming DMA mapping, so it's good for small transfers on + systems where the I/O would otherwise tie up an IOMMU mapping. + + Asking for 1/Nth of a page (as well as asking for N pages) is reasonably + space-efficient. + +- Devices on some EHCI controllers could handle DMA to/from high memory. + Driver probe() routines can notice this using a generic DMA call, then + tell higher level code (network, scsi, etc) about it like this: + + if (dma_supported (&intf->dev, 0xffffffffffffffffULL)) + net->features |= NETIF_F_HIGHDMA; + + That can eliminate dma bounce buffering of requests that originate (or + terminate) in high memory, in cases where the buffers aren't allocated + with usb_buffer_alloc() but instead are dma-mapped. + + +WORKING WITH EXISTING BUFFERS + +Existing buffers aren't usable for DMA without first being mapped into the +DMA address space of the device. + +- When you're using scatterlists, you can map everything at once. On some + systems, this kicks in an IOMMU and turns the scatterlists into single + DMA transactions: + + int usb_buffer_map_sg (struct usb_device *dev, unsigned pipe, + struct scatterlist *sg, int nents); + + void usb_buffer_dmasync_sg (struct usb_device *dev, unsigned pipe, + struct scatterlist *sg, int n_hw_ents); + + void usb_buffer_unmap_sg (struct usb_device *dev, unsigned pipe, + struct scatterlist *sg, int n_hw_ents); + + It's probably easier to use the new usb_sg_*() calls, which do the DMA + mapping and apply other tweaks to make scatterlist i/o be fast. + +- Some drivers may prefer to work with the model that they're mapping large + buffers, synchronizing their safe re-use. (If there's no re-use, then let + usbcore do the map/unmap.) Large periodic transfers make good examples + here, since it's cheaper to just synchronize the buffer than to unmap it + each time an urb completes and then re-map it on during resubmission. + + These calls all work with initialized urbs: urb->dev, urb->pipe, + urb->transfer_buffer, and urb->transfer_buffer_length must all be + valid when these calls are used: + + struct urb *usb_buffer_map (struct urb *urb); + + void usb_buffer_dmasync (struct urb *urb); + + void usb_buffer_unmap (struct urb *urb); + + The calls manage urb->transfer_dma for you, and set URB_NO_DMA_MAP so that + usbcore won't map or unmap the buffer. + diff -Nru a/drivers/usb/core/buffer.c b/drivers/usb/core/buffer.c --- a/drivers/usb/core/buffer.c Fri Dec 27 23:56:55 2002 +++ b/drivers/usb/core/buffer.c Fri Dec 27 23:56:55 2002 @@ -24,11 +24,14 @@ /* - * DMA-Consistent Buffers + * DMA-Coherent Buffers */ /* FIXME tune these based on pool statistics ... */ static const size_t pool_max [HCD_BUFFER_POOLS] = { + /* platforms without dma-friendly caches might need to + * prevent cacheline sharing... + */ 32, 128, 512, @@ -132,99 +135,4 @@ } } pci_free_consistent (hcd->pdev, size, addr, dma); -} - - -/* - * DMA-Mappings for arbitrary memory buffers - */ - -int hcd_buffer_map ( - struct usb_bus *bus, - void *addr, - dma_addr_t *dma, - size_t size, - int direction -) { - struct usb_hcd *hcd = bus->hcpriv; - - // FIXME pci_map_single() has no standard failure mode! - *dma = pci_map_single (hcd->pdev, addr, size, - (direction == USB_DIR_IN) - ? PCI_DMA_FROMDEVICE - : PCI_DMA_TODEVICE); - return 0; -} - -void hcd_buffer_dmasync ( - struct usb_bus *bus, - dma_addr_t dma, - size_t size, - int direction -) { - struct usb_hcd *hcd = bus->hcpriv; - - pci_dma_sync_single (hcd->pdev, dma, size, - (direction == USB_DIR_IN) - ? PCI_DMA_FROMDEVICE - : PCI_DMA_TODEVICE); -} - -void hcd_buffer_unmap ( - struct usb_bus *bus, - dma_addr_t dma, - size_t size, - int direction -) { - struct usb_hcd *hcd = bus->hcpriv; - - pci_unmap_single (hcd->pdev, dma, size, - (direction == USB_DIR_IN) - ? PCI_DMA_FROMDEVICE - : PCI_DMA_TODEVICE); -} - -int hcd_buffer_map_sg ( - struct usb_bus *bus, - struct scatterlist *sg, - int *n_hw_ents, - int nents, - int direction -) { - struct usb_hcd *hcd = bus->hcpriv; - - // FIXME pci_map_sg() has no standard failure mode! - *n_hw_ents = pci_map_sg(hcd->pdev, sg, nents, - (direction == USB_DIR_IN) - ? PCI_DMA_FROMDEVICE - : PCI_DMA_TODEVICE); - return 0; -} - -void hcd_buffer_sync_sg ( - struct usb_bus *bus, - struct scatterlist *sg, - int n_hw_ents, - int direction -) { - struct usb_hcd *hcd = bus->hcpriv; - - pci_dma_sync_sg(hcd->pdev, sg, n_hw_ents, - (direction == USB_DIR_IN) - ? PCI_DMA_FROMDEVICE - : PCI_DMA_TODEVICE); -} - -void hcd_buffer_unmap_sg ( - struct usb_bus *bus, - struct scatterlist *sg, - int n_hw_ents, - int direction -) { - struct usb_hcd *hcd = bus->hcpriv; - - pci_unmap_sg(hcd->pdev, sg, n_hw_ents, - (direction == USB_DIR_IN) - ? PCI_DMA_FROMDEVICE - : PCI_DMA_TODEVICE); } diff -Nru a/drivers/usb/core/hcd-pci.c b/drivers/usb/core/hcd-pci.c --- a/drivers/usb/core/hcd-pci.c Fri Dec 27 23:56:55 2002 +++ b/drivers/usb/core/hcd-pci.c Fri Dec 27 23:56:55 2002 @@ -138,7 +138,8 @@ hcd->pdev = dev; hcd->self.bus_name = dev->slot_name; hcd->product_desc = dev->dev.name; - hcd->controller = &dev->dev; + hcd->self.controller = &dev->dev; + hcd->controller = hcd->self.controller; if ((retval = hcd_buffer_create (hcd)) != 0) { clean_3: diff -Nru a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c --- a/drivers/usb/core/hcd.c Fri Dec 27 23:56:55 2002 +++ b/drivers/usb/core/hcd.c Fri Dec 27 23:56:55 2002 @@ -1031,19 +1031,19 @@ /* lower level hcd code should use *_dma exclusively */ if (!(urb->transfer_flags & URB_NO_DMA_MAP)) { if (usb_pipecontrol (urb->pipe)) - urb->setup_dma = pci_map_single ( - hcd->pdev, + urb->setup_dma = dma_map_single ( + hcd->controller, urb->setup_packet, sizeof (struct usb_ctrlrequest), - PCI_DMA_TODEVICE); + DMA_TO_DEVICE); if (urb->transfer_buffer_length != 0) - urb->transfer_dma = pci_map_single ( - hcd->pdev, + urb->transfer_dma = dma_map_single ( + hcd->controller, urb->transfer_buffer, urb->transfer_buffer_length, usb_pipein (urb->pipe) - ? PCI_DMA_FROMDEVICE - : PCI_DMA_TODEVICE); + ? DMA_FROM_DEVICE + : DMA_TO_DEVICE); } status = hcd->driver->urb_enqueue (hcd, urb, mem_flags); @@ -1265,12 +1265,6 @@ .deallocate = hcd_free_dev, .buffer_alloc = hcd_buffer_alloc, .buffer_free = hcd_buffer_free, - .buffer_map = hcd_buffer_map, - .buffer_dmasync = hcd_buffer_dmasync, - .buffer_unmap = hcd_buffer_unmap, - .buffer_map_sg = hcd_buffer_map_sg, - .buffer_dmasync_sg = hcd_buffer_sync_sg, - .buffer_unmap_sg = hcd_buffer_unmap_sg, }; EXPORT_SYMBOL (usb_hcd_operations); diff -Nru a/drivers/usb/core/hcd.h b/drivers/usb/core/hcd.h --- a/drivers/usb/core/hcd.h Fri Dec 27 23:56:55 2002 +++ b/drivers/usb/core/hcd.h Fri Dec 27 23:56:55 2002 @@ -145,26 +145,6 @@ dma_addr_t *dma); void (*buffer_free)(struct usb_bus *bus, size_t size, void *addr, dma_addr_t dma); - - int (*buffer_map) (struct usb_bus *bus, - void *addr, dma_addr_t *dma, - size_t size, int direction); - void (*buffer_dmasync) (struct usb_bus *bus, - dma_addr_t dma, - size_t size, int direction); - void (*buffer_unmap) (struct usb_bus *bus, - dma_addr_t dma, - size_t size, int direction); - - int (*buffer_map_sg) (struct usb_bus *bus, - struct scatterlist *sg, int *n_hw_ents, - int nents, int direction); - void (*buffer_dmasync_sg) (struct usb_bus *bus, - struct scatterlist *sg, - int n_hw_ents, int direction); - void (*buffer_unmap_sg) (struct usb_bus *bus, - struct scatterlist *sg, - int n_hw_ents, int direction); }; /* each driver provides one of these, and hardware init support */ @@ -247,23 +227,6 @@ int mem_flags, dma_addr_t *dma); void hcd_buffer_free (struct usb_bus *bus, size_t size, void *addr, dma_addr_t dma); - -int hcd_buffer_map (struct usb_bus *bus, - void *addr, dma_addr_t *dma, - size_t size, int direction); -void hcd_buffer_dmasync (struct usb_bus *bus, - dma_addr_t dma, - size_t size, int direction); -void hcd_buffer_unmap (struct usb_bus *bus, - dma_addr_t dma, - size_t size, int direction); -int hcd_buffer_map_sg (struct usb_bus *bus, struct scatterlist *sg, - int *n_hw_ents, int nents, int direction); -void hcd_buffer_sync_sg (struct usb_bus *bus, struct scatterlist *sg, - int n_hw_ents, int direction); - -void hcd_buffer_unmap_sg (struct usb_bus *bus, struct scatterlist *sg, - int n_hw_ents, int direction); /* generic bus glue, needed for host controllers that don't use PCI */ extern struct usb_operations usb_hcd_operations; diff -Nru a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c --- a/drivers/usb/core/usb.c Fri Dec 27 23:56:55 2002 +++ b/drivers/usb/core/usb.c Fri Dec 27 23:56:55 2002 @@ -41,6 +41,11 @@ #endif #include +#include +#include +#include +#include + #include "hcd.h" #include "usb.h" @@ -803,7 +808,7 @@ *pdev = NULL; - info("USB disconnect on device %d", dev->devnum); + dev_info (dev->dev, "USB disconnect, address %d\n", dev->devnum); /* Free up all the children before we remove this device */ for (i = 0; i < USB_MAXCHILDREN; i++) { @@ -812,7 +817,7 @@ usb_disconnect(child); } - dbg ("unregistering interfaces on device %d", dev->devnum); + dev_dbg (dev->dev, "unregistering interfaces\n"); if (dev->actconfig) { for (i = 0; i < dev->actconfig->desc.bNumInterfaces; i++) { struct usb_interface *interface = &dev->actconfig->interface[i]; @@ -822,7 +827,7 @@ } } - dbg ("unregistering the device %d", dev->devnum); + dev_dbg (dev->dev, "unregistering device\n"); /* Free the device number and remove the /proc/bus/usb entry */ if (dev->devnum > 0) { clear_bit(dev->devnum, dev->bus->devmap.devicemap); @@ -980,6 +985,9 @@ sprintf (&dev->dev.bus_id[0], "%d-%s", dev->bus->busnum, dev->devpath); + /* dma masks come from the controller; readonly, except to hcd */ + dev->dev.dma_mask = parent->dma_mask; + /* USB device state == default ... it's not usable yet */ /* USB 2.0 section 5.5.3 talks about ep0 maxpacket ... @@ -1104,6 +1112,7 @@ interface->dev.parent = &dev->dev; interface->dev.driver = NULL; interface->dev.bus = &usb_bus_type; + interface->dev.dma_mask = parent->dma_mask; sprintf (&interface->dev.bus_id[0], "%d-%s:%d", dev->bus->busnum, dev->devpath, desc->bInterfaceNumber); @@ -1206,24 +1215,21 @@ struct urb *usb_buffer_map (struct urb *urb) { struct usb_bus *bus; - struct usb_operations *op; + struct device *controller; if (!urb || usb_pipecontrol (urb->pipe) || !urb->dev || !(bus = urb->dev->bus) - || !(op = bus->op) - || !op->buffer_map) + || !(controller = bus->controller)) return 0; - if (op->buffer_map (bus, - urb->transfer_buffer, - &urb->transfer_dma, - urb->transfer_buffer_length, + urb->transfer_dma = dma_map_single (controller, + urb->transfer_buffer, urb->transfer_buffer_length, usb_pipein (urb->pipe) - ? USB_DIR_IN - : USB_DIR_OUT)) - return 0; + ? DMA_FROM_DEVICE : DMA_TO_DEVICE); + // FIXME generic api broken like pci, can't report errors + // if (urb->transfer_dma == DMA_ADDR_INVALID) return 0; urb->transfer_flags |= URB_NO_DMA_MAP; return urb; } @@ -1235,22 +1241,19 @@ void usb_buffer_dmasync (struct urb *urb) { struct usb_bus *bus; - struct usb_operations *op; + struct device *controller; if (!urb || !(urb->transfer_flags & URB_NO_DMA_MAP) || !urb->dev || !(bus = urb->dev->bus) - || !(op = bus->op) - || !op->buffer_dmasync) + || !(controller = bus->controller)) return; - op->buffer_dmasync (bus, - urb->transfer_dma, - urb->transfer_buffer_length, + dma_sync_single (controller, + urb->transfer_dma, urb->transfer_buffer_length, usb_pipein (urb->pipe) - ? USB_DIR_IN - : USB_DIR_OUT); + ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } /** @@ -1262,23 +1265,21 @@ void usb_buffer_unmap (struct urb *urb) { struct usb_bus *bus; - struct usb_operations *op; + struct device *controller; if (!urb || !(urb->transfer_flags & URB_NO_DMA_MAP) || !urb->dev || !(bus = urb->dev->bus) - || !(op = bus->op) - || !op->buffer_unmap) + || !(controller = bus->controller)) return; - op->buffer_unmap (bus, - urb->transfer_dma, - urb->transfer_buffer_length, + dma_unmap_single (controller, + urb->transfer_dma, urb->transfer_buffer_length, usb_pipein (urb->pipe) - ? USB_DIR_IN - : USB_DIR_OUT); + ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } + /** * usb_buffer_map_sg - create scatterlist DMA mapping(s) for an endpoint * @dev: device to which the scatterlist will be mapped @@ -1297,6 +1298,7 @@ * to complete before starting the next I/O. This is particularly easy * to do with scatterlists. Just allocate and submit one URB for each DMA * mapping entry returned, stopping on the first error or when all succeed. + * Better yet, use the usb_sg_*() calls, which do that (and more) for you. * * This call would normally be used when translating scatterlist requests, * rather than usb_buffer_map(), since on some hardware (with IOMMUs) it @@ -1308,26 +1310,17 @@ struct scatterlist *sg, int nents) { struct usb_bus *bus; - struct usb_operations *op; - int n_hw_ents; + struct device *controller; if (!dev || usb_pipecontrol (pipe) || !(bus = dev->bus) - || !(op = bus->op) - || !op->buffer_map_sg) - return -1; - - if (op->buffer_map_sg (bus, - sg, - &n_hw_ents, - nents, - usb_pipein (pipe) - ? USB_DIR_IN - : USB_DIR_OUT)) + || !(controller = bus->controller)) return -1; - return n_hw_ents; + // FIXME generic api broken like pci, can't report errors + return dma_map_sg (controller, sg, nents, + usb_pipein (pipe) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } /** @@ -1344,20 +1337,15 @@ struct scatterlist *sg, int n_hw_ents) { struct usb_bus *bus; - struct usb_operations *op; + struct device *controller; if (!dev || !(bus = dev->bus) - || !(op = bus->op) - || !op->buffer_dmasync_sg) + || !(controller = bus->controller)) return; - op->buffer_dmasync_sg (bus, - sg, - n_hw_ents, - usb_pipein (pipe) - ? USB_DIR_IN - : USB_DIR_OUT); + dma_sync_sg (controller, sg, n_hw_ents, + usb_pipein (pipe) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } /** @@ -1373,20 +1361,15 @@ struct scatterlist *sg, int n_hw_ents) { struct usb_bus *bus; - struct usb_operations *op; + struct device *controller; if (!dev || !(bus = dev->bus) - || !(op = bus->op) - || !op->buffer_unmap_sg) + || !(controller = bus->controller)) return; - op->buffer_unmap_sg (bus, - sg, - n_hw_ents, - usb_pipein (pipe) - ? USB_DIR_IN - : USB_DIR_OUT); + dma_unmap_sg (controller, sg, n_hw_ents, + usb_pipein (pipe) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } diff -Nru a/drivers/usb/host/ehci-hcd.c b/drivers/usb/host/ehci-hcd.c --- a/drivers/usb/host/ehci-hcd.c Fri Dec 27 23:56:55 2002 +++ b/drivers/usb/host/ehci-hcd.c Fri Dec 27 23:56:55 2002 @@ -405,9 +405,10 @@ * streaming mappings for I/O buffers, like pci_map_single(), * can return segments above 4GB, if the device allows. * - * NOTE: layered drivers can't yet tell when we enable that, - * so they can't pass this info along (like NETIF_F_HIGHDMA) - * (or like Scsi_Host.highmem_io) ... usb_bus.flags? + * NOTE: the dma mask is visible through dma_supported(), so + * drivers can pass this info along ... like NETIF_F_HIGHDMA, + * Scsi_Host.highmem_io, and so forth. It's readonly to all + * host side drivers though. */ if (HCC_64BIT_ADDR (hcc_params)) { writel (0, &ehci->regs->segment); diff -Nru a/drivers/usb/host/ohci-sa1111.c b/drivers/usb/host/ohci-sa1111.c --- a/drivers/usb/host/ohci-sa1111.c Fri Dec 27 23:56:55 2002 +++ b/drivers/usb/host/ohci-sa1111.c Fri Dec 27 23:56:55 2002 @@ -176,7 +176,8 @@ hcd->irq = dev->irq[1]; hcd->regs = dev->mapbase; hcd->pdev = SA1111_FAKE_PCIDEV; - hcd->controller = &dev->dev; + hcd->self.controller = &dev->dev; + hcd->controller = hcd->self.controller; retval = hcd_buffer_create (hcd); if (retval != 0) { diff -Nru a/drivers/usb/net/usbnet.c b/drivers/usb/net/usbnet.c --- a/drivers/usb/net/usbnet.c Fri Dec 27 23:56:55 2002 +++ b/drivers/usb/net/usbnet.c Fri Dec 27 23:56:55 2002 @@ -146,6 +146,11 @@ #endif #include +#include +#include +#include +#include + /* minidrivers _could_ be individually configured */ #define CONFIG_USB_AN2720 @@ -2169,8 +2174,12 @@ memcpy (net->dev_addr, node_id, sizeof node_id); // point-to-point link ... we always use Ethernet headers - // supports win32 interop and the bridge driver. + // supports win32 interop (some devices) and the bridge driver. ether_setup (net); + + // possible with some EHCI controllers + if (dma_supported (&udev->dev, 0xffffffffffffffffULL)) + net->features |= NETIF_F_HIGHDMA; net->change_mtu = usbnet_change_mtu; net->get_stats = usbnet_get_stats; diff -Nru a/include/linux/usb.h b/include/linux/usb.h --- a/include/linux/usb.h Fri Dec 27 23:56:55 2002 +++ b/include/linux/usb.h Fri Dec 27 23:56:55 2002 @@ -28,14 +28,6 @@ mdelay(ms); } -/* - * USB device number allocation bitmap. There's one bitmap - * per USB tree. - */ -struct usb_devmap { - unsigned long devicemap[128 / (8*sizeof(unsigned long))]; -}; - struct usb_device; /*-------------------------------------------------------------------------*/ @@ -159,10 +151,16 @@ struct usb_operations; +/* USB device number allocation bitmap */ +struct usb_devmap { + unsigned long devicemap[128 / (8*sizeof(unsigned long))]; +}; + /* - * Allocated per bus we have + * Allocated per bus (tree of devices) we have: */ struct usb_bus { + struct device *controller; /* host/master side hardware */ int busnum; /* Bus number (in order of reg) */ char *bus_name; /* stable id (PCI slot_name etc) */