diff --git a/CREDITS b/CREDITS index 4fcf9cd8544cb3..bcb8efaa945903 100644 --- a/CREDITS +++ b/CREDITS @@ -20,6 +20,10 @@ D: One of assisting postmasters for vger.kernel.org's lists S: (ask for current address) S: Finland +N: Thomas Abraham +E: thomas.ab@samsung.com +D: Samsung pin controller driver + N: Dragos Acostachioaie E: dragos@iname.com W: http://www.arbornet.org/~dragos diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl index acfe9df83139a9..b07e86d4597f21 100644 --- a/Documentation/ABI/testing/sysfs-class-cxl +++ b/Documentation/ABI/testing/sysfs-class-cxl @@ -223,3 +223,13 @@ Description: write only Writing 1 will issue a PERST to card which may cause the card to reload the FPGA depending on load_image_on_perst. Users: https://github.com/ibm-capi/libcxl + +What: /sys/class/cxl//perst_reloads_same_image +Date: July 2015 +Contact: linuxppc-dev@lists.ozlabs.org +Description: read/write + Trust that when an image is reloaded via PERST, it will not + have changed. + 0 = don't trust, the image may be different (default) + 1 = trust that the image will not change. +Users: https://github.com/ibm-capi/libcxl diff --git a/Documentation/ABI/testing/sysfs-gpio b/Documentation/ABI/testing/sysfs-gpio index 80f4c94c7befdc..55ffa2df1c1022 100644 --- a/Documentation/ABI/testing/sysfs-gpio +++ b/Documentation/ABI/testing/sysfs-gpio @@ -16,7 +16,8 @@ Description: /sys/class/gpio /export ... asks the kernel to export a GPIO to userspace /unexport ... to return a GPIO to the kernel - /gpioN ... for each exported GPIO #N + /gpioN ... for each exported GPIO #N OR + / ... for a properly named GPIO line /value ... always readable, writes fail for input GPIOs /direction ... r/w as: in, out (default low); write: high, low /edge ... r/w as: none, falling, rising, both diff --git a/Documentation/DocBook/filesystems.tmpl b/Documentation/DocBook/filesystems.tmpl index bcdfdb9a927700..6006b6358c867f 100644 --- a/Documentation/DocBook/filesystems.tmpl +++ b/Documentation/DocBook/filesystems.tmpl @@ -146,36 +146,30 @@ The journalling layer is easy to use. You need to first of all create a journal_t data structure. There are two calls to do this dependent on how you decide to allocate the physical -media on which the journal resides. The journal_init_inode() call -is for journals stored in filesystem inodes, or the journal_init_dev() -call can be use for journal stored on a raw device (in a continuous range +media on which the journal resides. The jbd2_journal_init_inode() call +is for journals stored in filesystem inodes, or the jbd2_journal_init_dev() +call can be used for journal stored on a raw device (in a continuous range of blocks). A journal_t is a typedef for a struct pointer, so when -you are finally finished make sure you call journal_destroy() on it +you are finally finished make sure you call jbd2_journal_destroy() on it to free up any used kernel memory. Once you have got your journal_t object you need to 'mount' or load the journal -file, unless of course you haven't initialised it yet - in which case you -need to call journal_create(). +file. The journalling layer expects the space for the journal was already +allocated and initialized properly by the userspace tools. When loading the +journal you must call jbd2_journal_load() to process journal contents. If the +client file system detects the journal contents does not need to be processed +(or even need not have valid contents), it may call jbd2_journal_wipe() to +clear the journal contents before calling jbd2_journal_load(). -Most of the time however your journal file will already have been created, but -before you load it you must call journal_wipe() to empty the journal file. -Hang on, you say , what if the filesystem wasn't cleanly umount()'d . Well, it is the -job of the client file system to detect this and skip the call to journal_wipe(). - - - -In either case the next call should be to journal_load() which prepares the -journal file for use. Note that journal_wipe(..,0) calls journal_skip_recovery() -for you if it detects any outstanding transactions in the journal and similarly -journal_load() will call journal_recover() if necessary. -I would advise reading fs/ext3/super.c for examples on this stage. -[RGG: Why is the journal_wipe() call necessary - doesn't this needlessly -complicate the API. Or isn't a good idea for the journal layer to hide -dirty mounts from the client fs] +Note that jbd2_journal_wipe(..,0) calls jbd2_journal_skip_recovery() for you if +it detects any outstanding transactions in the journal and similarly +jbd2_journal_load() will call jbd2_journal_recover() if necessary. I would +advise reading ext4_load_journal() in fs/ext4/super.c for examples on this +stage. @@ -189,41 +183,41 @@ You still need to actually journal your filesystem changes, this is done by wrapping them into transactions. Additionally you also need to wrap the modification of each of the buffers with calls to the journal layer, so it knows what the modifications -you are actually making are. To do this use journal_start() which +you are actually making are. To do this use jbd2_journal_start() which returns a transaction handle. -journal_start() -and its counterpart journal_stop(), which indicates the end of a transaction -are nestable calls, so you can reenter a transaction if necessary, -but remember you must call journal_stop() the same number of times as -journal_start() before the transaction is completed (or more accurately -leaves the update phase). Ext3/VFS makes use of this feature to simplify -quota support. +jbd2_journal_start() +and its counterpart jbd2_journal_stop(), which indicates the end of a +transaction are nestable calls, so you can reenter a transaction if necessary, +but remember you must call jbd2_journal_stop() the same number of times as +jbd2_journal_start() before the transaction is completed (or more accurately +leaves the update phase). Ext4/VFS makes use of this feature to simplify +handling of inode dirtying, quota support, etc. Inside each transaction you need to wrap the modifications to the individual buffers (blocks). Before you start to modify a buffer you -need to call journal_get_{create,write,undo}_access() as appropriate, +need to call jbd2_journal_get_{create,write,undo}_access() as appropriate, this allows the journalling layer to copy the unmodified data if it needs to. After all the buffer may be part of a previously uncommitted transaction. At this point you are at last ready to modify a buffer, and once -you are have done so you need to call journal_dirty_{meta,}data(). +you are have done so you need to call jbd2_journal_dirty_{meta,}data(). Or if you've asked for access to a buffer you now know is now longer -required to be pushed back on the device you can call journal_forget() +required to be pushed back on the device you can call jbd2_journal_forget() in much the same way as you might have used bforget() in the past. -A journal_flush() may be called at any time to commit and checkpoint +A jbd2_journal_flush() may be called at any time to commit and checkpoint all your transactions. -Then at umount time , in your put_super() you can then call journal_destroy() +Then at umount time , in your put_super() you can then call jbd2_journal_destroy() to clean up your in-core journal object. @@ -231,82 +225,74 @@ to clean up your in-core journal object. Unfortunately there a couple of ways the journal layer can cause a deadlock. The first thing to note is that each task can only have a single outstanding transaction at any one time, remember nothing -commits until the outermost journal_stop(). This means +commits until the outermost jbd2_journal_stop(). This means you must complete the transaction at the end of each file/inode/address etc. operation you perform, so that the journalling system isn't re-entered on another journal. Since transactions can't be nested/batched across differing journals, and another filesystem other than -yours (say ext3) may be modified in a later syscall. +yours (say ext4) may be modified in a later syscall. -The second case to bear in mind is that journal_start() can +The second case to bear in mind is that jbd2_journal_start() can block if there isn't enough space in the journal for your transaction (based on the passed nblocks param) - when it blocks it merely(!) needs to wait for transactions to complete and be committed from other tasks, -so essentially we are waiting for journal_stop(). So to avoid -deadlocks you must treat journal_start/stop() as if they +so essentially we are waiting for jbd2_journal_stop(). So to avoid +deadlocks you must treat jbd2_journal_start/stop() as if they were semaphores and include them in your semaphore ordering rules to prevent -deadlocks. Note that journal_extend() has similar blocking behaviour to -journal_start() so you can deadlock here just as easily as on journal_start(). +deadlocks. Note that jbd2_journal_extend() has similar blocking behaviour to +jbd2_journal_start() so you can deadlock here just as easily as on +jbd2_journal_start(). Try to reserve the right number of blocks the first time. ;-). This will be the maximum number of blocks you are going to touch in this transaction. -I advise having a look at at least ext3_jbd.h to see the basis on which -ext3 uses to make these decisions. +I advise having a look at at least ext4_jbd.h to see the basis on which +ext4 uses to make these decisions. Another wriggle to watch out for is your on-disk block allocation strategy. -why? Because, if you undo a delete, you need to ensure you haven't reused any -of the freed blocks in a later transaction. One simple way of doing this -is make sure any blocks you allocate only have checkpointed transactions -listed against them. Ext3 does this in ext3_test_allocatable(). +Why? Because, if you do a delete, you need to ensure you haven't reused any +of the freed blocks until the transaction freeing these blocks commits. If you +reused these blocks and crash happens, there is no way to restore the contents +of the reallocated blocks at the end of the last fully committed transaction. + +One simple way of doing this is to mark blocks as free in internal in-memory +block allocation structures only after the transaction freeing them commits. +Ext4 uses journal commit callback for this purpose. + + + +With journal commit callbacks you can ask the journalling layer to call a +callback function when the transaction is finally committed to disk, so that +you can do some of your own management. You ask the journalling layer for +calling the callback by simply setting journal->j_commit_callback function +pointer and that function is called after each transaction commit. You can also +use transaction->t_private_list for attaching entries to a transaction that +need processing when the transaction commits. -Lock is also providing through journal_{un,}lock_updates(), -ext3 uses this when it wants a window with a clean and stable fs for a moment. -eg. +JBD2 also provides a way to block all transaction updates via +jbd2_journal_{un,}lock_updates(). Ext4 uses this when it wants a window with a +clean and stable fs for a moment. E.g. - journal_lock_updates() //stop new stuff happening.. - journal_flush() // checkpoint everything. + jbd2_journal_lock_updates() //stop new stuff happening.. + jbd2_journal_flush() // checkpoint everything. ..do stuff on stable fs - journal_unlock_updates() // carry on with filesystem use. + jbd2_journal_unlock_updates() // carry on with filesystem use. The opportunities for abuse and DOS attacks with this should be obvious, if you allow unprivileged userspace to trigger codepaths containing these calls. - - - -A new feature of jbd since 2.5.25 is commit callbacks with the new -journal_callback_set() function you can now ask the journalling layer -to call you back when the transaction is finally committed to disk, so that -you can do some of your own management. The key to this is the journal_callback -struct, this maintains the internal callback information but you can -extend it like this:- - - - struct myfs_callback_s { - //Data structure element required by jbd.. - struct journal_callback for_jbd; - // Stuff for myfs allocated together. - myfs_inode* i_commited; - - } - - - -this would be useful if you needed to know when data was committed to a -particular inode. @@ -319,36 +305,6 @@ being each mount, each modification (transaction) and each changed buffer to tell the journalling layer about them. - -Here is a some pseudo code to give you an idea of how it works, as -an example. - - - - journal_t* my_jnrl = journal_create(); - journal_init_{dev,inode}(jnrl,...) - if (clean) journal_wipe(); - journal_load(); - - foreach(transaction) { /*transactions must be - completed before - a syscall returns to - userspace*/ - - handle_t * xct=journal_start(my_jnrl); - foreach(bh) { - journal_get_{create,write,undo}_access(xact,bh); - if ( myfs_modify(bh) ) { /* returns true - if makes changes */ - journal_dirty_{meta,}data(xact,bh); - } else { - journal_forget(bh); - } - } - journal_stop(xct); - } - journal_destroy(my_jrnl); - @@ -357,13 +313,13 @@ an example. Data Types The journalling layer uses typedefs to 'hide' the concrete definitions - of the structures used. As a client of the JBD layer you can + of the structures used. As a client of the JBD2 layer you can just rely on the using the pointer as a magic cookie of some sort. Obviously the hiding is not enforced as this is 'C'. Structures -!Iinclude/linux/jbd.h +!Iinclude/linux/jbd2.h @@ -375,11 +331,11 @@ an example. manage transactions Journal Level -!Efs/jbd/journal.c -!Ifs/jbd/recovery.c +!Efs/jbd2/journal.c +!Ifs/jbd2/recovery.c Transasction Level -!Efs/jbd/transaction.c +!Efs/jbd2/transaction.c diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt index 1690350f16e77b..7d9d3c2286b282 100644 --- a/Documentation/arm64/booting.txt +++ b/Documentation/arm64/booting.txt @@ -81,7 +81,7 @@ The decompressed kernel image contains a 64-byte header as follows: u64 res3 = 0; /* reserved */ u64 res4 = 0; /* reserved */ u32 magic = 0x644d5241; /* Magic number, little endian, "ARM\x64" */ - u32 res5; /* reserved (used for PE COFF offset) */ + u32 res5; /* reserved (used for PE COFF offset) */ Header notes: @@ -103,7 +103,7 @@ Header notes: - The flags field (introduced in v3.17) is a little-endian 64-bit field composed as follows: - Bit 0: Kernel endianness. 1 if BE, 0 if LE. + Bit 0: Kernel endianness. 1 if BE, 0 if LE. Bits 1-63: Reserved. - When image_size is zero, a bootloader should attempt to keep as much @@ -115,11 +115,14 @@ The Image must be placed text_offset bytes from a 2MB aligned base address near the start of usable system RAM and called there. Memory below that base address is currently unusable by Linux, and therefore it is strongly recommended that this location is the start of system RAM. +The region between the 2 MB aligned base address and the start of the +image has no special significance to the kernel, and may be used for +other purposes. At least image_size bytes from the start of the image must be free for use by the kernel. -Any memory described to the kernel (even that below the 2MB aligned base -address) which is not marked as reserved from the kernel e.g. with a +Any memory described to the kernel (even that below the start of the +image) which is not marked as reserved from the kernel (e.g., with a memreserve region in the device tree) will be considered as available to the kernel. diff --git a/Documentation/atomic_ops.txt b/Documentation/atomic_ops.txt index dab6da3382d9f0..b19fc34efdb179 100644 --- a/Documentation/atomic_ops.txt +++ b/Documentation/atomic_ops.txt @@ -266,7 +266,9 @@ with the given old and new values. Like all atomic_xxx operations, atomic_cmpxchg will only satisfy its atomicity semantics as long as all other accesses of *v are performed through atomic_xxx operations. -atomic_cmpxchg must provide explicit memory barriers around the operation. +atomic_cmpxchg must provide explicit memory barriers around the operation, +although if the comparison fails then no memory ordering guarantees are +required. The semantics for atomic_cmpxchg are the same as those defined for 'cas' below. diff --git a/Documentation/devicetree/bindings/arm/l2cc.txt b/Documentation/devicetree/bindings/arm/l2cc.txt index 2251dccb141eac..06c88a4d28aced 100644 --- a/Documentation/devicetree/bindings/arm/l2cc.txt +++ b/Documentation/devicetree/bindings/arm/l2cc.txt @@ -67,6 +67,12 @@ Optional properties: disable if zero. - arm,prefetch-offset : Override prefetch offset value. Valid values are 0-7, 15, 23, and 31. +- arm,shared-override : The default behavior of the pl310 cache controller with + respect to the shareable attribute is to transform "normal memory + non-cacheable transactions" into "cacheable no allocate" (for reads) or + "write through no write allocate" (for writes). + On systems where this may cause DMA buffer corruption, this property must be + specified to indicate that such transforms are precluded. - prefetch-data : Data prefetch. Value: <0> (forcibly disable), <1> (forcibly enable), property absent (retain settings set by firmware) - prefetch-instr : Instruction prefetch. Value: <0> (forcibly disable), diff --git a/Documentation/devicetree/bindings/arm/pmu.txt b/Documentation/devicetree/bindings/arm/pmu.txt index 3b5f5d1088c639..435251fa9ce0d4 100644 --- a/Documentation/devicetree/bindings/arm/pmu.txt +++ b/Documentation/devicetree/bindings/arm/pmu.txt @@ -26,13 +26,19 @@ Required properties: Optional properties: -- interrupt-affinity : Valid only when using SPIs, specifies a list of phandles - to CPU nodes corresponding directly to the affinity of +- interrupt-affinity : When using SPIs, specifies a list of phandles to CPU + nodes corresponding directly to the affinity of the SPIs listed in the interrupts property. - This property should be present when there is more than + When using a PPI, specifies a list of phandles to CPU + nodes corresponding to the set of CPUs which have + a PMU of this type signalling the PPI listed in the + interrupts property. + + This property should be present when there is more than a single SPI. + - qcom,no-pc-write : Indicates that this PMU doesn't support the 0xc and 0xd events. diff --git a/Documentation/devicetree/bindings/gpio/brcm,brcmstb-gpio.txt b/Documentation/devicetree/bindings/gpio/brcm,brcmstb-gpio.txt index 435f1bcca3415d..b405b4410bfb89 100644 --- a/Documentation/devicetree/bindings/gpio/brcm,brcmstb-gpio.txt +++ b/Documentation/devicetree/bindings/gpio/brcm,brcmstb-gpio.txt @@ -33,6 +33,13 @@ Optional properties: - interrupt-parent: phandle of the parent interrupt controller +- interrupts-extended: + Alternate form of specifying interrupts and parents that allows for + multiple parents. This takes precedence over 'interrupts' and + 'interrupt-parent'. Wakeup-capable GPIO controllers often route their + wakeup interrupt lines through a different interrupt controller than the + primary interrupt line, making this property necessary. + - #interrupt-cells: Should be <2>. The first cell is the GPIO number, the second should specify flags. The following subset of flags is supported: @@ -47,19 +54,33 @@ Optional properties: - interrupt-controller: Marks the device node as an interrupt controller -- interrupt-names: - The name of the IRQ resource used by this controller +- wakeup-source: + GPIOs for this controller can be used as a wakeup source Example: upg_gio: gpio@f040a700 { - #gpio-cells = <0x2>; - #interrupt-cells = <0x2>; + #gpio-cells = <2>; + #interrupt-cells = <2>; compatible = "brcm,bcm7445-gpio", "brcm,brcmstb-gpio"; gpio-controller; interrupt-controller; reg = <0xf040a700 0x80>; - interrupt-parent = <0xf>; + interrupt-parent = <&irq0_intc>; + interrupts = <0x6>; + brcm,gpio-bank-widths = <32 32 32 24>; + }; + + upg_gio_aon: gpio@f04172c0 { + #gpio-cells = <2>; + #interrupt-cells = <2>; + compatible = "brcm,bcm7445-gpio", "brcm,brcmstb-gpio"; + gpio-controller; + interrupt-controller; + reg = <0xf04172c0 0x40>; + interrupt-parent = <&irq0_aon_intc>; interrupts = <0x6>; - interrupt-names = "upg_gio"; - brcm,gpio-bank-widths = <0x20 0x20 0x20 0x18>; + interrupts-extended = <&irq0_aon_intc 0x6>, + <&aon_pm_l2_intc 0x5>; + wakeup-source; + brcm,gpio-bank-widths = <18 4>; }; diff --git a/Documentation/devicetree/bindings/gpio/gpio-etraxfs.txt b/Documentation/devicetree/bindings/gpio/gpio-etraxfs.txt index abf4db736c6e3f..170194af302700 100644 --- a/Documentation/devicetree/bindings/gpio/gpio-etraxfs.txt +++ b/Documentation/devicetree/bindings/gpio/gpio-etraxfs.txt @@ -2,8 +2,9 @@ Axis ETRAX FS General I/O controller bindings Required properties: -- compatible: +- compatible: one of: - "axis,etraxfs-gio" + - "axis,artpec3-gio" - reg: Physical base address and length of the controller's registers. - #gpio-cells: Should be 3 - The first cell is the gpio offset number. diff --git a/Documentation/devicetree/bindings/gpio/gpio-mpc8xxx.txt b/Documentation/devicetree/bindings/gpio/gpio-mpc8xxx.txt new file mode 100644 index 00000000000000..805ddcd79a572d --- /dev/null +++ b/Documentation/devicetree/bindings/gpio/gpio-mpc8xxx.txt @@ -0,0 +1,22 @@ +* Freescale MPC512x/MPC8xxx GPIO controller + +Required properties: +- compatible : Should be "fsl,-gpio" + The following s are known to be supported: + mpc5121, mpc5125, mpc8349, mpc8572, mpc8610, pq3, qoriq +- reg : Address and length of the register set for the device +- interrupts : Should be the port interrupt shared by all 32 pins. +- #gpio-cells : Should be two. The first cell is the pin number and + the second cell is used to specify the gpio polarity: + 0 = active high + 1 = active low + +Example: + +gpio0: gpio@1100 { + compatible = "fsl,mpc5125-gpio"; + #gpio-cells = <2>; + reg = <0x1100 0x080>; + interrupts = <78 0x8>; + status = "okay"; +}; diff --git a/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt b/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt index 38fb86f28ba2ce..f60e2f477e93ca 100644 --- a/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt +++ b/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt @@ -9,6 +9,7 @@ Required Properties: - "renesas,gpio-r8a7791": for R8A7791 (R-Car M2-W) compatible GPIO controller. - "renesas,gpio-r8a7793": for R8A7793 (R-Car M2-N) compatible GPIO controller. - "renesas,gpio-r8a7794": for R8A7794 (R-Car E2) compatible GPIO controller. + - "renesas,gpio-r8a7795": for R8A7795 (R-Car H3) compatible GPIO controller. - "renesas,gpio-rcar": for generic R-Car GPIO controller. - reg: Base address and length of each memory resource used by the GPIO diff --git a/Documentation/devicetree/bindings/gpio/zx296702-gpio.txt b/Documentation/devicetree/bindings/gpio/zx296702-gpio.txt new file mode 100644 index 00000000000000..0dab156fcf4177 --- /dev/null +++ b/Documentation/devicetree/bindings/gpio/zx296702-gpio.txt @@ -0,0 +1,24 @@ +ZTE ZX296702 GPIO controller + +Required properties: +- compatible : "zte,zx296702-gpio" +- #gpio-cells : Should be two. The first cell is the pin number and the + second cell is used to specify optional parameters: + - bit 0 specifies polarity (0 for normal, 1 for inverted) +- gpio-controller : Marks the device node as a GPIO controller. +- interrupts : Interrupt mapping for GPIO IRQ. +- gpio-ranges : Interaction with the PINCTRL subsystem. + +gpio1: gpio@b008040 { + compatible = "zte,zx296702-gpio"; + reg = <0xb008040 0x40>; + gpio-controller; + #gpio-cells = <2>; + gpio-ranges = < &pmx0 0 54 2 &pmx0 2 59 14>; + interrupts = ; + interrupt-parent = <&intc>; + interrupt-controller; + #interrupt-cells = <2>; + clock-names = "gpio_pclk"; + clocks = <&lsp0clk ZX296702_GPIO_CLK>; +}; diff --git a/Documentation/devicetree/bindings/leds/leds-powernv.txt b/Documentation/devicetree/bindings/leds/leds-powernv.txt new file mode 100644 index 00000000000000..66655690f749ac --- /dev/null +++ b/Documentation/devicetree/bindings/leds/leds-powernv.txt @@ -0,0 +1,26 @@ +Device Tree binding for LEDs on IBM Power Systems +------------------------------------------------- + +Required properties: +- compatible : Should be "ibm,opal-v3-led". +- led-mode : Should be "lightpath" or "guidinglight". + +Each location code of FRU/Enclosure must be expressed in the +form of a sub-node. + +Required properties for the sub nodes: +- led-types : Supported LED types (attention/identify/fault) provided + in the form of string array. + +Example: + +leds { + compatible = "ibm,opal-v3-led"; + led-mode = "lightpath"; + + U78C9.001.RST0027-P1-C1 { + led-types = "identify", "fault"; + }; + ... + ... +}; diff --git a/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt b/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt index d5e370450ac0b2..89427b018ba750 100644 --- a/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt +++ b/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt @@ -18,6 +18,8 @@ Properties: interrupt (NAND_EVTER_STAT). If there is only one, that interrupt reports both types of event. +- little-endian : If this property is absent, the big-endian mode will + be in use as default for registers. - ranges : Each range corresponds to a single chipselect, and covers the entire access window as configured. @@ -34,6 +36,7 @@ Example: #size-cells = <1>; reg = <0x0 0xffe1e000 0 0x2000>; interrupts = <16 2 19 2>; + little-endian; /* NOR, NAND Flashes and CPLD on board */ ranges = <0x0 0x0 0x0 0xee000000 0x02000000 diff --git a/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt b/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt index 9462ab7ddd1f5a..3c821cda1ad059 100644 --- a/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt +++ b/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt @@ -48,7 +48,7 @@ Optional subnode-properties: Examples: -pinctrl@01c20800 { +pio: pinctrl@01c20800 { compatible = "allwinner,sun5i-a13-pinctrl"; reg = <0x01c20800 0x400>; #address-cells = <1>; @@ -68,3 +68,38 @@ pinctrl@01c20800 { allwinner,pull = <0>; }; }; + + +GPIO and interrupt controller +----------------------------- + +This hardware also acts as a GPIO controller and an interrupt +controller. + +Consumers that would want to refer to one or the other (or both) +should provide through the usual *-gpios and interrupts properties a +cell with 3 arguments, first the number of the bank, then the pin +inside that bank, and finally the flags for the GPIO/interrupts. + +Example: + +xio: gpio@38 { + compatible = "nxp,pcf8574a"; + reg = <0x38>; + + gpio-controller; + #gpio-cells = <2>; + + interrupt-parent = <&pio>; + interrupts = <6 0 IRQ_TYPE_EDGE_FALLING>; + interrupt-controller; + #interrupt-cells = <2>; +}; + +reg_usb1_vbus: usb1-vbus { + compatible = "regulator-fixed"; + regulator-name = "usb1-vbus"; + regulator-min-microvolt = <5000000>; + regulator-max-microvolt = <5000000>; + gpio = <&pio 7 6 GPIO_ACTIVE_HIGH>; +}; diff --git a/Documentation/devicetree/bindings/pinctrl/cnxt,cx92755-pinctrl.txt b/Documentation/devicetree/bindings/pinctrl/cnxt,cx92755-pinctrl.txt new file mode 100644 index 00000000000000..23ce8dc26990b6 --- /dev/null +++ b/Documentation/devicetree/bindings/pinctrl/cnxt,cx92755-pinctrl.txt @@ -0,0 +1,86 @@ +Conexant Digicolor CX92755 General Purpose Pin Mapping + +This document describes the device tree binding of the pin mapping hardware +modules in the Conexant Digicolor CX92755 SoCs. The CX92755 in one of the +Digicolor series of SoCs. + +=== Pin Controller Node === + +Required Properties: + +- compatible: Must be "cnxt,cx92755-pinctrl" +- reg: Base address of the General Purpose Pin Mapping register block and the + size of the block. +- gpio-controller: Marks the device node as a GPIO controller. +- #gpio-cells: Must be <2>. The first cell is the pin number and the + second cell is used to specify flags. See include/dt-bindings/gpio/gpio.h + for possible values. + +For example, the following is the bare minimum node: + + pinctrl: pinctrl@f0000e20 { + compatible = "cnxt,cx92755-pinctrl"; + reg = <0xf0000e20 0x100>; + gpio-controller; + #gpio-cells = <2>; + }; + +As a pin controller device, in addition to the required properties, this node +should also contain the pin configuration nodes that client devices reference, +if any. + +For a general description of GPIO bindings, please refer to ../gpio/gpio.txt. + +=== Pin Configuration Node === + +Each pin configuration node is a sub-node of the pin controller node and is a +container of an arbitrary number of subnodes, called pin group nodes in this +document. + +Please refer to the pinctrl-bindings.txt in this directory for details of the +common pinctrl bindings used by client devices, including the definition of a +"pin configuration node". + +=== Pin Group Node === + +A pin group node specifies the desired pin mux for an arbitrary number of +pins. The name of the pin group node is optional and not used. + +A pin group node only affects the properties specified in the node, and has no +effect on any properties that are omitted. + +The pin group node accepts a subset of the generic pin config properties. For +details generic pin config properties, please refer to pinctrl-bindings.txt +and . + +Required Pin Group Node Properties: + +- pins: Multiple strings. Specifies the name(s) of one or more pins to be + configured by this node. The format of a pin name string is "GP_xy", where x + is an uppercase character from 'A' to 'R', and y is a digit from 0 to 7. +- function: String. Specifies the pin mux selection. Values must be one of: + "gpio", "client_a", "client_b", "client_c" + +Example: + pinctrl: pinctrl@f0000e20 { + compatible = "cnxt,cx92755-pinctrl"; + reg = <0xf0000e20 0x100>; + + uart0_default: uart0_active { + data_signals { + pins = "GP_O0", "GP_O1"; + function = "client_b"; + }; + }; + }; + + uart0: uart@f0000740 { + compatible = "cnxt,cx92755-usart"; + ... + pinctrl-0 = <&uart0_default>; + pinctrl-names = "default"; + }; + +In the example above, a single pin group configuration node defines the +"client select" for the Rx and Tx signals of uart0. The uart0 node references +that pin configuration node using the &uart0_default phandle. diff --git a/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt b/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt index ed19991aad35e2..d7803a2a94e9be 100644 --- a/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt +++ b/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt @@ -7,8 +7,13 @@ of PMIC's from Qualcomm. Usage: required Value type: Definition: Should contain one of: + "qcom,pm8018-mpp", + "qcom,pm8038-mpp", + "qcom,pm8821-mpp", "qcom,pm8841-mpp", "qcom,pm8916-mpp", + "qcom,pm8917-mpp", + "qcom,pm8921-mpp", "qcom,pm8941-mpp", "qcom,pma8084-mpp", @@ -77,12 +82,9 @@ to specify in a pin configuration subnode: Value type: Definition: Specify the alternative function to be configured for the specified pins. Valid values are: - "normal", - "paired", - "dtest1", - "dtest2", - "dtest3", - "dtest4" + "digital", + "analog", + "sink" - bias-disable: Usage: optional @@ -127,12 +129,18 @@ to specify in a pin configuration subnode: Definition: Selects the power source for the specified pins. Valid power sources are defined in -- qcom,analog-mode: +- qcom,analog-level: Usage: optional - Value type: - Definition: Selects Analog mode of operation: combined with input-enable - and/or output-high, output-low MPP could operate as - Bidirectional Logic, Analog Input, Analog Output. + Value type: + Definition: Selects the source for analog output. Valued values are + defined in + PMIC_MPP_AOUT_LVL_* + +- qcom,dtest: + Usage: optional + Value type: + Definition: Selects which dtest rail to be routed in the various functions. + Valid values are 1-4 - qcom,amux-route: Usage: optional @@ -140,6 +148,10 @@ to specify in a pin configuration subnode: Definition: Selects the source for analog input. Valid values are defined in PMIC_MPP_AMUX_ROUTE_CH5, PMIC_MPP_AMUX_ROUTE_CH6... +- qcom,paired: + Usage: optional + Value type: + Definition: Indicates that the pin should be operating in paired mode. Example: @@ -156,7 +168,7 @@ Example: pm8841_default: default { gpio { pins = "mpp1", "mpp2", "mpp3", "mpp4"; - function = "normal"; + function = "digital"; input-enable; power-source = ; }; diff --git a/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt b/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt index 51cee44fc1406a..9496934528bdef 100644 --- a/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt +++ b/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt @@ -58,12 +58,12 @@ are parsed through phandles and processed purely based on their content. Pin Configuration Node Properties: -- renesas,pins : An array of strings, each string containing the name of a pin. -- renesas,groups : An array of strings, each string containing the name of a pin +- pins : An array of strings, each string containing the name of a pin. +- groups : An array of strings, each string containing the name of a pin group. -- renesas,function: A string containing the name of the function to mux to the - pin group(s) specified by the renesas,groups property +- function: A string containing the name of the function to mux to the pin + group(s) specified by the groups property. Valid values for pin, group and function names can be found in the group and function arrays of the PFC data file corresponding to the SoC @@ -71,7 +71,9 @@ Pin Configuration Node Properties: The pin configuration parameters use the generic pinconf bindings defined in pinctrl-bindings.txt in this directory. The supported parameters are -bias-disable, bias-pull-up and bias-pull-down. +bias-disable, bias-pull-up, bias-pull-down and power-source. For pins that +have a configurable I/O voltage, the power-source value should be the +nominal I/O voltage in millivolts. GPIO @@ -141,19 +143,19 @@ Example 3: KZM-A9-GT (SH-Mobile AG5) default pin state hog and pin control maps mmcif_pins: mmcif { mux { - renesas,groups = "mmc0_data8_0", "mmc0_ctrl_0"; - renesas,function = "mmc0"; + groups = "mmc0_data8_0", "mmc0_ctrl_0"; + function = "mmc0"; }; cfg { - renesas,groups = "mmc0_data8_0"; - renesas,pins = "PORT279"; + groups = "mmc0_data8_0"; + pins = "PORT279"; bias-pull-up; }; }; scifa4_pins: scifa4 { - renesas,groups = "scifa4_data", "scifa4_ctrl"; - renesas,function = "scifa4"; + groups = "scifa4_data", "scifa4_ctrl"; + function = "scifa4"; }; }; diff --git a/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt b/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt index f63fcb3ed35288..2213802435e00d 100644 --- a/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt +++ b/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt @@ -3,7 +3,9 @@ ST Ericsson Nomadik pinmux controller Required properties: - compatible: "stericsson,db8500-pinctrl", "stericsson,db8540-pinctrl", "stericsson,stn8815-pinctrl" -- reg: Should contain the register physical address and length of the PRCMU. +- nomadik-gpio-chips: array of phandles to the corresponding GPIO chips + (these have the register ranges used by the pin controller). +- prcm: phandle to the PRCMU managing the back end of this pin controller Please refer to pinctrl-bindings.txt in this directory for details of the common pinctrl bindings used by client devices, including the meaning of the @@ -74,7 +76,8 @@ Example board file extract: pinctrl@80157000 { compatible = "stericsson,db8500-pinctrl"; - reg = <0x80157000 0x2000>; + nomadik-gpio-chips = <&gpio0>, <&gpio1>, <&gpio2>, <&gpio3>; + prcm = <&prcmu>; pinctrl-names = "default"; diff --git a/Documentation/devicetree/bindings/powerpc/fsl/scfg.txt b/Documentation/devicetree/bindings/powerpc/fsl/scfg.txt new file mode 100644 index 00000000000000..0532c46b3372a6 --- /dev/null +++ b/Documentation/devicetree/bindings/powerpc/fsl/scfg.txt @@ -0,0 +1,18 @@ +Freescale Supplement configuration unit (SCFG) + +SCFG is the supplemental configuration unit, that provides SoC specific +configuration and status registers for the chip. Such as getting PEX port +status. + +Required properties: + +- compatible: should be "fsl,-scfg" +- reg: should contain base address and length of SCFG memory-mapped +registers + +Example: + + scfg: global-utilities@fc000 { + compatible = "fsl,t1040-scfg"; + reg = <0xfc000 0x1000>; + }; diff --git a/Documentation/devicetree/bindings/timer/img,pistachio-gptimer.txt b/Documentation/devicetree/bindings/timer/img,pistachio-gptimer.txt new file mode 100644 index 00000000000000..7afce80bf6a016 --- /dev/null +++ b/Documentation/devicetree/bindings/timer/img,pistachio-gptimer.txt @@ -0,0 +1,28 @@ +* Pistachio general-purpose timer based clocksource + +Required properties: + - compatible: "img,pistachio-gptimer". + - reg: Address range of the timer registers. + - interrupts: An interrupt for each of the four timers + - clocks: Should contain a clock specifier for each entry in clock-names + - clock-names: Should contain the following entries: + "sys", interface clock + "slow", slow counter clock + "fast", fast counter clock + - img,cr-periph: Must contain a phandle to the peripheral control + syscon node. + +Example: + timer: timer@18102000 { + compatible = "img,pistachio-gptimer"; + reg = <0x18102000 0x100>; + interrupts = , + , + , + ; + clocks = <&clk_periph PERIPH_CLK_COUNTER_FAST>, + <&clk_periph PERIPH_CLK_COUNTER_SLOW>, + <&cr_periph SYS_CLK_TIMER>; + clock-names = "fast", "slow", "sys"; + img,cr-periph = <&cr_periph>; + }; diff --git a/Documentation/fault-injection/fault-injection.txt b/Documentation/fault-injection/fault-injection.txt index 4cf1a2a6bd7257..415484f3d59a24 100644 --- a/Documentation/fault-injection/fault-injection.txt +++ b/Documentation/fault-injection/fault-injection.txt @@ -15,6 +15,10 @@ o fail_page_alloc injects page allocation failures. (alloc_pages(), get_free_pages(), ...) +o fail_futex + + injects futex deadlock and uaddr fault errors. + o fail_make_request injects disk IO errors on devices permitted by setting @@ -113,6 +117,12 @@ configuration of fault-injection capabilities. specifies the minimum page allocation order to be injected failures. +- /sys/kernel/debug/fail_futex/ignore-private: + + Format: { 'Y' | 'N' } + default is 'N', setting it to 'Y' will disable failure injections + when dealing with private (address space) futexes. + o Boot option In order to inject faults while debugfs is not available (early boot time), @@ -121,6 +131,7 @@ use the boot option: failslab= fail_page_alloc= fail_make_request= + fail_futex= mmc_core.fail_request=,,, How to add new fault injection capability diff --git a/Documentation/features/debug/uprobes/arch-support.txt b/Documentation/features/debug/uprobes/arch-support.txt index 4efe36c3ace93b..d605c3fc38fd5c 100644 --- a/Documentation/features/debug/uprobes/arch-support.txt +++ b/Documentation/features/debug/uprobes/arch-support.txt @@ -22,7 +22,7 @@ | m68k: | TODO | | metag: | TODO | | microblaze: | TODO | - | mips: | TODO | + | mips: | ok | | mn10300: | TODO | | nios2: | TODO | | openrisc: | TODO | diff --git a/Documentation/features/seccomp/seccomp-filter/arch-support.txt b/Documentation/features/seccomp/seccomp-filter/arch-support.txt index bea800910342a1..76d39d66a5d754 100644 --- a/Documentation/features/seccomp/seccomp-filter/arch-support.txt +++ b/Documentation/features/seccomp/seccomp-filter/arch-support.txt @@ -32,7 +32,7 @@ | score: | TODO | | sh: | TODO | | sparc: | TODO | - | tile: | TODO | + | tile: | ok | | um: | TODO | | unicore32: | TODO | | x86: | ok | diff --git a/Documentation/filesystems/ext2.txt b/Documentation/filesystems/ext2.txt index b9714569e472b7..55755395d3dc50 100644 --- a/Documentation/filesystems/ext2.txt +++ b/Documentation/filesystems/ext2.txt @@ -360,8 +360,8 @@ and are copied into the filesystem. If a transaction is incomplete at the time of the crash, then there is no guarantee of consistency for the blocks in that transaction so they are discarded (which means any filesystem changes they represent are also lost). -Check Documentation/filesystems/ext3.txt if you want to read more about -ext3 and journaling. +Check Documentation/filesystems/ext4.txt if you want to read more about +ext4 and journaling. References ========== diff --git a/Documentation/filesystems/ext3.txt b/Documentation/filesystems/ext3.txt index 7ed0d17d672185..58758fbef9e0da 100644 --- a/Documentation/filesystems/ext3.txt +++ b/Documentation/filesystems/ext3.txt @@ -6,210 +6,7 @@ Ext3 was originally released in September 1999. Written by Stephen Tweedie for the 2.2 branch, and ported to 2.4 kernels by Peter Braam, Andreas Dilger, Andrew Morton, Alexander Viro, Ted Ts'o and Stephen Tweedie. -Ext3 is the ext2 filesystem enhanced with journalling capabilities. +Ext3 is the ext2 filesystem enhanced with journalling capabilities. The +filesystem is a subset of ext4 filesystem so use ext4 driver for accessing +ext3 filesystems. -Options -======= - -When mounting an ext3 filesystem, the following option are accepted: -(*) == default - -ro Mount filesystem read only. Note that ext3 will replay - the journal (and thus write to the partition) even when - mounted "read only". Mount options "ro,noload" can be - used to prevent writes to the filesystem. - -journal=update Update the ext3 file system's journal to the current - format. - -journal=inum When a journal already exists, this option is ignored. - Otherwise, it specifies the number of the inode which - will represent the ext3 file system's journal file. - -journal_path=path -journal_dev=devnum When the external journal device's major/minor numbers - have changed, these options allow the user to specify - the new journal location. The journal device is - identified through either its new major/minor numbers - encoded in devnum, or via a path to the device. - -norecovery Don't load the journal on mounting. Note that this forces -noload mount of inconsistent filesystem, which can lead to - various problems. - -data=journal All data are committed into the journal prior to being - written into the main file system. - -data=ordered (*) All data are forced directly out to the main file - system prior to its metadata being committed to the - journal. - -data=writeback Data ordering is not preserved, data may be written - into the main file system after its metadata has been - committed to the journal. - -commit=nrsec (*) Ext3 can be told to sync all its data and metadata - every 'nrsec' seconds. The default value is 5 seconds. - This means that if you lose your power, you will lose - as much as the latest 5 seconds of work (your - filesystem will not be damaged though, thanks to the - journaling). This default value (or any low value) - will hurt performance, but it's good for data-safety. - Setting it to 0 will have the same effect as leaving - it at the default (5 seconds). - Setting it to very large values will improve - performance. - -barrier=<0|1(*)> This enables/disables the use of write barriers in -barrier (*) the jbd code. barrier=0 disables, barrier=1 enables. -nobarrier This also requires an IO stack which can support - barriers, and if jbd gets an error on a barrier - write, it will disable again with a warning. - Write barriers enforce proper on-disk ordering - of journal commits, making volatile disk write caches - safe to use, at some performance penalty. If - your disks are battery-backed in one way or another, - disabling barriers may safely improve performance. - The mount options "barrier" and "nobarrier" can - also be used to enable or disable barriers, for - consistency with other ext3 mount options. - -user_xattr Enables Extended User Attributes. Additionally, you - need to have extended attribute support enabled in the - kernel configuration (CONFIG_EXT3_FS_XATTR). See the - attr(5) manual page and http://acl.bestbits.at/ to - learn more about extended attributes. - -nouser_xattr Disables Extended User Attributes. - -acl Enables POSIX Access Control Lists support. - Additionally, you need to have ACL support enabled in - the kernel configuration (CONFIG_EXT3_FS_POSIX_ACL). - See the acl(5) manual page and http://acl.bestbits.at/ - for more information. - -noacl This option disables POSIX Access Control List - support. - -reservation - -noreservation - -bsddf (*) Make 'df' act like BSD. -minixdf Make 'df' act like Minix. - -check=none Don't do extra checking of bitmaps on mount. -nocheck - -debug Extra debugging information is sent to syslog. - -errors=remount-ro Remount the filesystem read-only on an error. -errors=continue Keep going on a filesystem error. -errors=panic Panic and halt the machine if an error occurs. - (These mount options override the errors behavior - specified in the superblock, which can be - configured using tune2fs.) - -data_err=ignore(*) Just print an error message if an error occurs - in a file data buffer in ordered mode. -data_err=abort Abort the journal if an error occurs in a file - data buffer in ordered mode. - -grpid Give objects the same group ID as their creator. -bsdgroups - -nogrpid (*) New objects have the group ID of their creator. -sysvgroups - -resgid=n The group ID which may use the reserved blocks. - -resuid=n The user ID which may use the reserved blocks. - -sb=n Use alternate superblock at this location. - -quota These options are ignored by the filesystem. They -noquota are used only by quota tools to recognize volumes -grpquota where quota should be turned on. See documentation -usrquota in the quota-tools package for more details - (http://sourceforge.net/projects/linuxquota). - -jqfmt= These options tell filesystem details about quota -usrjquota= so that quota information can be properly updated -grpjquota= during journal replay. They replace the above - quota options. See documentation in the quota-tools - package for more details - (http://sourceforge.net/projects/linuxquota). - -Specification -============= -Ext3 shares all disk implementation with the ext2 filesystem, and adds -transactions capabilities to ext2. Journaling is done by the Journaling Block -Device layer. - -Journaling Block Device layer ------------------------------ -The Journaling Block Device layer (JBD) isn't ext3 specific. It was designed -to add journaling capabilities to a block device. The ext3 filesystem code -will inform the JBD of modifications it is performing (called a transaction). -The journal supports the transactions start and stop, and in case of a crash, -the journal can replay the transactions to quickly put the partition back into -a consistent state. - -Handles represent a single atomic update to a filesystem. JBD can handle an -external journal on a block device. - -Data Mode ---------- -There are 3 different data modes: - -* writeback mode -In data=writeback mode, ext3 does not journal data at all. This mode provides -a similar level of journaling as that of XFS, JFS, and ReiserFS in its default -mode - metadata journaling. A crash+recovery can cause incorrect data to -appear in files which were written shortly before the crash. This mode will -typically provide the best ext3 performance. - -* ordered mode -In data=ordered mode, ext3 only officially journals metadata, but it logically -groups metadata and data blocks into a single unit called a transaction. When -it's time to write the new metadata out to disk, the associated data blocks -are written first. In general, this mode performs slightly slower than -writeback but significantly faster than journal mode. - -* journal mode -data=journal mode provides full data and metadata journaling. All new data is -written to the journal first, and then to its final location. -In the event of a crash, the journal can be replayed, bringing both data and -metadata into a consistent state. This mode is the slowest except when data -needs to be read from and written to disk at the same time where it -outperforms all other modes. - -Compatibility -------------- - -Ext2 partitions can be easily convert to ext3, with `tune2fs -j `. -Ext3 is fully compatible with Ext2. Ext3 partitions can easily be mounted as -Ext2. - - -External Tools -============== -See manual pages to learn more. - -tune2fs: create a ext3 journal on a ext2 partition with the -j flag. -mke2fs: create a ext3 partition with the -j flag. -debugfs: ext2 and ext3 file system debugger. -ext2online: online (mounted) ext2 and ext3 filesystem resizer - - -References -========== - -kernel source: - - -programs: http://e2fsprogs.sourceforge.net/ - http://ext2resize.sourceforge.net - -useful links: http://www.ibm.com/developerworks/library/l-fs7/index.html - http://www.ibm.com/developerworks/library/l-fs8/index.html diff --git a/Documentation/filesystems/f2fs.txt b/Documentation/filesystems/f2fs.txt index e9e750e59efc50..e2d5105b7214ed 100644 --- a/Documentation/filesystems/f2fs.txt +++ b/Documentation/filesystems/f2fs.txt @@ -143,7 +143,9 @@ fastboot This option is used when a system wants to reduce mount extent_cache Enable an extent cache based on rb-tree, it can cache as many as extent which map between contiguous logical address and physical address per inode, resulting in - increasing the cache hit ratio. + increasing the cache hit ratio. Set by default. +noextent_cache Diable an extent cache based on rb-tree explicitly, see + the above extent_cache mount option. noinline_data Disable the inline data feature, inline data feature is enabled by default. diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt index 5eb8456fc41ebc..8c6f07ad373aa6 100644 --- a/Documentation/filesystems/vfs.txt +++ b/Documentation/filesystems/vfs.txt @@ -769,7 +769,7 @@ struct address_space_operations { to stall to allow flushers a chance to complete some IO. Ordinarily it can use PageDirty and PageWriteback but some filesystems have more complex state (unstable pages in NFS prevent reclaim) or - do not set those flags due to locking problems (jbd). This callback + do not set those flags due to locking problems. This callback allows a filesystem to indicate to the VM if a page should be treated as dirty or writeback for the purposes of stalling. diff --git a/Documentation/gpio/00-INDEX b/Documentation/gpio/00-INDEX index 1de43ae46ae6b2..179beb234f989e 100644 --- a/Documentation/gpio/00-INDEX +++ b/Documentation/gpio/00-INDEX @@ -6,6 +6,9 @@ consumer.txt - How to obtain and use GPIOs in a driver driver.txt - How to write a GPIO driver +drivers-on-gpio.txt: + - Drivers in other subsystems that can use GPIO to provide more + complex functionality. board.txt - How to assign GPIOs to a consumer device and a function sysfs.txt diff --git a/Documentation/gpio/consumer.txt b/Documentation/gpio/consumer.txt index 75542b91b766c2..a206639454ab7a 100644 --- a/Documentation/gpio/consumer.txt +++ b/Documentation/gpio/consumer.txt @@ -237,6 +237,39 @@ Note that these functions should only be used with great moderation ; a driver should not have to care about the physical line level. +The active-low property +----------------------- + +As a driver should not have to care about the physical line level, all of the +gpiod_set_value_xxx() or gpiod_set_array_value_xxx() functions operate with +the *logical* value. With this they take the active-low property into account. +This means that they check whether the GPIO is configured to be active-low, +and if so, they manipulate the passed value before the physical line level is +driven. + +With this, all the gpiod_set_(array)_value_xxx() functions interpret the +parameter "value" as "active" ("1") or "inactive" ("0"). The physical line +level will be driven accordingly. + +As an example, if the active-low property for a dedicated GPIO is set, and the +gpiod_set_(array)_value_xxx() passes "active" ("1"), the physical line level +will be driven low. + +To summarize: + +Function (example) active-low proporty physical line +gpiod_set_raw_value(desc, 0); don't care low +gpiod_set_raw_value(desc, 1); don't care high +gpiod_set_value(desc, 0); default (active-high) low +gpiod_set_value(desc, 1); default (active-high) high +gpiod_set_value(desc, 0); active-low high +gpiod_set_value(desc, 1); active-low low + +Please note again that the set_raw/get_raw functions should be avoided as much +as possible, especially by drivers which should not care about the actual +physical line level and worry about the logical value instead. + + Set multiple GPIO outputs with a single function call ----------------------------------------------------- The following functions set the output values of an array of GPIOs: diff --git a/Documentation/gpio/drivers-on-gpio.txt b/Documentation/gpio/drivers-on-gpio.txt new file mode 100644 index 00000000000000..f6121328630f59 --- /dev/null +++ b/Documentation/gpio/drivers-on-gpio.txt @@ -0,0 +1,95 @@ +Subsystem drivers using GPIO +============================ + +Note that standard kernel drivers exist for common GPIO tasks and will provide +the right in-kernel and userspace APIs/ABIs for the job, and that these +drivers can quite easily interconnect with other kernel subsystems using +hardware descriptions such as device tree or ACPI: + +- leds-gpio: drivers/leds/leds-gpio.c will handle LEDs connected to GPIO + lines, giving you the LED sysfs interface + +- ledtrig-gpio: drivers/leds/trigger/ledtrig-gpio.c will provide a LED trigger, + i.e. a LED will turn on/off in response to a GPIO line going high or low + (and that LED may in turn use the leds-gpio as per above). + +- gpio-keys: drivers/input/keyboard/gpio_keys.c is used when your GPIO line + can generate interrupts in response to a key press. Also supports debounce. + +- gpio-keys-polled: drivers/input/keyboard/gpio_keys_polled.c is used when your + GPIO line cannot generate interrupts, so it needs to be periodically polled + by a timer. + +- gpio_mouse: drivers/input/mouse/gpio_mouse.c is used to provide a mouse with + up to three buttons by simply using GPIOs and no mouse port. You can cut the + mouse cable and connect the wires to GPIO lines or solder a mouse connector + to the lines for a more permanent solution of this type. + +- gpio-beeper: drivers/input/misc/gpio-beeper.c is used to provide a beep from + an external speaker connected to a GPIO line. + +- gpio-tilt-polled: drivers/input/misc/gpio_tilt_polled.c provides tilt + detection switches using GPIO, which is useful for your homebrewn pinball + machine if for nothing else. It can detect different tilt angles of the + monitored object. + +- extcon-gpio: drivers/extcon/extcon-gpio.c is used when you need to read an + external connector status, such as a headset line for an audio driver or an + HDMI connector. It will provide a better userspace sysfs interface than GPIO. + +- restart-gpio: drivers/power/gpio-restart.c is used to restart/reboot the + system by pulling a GPIO line and will register a restart handler so + userspace can issue the right system call to restart the system. + +- poweroff-gpio: drivers/power/gpio-poweroff.c is used to power the system down + by pulling a GPIO line and will register a pm_power_off() callback so that + userspace can issue the right system call to power down the system. + +- gpio-gate-clock: drivers/clk/clk-gpio-gate.c is used to control a gated clock + (off/on) that uses a GPIO, and integrated with the clock subsystem. + +- i2c-gpio: drivers/i2c/busses/i2c-gpio.c is used to drive an I2C bus + (two wires, SDA and SCL lines) by hammering (bitbang) two GPIO lines. It will + appear as any other I2C bus to the system and makes it possible to connect + drivers for the I2C devices on the bus like any other I2C bus driver. + +- spi_gpio: drivers/spi/spi-gpio.c is used to drive an SPI bus (variable number + of wires, atleast SCK and optionally MISO, MOSI and chip select lines) using + GPIO hammering (bitbang). It will appear as any other SPI bus on the system + and makes it possible to connect drivers for SPI devices on the bus like + any other SPI bus driver. For example any MMC/SD card can then be connected + to this SPI by using the mmc_spi host from the MMC/SD card subsystem. + +- w1-gpio: drivers/w1/masters/w1-gpio.c is used to drive a one-wire bus using + a GPIO line, integrating with the W1 subsystem and handling devices on + the bus like any other W1 device. + +- gpio-fan: drivers/hwmon/gpio-fan.c is used to control a fan for cooling the + system, connected to a GPIO line (and optionally a GPIO alarm line), + presenting all the right in-kernel and sysfs interfaces to make your system + not overheat. + +- gpio-regulator: drivers/regulator/gpio-regulator.c is used to control a + regulator providing a certain voltage by pulling a GPIO line, integrating + with the regulator subsystem and giving you all the right interfaces. + +- gpio-wdt: drivers/watchdog/gpio_wdt.c is used to provide a watchdog timer + that will periodically "ping" a hardware connected to a GPIO line by toggling + it from 1-to-0-to-1. If that hardware does not recieve its "ping" + periodically, it will reset the system. + +- gpio-nand: drivers/mtd/nand/gpio.c is used to connect a NAND flash chip to + a set of simple GPIO lines: RDY, NCE, ALE, CLE, NWP. It interacts with the + NAND flash MTD subsystem and provides chip access and partition parsing like + any other NAND driving hardware. + +Apart from this there are special GPIO drivers in subsystems like MMC/SD to +read card detect and write protect GPIO lines, and in the TTY serial subsystem +to emulate MCTRL (modem control) signals CTS/RTS by using two GPIO lines. The +MTD NOR flash has add-ons for extra GPIO lines too, though the address bus is +usually connected directly to the flash. + +Use those instead of talking directly to the GPIOs using sysfs; they integrate +with kernel frameworks better than your userspace code could. Needless to say, +just using the apropriate kernel drivers will simplify and speed up your +embedded hacking in particular by providing ready-made components. diff --git a/Documentation/gpio/sysfs.txt b/Documentation/gpio/sysfs.txt index 535b6a8a7a7cca..0700b55637f565 100644 --- a/Documentation/gpio/sysfs.txt +++ b/Documentation/gpio/sysfs.txt @@ -20,11 +20,10 @@ userspace GPIO can be used to determine system configuration data that standard kernels won't know about. And for some tasks, simple userspace GPIO drivers could be all that the system really needs. -Note that standard kernel drivers exist for common "LEDs and Buttons" -GPIO tasks: "leds-gpio" and "gpio_keys", respectively. Use those -instead of talking directly to the GPIOs; they integrate with kernel -frameworks better than your userspace code could. - +DO NOT ABUSE SYFS TO CONTROL HARDWARE THAT HAS PROPER KERNEL DRIVERS. +PLEASE READ THE DOCUMENT NAMED "drivers-on-gpio.txt" IN THIS DOCUMENTATION +DIRECTORY TO AVOID REINVENTING KERNEL WHEELS IN USERSPACE. I MEAN IT. +REALLY. Paths in Sysfs -------------- diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index eafa6a53f72cb1..2ba8461b0631de 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -2327,9 +2327,7 @@ about the state (old or new) implies an SMP-conditional general memory barrier explicit lock operations, described later). These include: xchg(); - cmpxchg(); atomic_xchg(); atomic_long_xchg(); - atomic_cmpxchg(); atomic_long_cmpxchg(); atomic_inc_return(); atomic_long_inc_return(); atomic_dec_return(); atomic_long_dec_return(); atomic_add_return(); atomic_long_add_return(); @@ -2342,7 +2340,9 @@ explicit lock operations, described later). These include: test_and_clear_bit(); test_and_change_bit(); - /* when succeeds (returns 1) */ + /* when succeeds */ + cmpxchg(); + atomic_cmpxchg(); atomic_long_cmpxchg(); atomic_add_unless(); atomic_long_add_unless(); These are used for such things as implementing ACQUIRE-class and RELEASE-class diff --git a/Documentation/static-keys.txt b/Documentation/static-keys.txt index c4407a41b0fc06..f4cb0b2d5cd790 100644 --- a/Documentation/static-keys.txt +++ b/Documentation/static-keys.txt @@ -1,7 +1,22 @@ Static Keys ----------- -By: Jason Baron +DEPRECATED API: + +The use of 'struct static_key' directly, is now DEPRECATED. In addition +static_key_{true,false}() is also DEPRECATED. IE DO NOT use the following: + +struct static_key false = STATIC_KEY_INIT_FALSE; +struct static_key true = STATIC_KEY_INIT_TRUE; +static_key_true() +static_key_false() + +The updated API replacements are: + +DEFINE_STATIC_KEY_TRUE(key); +DEFINE_STATIC_KEY_FALSE(key); +static_key_likely() +statick_key_unlikely() 0) Abstract @@ -9,22 +24,22 @@ Static keys allows the inclusion of seldom used features in performance-sensitive fast-path kernel code, via a GCC feature and a code patching technique. A quick example: - struct static_key key = STATIC_KEY_INIT_FALSE; + DEFINE_STATIC_KEY_FALSE(key); ... - if (static_key_false(&key)) + if (static_branch_unlikely(&key)) do unlikely code else do likely code ... - static_key_slow_inc(); + static_branch_enable(&key); ... - static_key_slow_inc(); + static_branch_disable(&key); ... -The static_key_false() branch will be generated into the code with as little +The static_branch_unlikely() branch will be generated into the code with as little impact to the likely code path as possible. @@ -56,7 +71,7 @@ the branch site to change the branch direction. For example, if we have a simple branch that is disabled by default: - if (static_key_false(&key)) + if (static_branch_unlikely(&key)) printk("I am the true branch\n"); Thus, by default the 'printk' will not be emitted. And the code generated will @@ -75,68 +90,55 @@ the basis for the static keys facility. In order to make use of this optimization you must first define a key: - struct static_key key; - -Which is initialized as: - - struct static_key key = STATIC_KEY_INIT_TRUE; + DEFINE_STATIC_KEY_TRUE(key); or: - struct static_key key = STATIC_KEY_INIT_FALSE; + DEFINE_STATIC_KEY_FALSE(key); + -If the key is not initialized, it is default false. The 'struct static_key', -must be a 'global'. That is, it can't be allocated on the stack or dynamically +The key must be global, that is, it can't be allocated on the stack or dynamically allocated at run-time. The key is then used in code as: - if (static_key_false(&key)) + if (static_branch_unlikely(&key)) do unlikely code else do likely code Or: - if (static_key_true(&key)) + if (static_branch_likely(&key)) do likely code else do unlikely code -A key that is initialized via 'STATIC_KEY_INIT_FALSE', must be used in a -'static_key_false()' construct. Likewise, a key initialized via -'STATIC_KEY_INIT_TRUE' must be used in a 'static_key_true()' construct. A -single key can be used in many branches, but all the branches must match the -way that the key has been initialized. +Keys defined via DEFINE_STATIC_KEY_TRUE(), or DEFINE_STATIC_KEY_FALSE, may +be used in either static_branch_likely() or static_branch_unlikely() +statemnts. -The branch(es) can then be switched via: +Branch(es) can be set true via: - static_key_slow_inc(&key); - ... - static_key_slow_dec(&key); +static_branch_enable(&key); -Thus, 'static_key_slow_inc()' means 'make the branch true', and -'static_key_slow_dec()' means 'make the branch false' with appropriate -reference counting. For example, if the key is initialized true, a -static_key_slow_dec(), will switch the branch to false. And a subsequent -static_key_slow_inc(), will change the branch back to true. Likewise, if the -key is initialized false, a 'static_key_slow_inc()', will change the branch to -true. And then a 'static_key_slow_dec()', will again make the branch false. +or false via: + +static_branch_disable(&key); -An example usage in the kernel is the implementation of tracepoints: +The branch(es) can then be switched via reference counts: - static inline void trace_##name(proto) \ - { \ - if (static_key_false(&__tracepoint_##name.key)) \ - __DO_TRACE(&__tracepoint_##name, \ - TP_PROTO(data_proto), \ - TP_ARGS(data_args), \ - TP_CONDITION(cond)); \ - } + static_branch_inc(&key); + ... + static_branch_dec(&key); -Tracepoints are disabled by default, and can be placed in performance critical -pieces of the kernel. Thus, by using a static key, the tracepoints can have -absolutely minimal impact when not in use. +Thus, 'static_branch_inc()' means 'make the branch true', and +'static_branch_dec()' means 'make the branch false' with appropriate +reference counting. For example, if the key is initialized true, a +static_branch_dec(), will switch the branch to false. And a subsequent +static_branch_inc(), will change the branch back to true. Likewise, if the +key is initialized false, a 'static_branch_inc()', will change the branch to +true. And then a 'static_branch_dec()', will again make the branch false. 4) Architecture level code patching interface, 'jump labels' @@ -150,9 +152,12 @@ simply fall back to a traditional, load, test, and jump sequence. * #define JUMP_LABEL_NOP_SIZE, see: arch/x86/include/asm/jump_label.h -* __always_inline bool arch_static_branch(struct static_key *key), see: +* __always_inline bool arch_static_branch(struct static_key *key, bool branch), see: arch/x86/include/asm/jump_label.h +* __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch), + see: arch/x86/include/asm/jump_label.h + * void arch_jump_label_transform(struct jump_entry *entry, enum jump_label_type type), see: arch/x86/kernel/jump_label.c @@ -173,7 +178,7 @@ SYSCALL_DEFINE0(getppid) { int pid; -+ if (static_key_false(&key)) ++ if (static_branch_unlikely(&key)) + printk("I am the true branch\n"); rcu_read_lock(); diff --git a/Documentation/sysrq.txt b/Documentation/sysrq.txt index 0e307c94809a02..267f39386f99f7 100644 --- a/Documentation/sysrq.txt +++ b/Documentation/sysrq.txt @@ -119,6 +119,7 @@ On all - write a character to /proc/sysrq-trigger. e.g.: 'x' - Used by xmon interface on ppc/powerpc platforms. Show global PMU Registers on sparc64. + Dump all TLB entries on MIPS. 'y' - Show global CPU Registers [SPARC-64 specific] diff --git a/Documentation/trace/ftrace.txt b/Documentation/trace/ftrace.txt index 7ddb1e319f84d1..87bb4aa6a6b985 100644 --- a/Documentation/trace/ftrace.txt +++ b/Documentation/trace/ftrace.txt @@ -346,6 +346,11 @@ of ftrace. Here is a list of some of the key files: x86-tsc: Architectures may define their own clocks. For example, x86 uses its own TSC cycle clock here. + ppc-tb: This uses the powerpc timebase register value. + This is in sync across CPUs and can also be used + to correlate events across hypervisor/guest if + tb_offset is known. + To set a clock, simply echo the clock name into this file. echo global > trace_clock diff --git a/MAINTAINERS b/MAINTAINERS index cb5e81811c819a..4660d8fae90422 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -806,11 +806,13 @@ F: arch/arm/include/asm/floppy.h ARM PMU PROFILING AND DEBUGGING M: Will Deacon S: Maintained -F: arch/arm/kernel/perf_event* +F: arch/arm/kernel/perf_* F: arch/arm/oprofile/common.c -F: arch/arm/include/asm/pmu.h F: arch/arm/kernel/hw_breakpoint.c F: arch/arm/include/asm/hw_breakpoint.h +F: arch/arm/include/asm/perf_event.h +F: drivers/perf/arm_pmu.c +F: include/linux/perf/arm_pmu.h ARM PORT M: Russell King @@ -4078,15 +4080,6 @@ F: Documentation/filesystems/ext2.txt F: fs/ext2/ F: include/linux/ext2* -EXT3 FILE SYSTEM -M: Jan Kara -M: Andrew Morton -M: Andreas Dilger -L: linux-ext4@vger.kernel.org -S: Maintained -F: Documentation/filesystems/ext3.txt -F: fs/ext3/ - EXT4 FILE SYSTEM M: "Theodore Ts'o" M: Andreas Dilger @@ -4425,6 +4418,7 @@ F: include/linux/fscache*.h F2FS FILE SYSTEM M: Jaegeuk Kim M: Changman Lee +R: Chao Yu L: linux-f2fs-devel@lists.sourceforge.net W: http://en.wikipedia.org/wiki/F2FS T: git git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git @@ -4433,6 +4427,7 @@ F: Documentation/filesystems/f2fs.txt F: Documentation/ABI/testing/sysfs-fs-f2fs F: fs/f2fs/ F: include/linux/f2fs_fs.h +F: include/trace/events/f2fs.h FUJITSU FR-V (FRV) PORT M: David Howells @@ -5787,16 +5782,9 @@ S: Maintained F: fs/jffs2/ F: include/uapi/linux/jffs2.h -JOURNALLING LAYER FOR BLOCK DEVICES (JBD) -M: Andrew Morton -M: Jan Kara -L: linux-ext4@vger.kernel.org -S: Maintained -F: fs/jbd/ -F: include/linux/jbd.h - JOURNALLING LAYER FOR BLOCK DEVICES (JBD2) M: "Theodore Ts'o" +M: Jan Kara L: linux-ext4@vger.kernel.org S: Maintained F: fs/jbd2/ @@ -8057,7 +8045,6 @@ F: drivers/pinctrl/sh-pfc/ PIN CONTROLLER - SAMSUNG M: Tomasz Figa -M: Thomas Abraham L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) L: linux-samsung-soc@vger.kernel.org (moderated for non-subscribers) S: Maintained @@ -8134,6 +8121,15 @@ F: include/linux/power_supply.h F: drivers/power/ X: drivers/power/avs/ +POWER STATE COORDINATION INTERFACE (PSCI) +M: Mark Rutland +M: Lorenzo Pieralisi +L: linux-arm-kernel@lists.infradead.org +S: Maintained +F: drivers/firmware/psci.c +F: include/linux/psci.h +F: include/uapi/linux/psci.h + PNP SUPPORT M: "Rafael J. Wysocki" S: Maintained diff --git a/arch/Kconfig b/arch/Kconfig index 8a8ea7110de844..a71cdbe2a04d7c 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -71,6 +71,12 @@ config JUMP_LABEL ( On 32-bit x86, the necessary options added to the compiler flags may increase the size of the kernel slightly. ) +config STATIC_KEYS_SELFTEST + bool "Static key selftest" + depends on JUMP_LABEL + help + Boot time self-test of the branch patching code. + config OPTPROBES def_bool y depends on KPROBES && HAVE_OPTPROBES diff --git a/arch/alpha/include/asm/atomic.h b/arch/alpha/include/asm/atomic.h index 8f8eafbedd7c29..e8c95609842436 100644 --- a/arch/alpha/include/asm/atomic.h +++ b/arch/alpha/include/asm/atomic.h @@ -29,13 +29,13 @@ * branch back to restart the operation. */ -#define ATOMIC_OP(op) \ +#define ATOMIC_OP(op, asm_op) \ static __inline__ void atomic_##op(int i, atomic_t * v) \ { \ unsigned long temp; \ __asm__ __volatile__( \ "1: ldl_l %0,%1\n" \ - " " #op "l %0,%2,%0\n" \ + " " #asm_op " %0,%2,%0\n" \ " stl_c %0,%1\n" \ " beq %0,2f\n" \ ".subsection 2\n" \ @@ -45,15 +45,15 @@ static __inline__ void atomic_##op(int i, atomic_t * v) \ :"Ir" (i), "m" (v->counter)); \ } \ -#define ATOMIC_OP_RETURN(op) \ +#define ATOMIC_OP_RETURN(op, asm_op) \ static inline int atomic_##op##_return(int i, atomic_t *v) \ { \ long temp, result; \ smp_mb(); \ __asm__ __volatile__( \ "1: ldl_l %0,%1\n" \ - " " #op "l %0,%3,%2\n" \ - " " #op "l %0,%3,%0\n" \ + " " #asm_op " %0,%3,%2\n" \ + " " #asm_op " %0,%3,%0\n" \ " stl_c %0,%1\n" \ " beq %0,2f\n" \ ".subsection 2\n" \ @@ -65,13 +65,13 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ return result; \ } -#define ATOMIC64_OP(op) \ +#define ATOMIC64_OP(op, asm_op) \ static __inline__ void atomic64_##op(long i, atomic64_t * v) \ { \ unsigned long temp; \ __asm__ __volatile__( \ "1: ldq_l %0,%1\n" \ - " " #op "q %0,%2,%0\n" \ + " " #asm_op " %0,%2,%0\n" \ " stq_c %0,%1\n" \ " beq %0,2f\n" \ ".subsection 2\n" \ @@ -81,15 +81,15 @@ static __inline__ void atomic64_##op(long i, atomic64_t * v) \ :"Ir" (i), "m" (v->counter)); \ } \ -#define ATOMIC64_OP_RETURN(op) \ +#define ATOMIC64_OP_RETURN(op, asm_op) \ static __inline__ long atomic64_##op##_return(long i, atomic64_t * v) \ { \ long temp, result; \ smp_mb(); \ __asm__ __volatile__( \ "1: ldq_l %0,%1\n" \ - " " #op "q %0,%3,%2\n" \ - " " #op "q %0,%3,%0\n" \ + " " #asm_op " %0,%3,%2\n" \ + " " #asm_op " %0,%3,%0\n" \ " stq_c %0,%1\n" \ " beq %0,2f\n" \ ".subsection 2\n" \ @@ -101,15 +101,27 @@ static __inline__ long atomic64_##op##_return(long i, atomic64_t * v) \ return result; \ } -#define ATOMIC_OPS(opg) \ - ATOMIC_OP(opg) \ - ATOMIC_OP_RETURN(opg) \ - ATOMIC64_OP(opg) \ - ATOMIC64_OP_RETURN(opg) +#define ATOMIC_OPS(op) \ + ATOMIC_OP(op, op##l) \ + ATOMIC_OP_RETURN(op, op##l) \ + ATOMIC64_OP(op, op##q) \ + ATOMIC64_OP_RETURN(op, op##q) ATOMIC_OPS(add) ATOMIC_OPS(sub) +#define atomic_andnot atomic_andnot +#define atomic64_andnot atomic64_andnot + +ATOMIC_OP(and, and) +ATOMIC_OP(andnot, bic) +ATOMIC_OP(or, bis) +ATOMIC_OP(xor, xor) +ATOMIC64_OP(and, and) +ATOMIC64_OP(andnot, bic) +ATOMIC64_OP(or, bis) +ATOMIC64_OP(xor, xor) + #undef ATOMIC_OPS #undef ATOMIC64_OP_RETURN #undef ATOMIC64_OP diff --git a/arch/arc/include/asm/atomic.h b/arch/arc/include/asm/atomic.h index 87d18ae5311559..c3ecda023e3a52 100644 --- a/arch/arc/include/asm/atomic.h +++ b/arch/arc/include/asm/atomic.h @@ -172,9 +172,13 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add, +=, add) ATOMIC_OPS(sub, -=, sub) -ATOMIC_OP(and, &=, and) -#define atomic_clear_mask(mask, v) atomic_and(~(mask), (v)) +#define atomic_andnot atomic_andnot + +ATOMIC_OP(and, &=, and) +ATOMIC_OP(andnot, &= ~, bic) +ATOMIC_OP(or, |=, or) +ATOMIC_OP(xor, ^=, xor) #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 41cbb4a53066b2..0d1b717e1eca67 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -188,6 +188,9 @@ config ARCH_HAS_ILOG2_U64 config ARCH_HAS_BANDGAP bool +config FIX_EARLYCON_MEM + def_bool y if MMU + config GENERIC_HWEIGHT bool default y @@ -1496,6 +1499,7 @@ config HOTPLUG_CPU config ARM_PSCI bool "Support for the ARM Power State Coordination Interface (PSCI)" depends on CPU_V7 + select ARM_PSCI_FW help Say Y here if you want Linux to communicate with system firmware implementing the PSCI specification for CPU-centric power @@ -1700,13 +1704,24 @@ config HIGHPTE consumed by page tables. Setting this option will allow user-space 2nd level page tables to reside in high memory. -config HW_PERF_EVENTS - bool "Enable hardware performance counter support for perf events" - depends on PERF_EVENTS +config CPU_SW_DOMAIN_PAN + bool "Enable use of CPU domains to implement privileged no-access" + depends on MMU && !ARM_LPAE default y help - Enable hardware performance counter support for perf events. If - disabled, perf events will use software events only. + Increase kernel security by ensuring that normal kernel accesses + are unable to access userspace addresses. This can help prevent + use-after-free bugs becoming an exploitable privilege escalation + by ensuring that magic values (such as LIST_POISON) will always + fault when dereferenced. + + CPUs with low-vector mappings use a best-efforts implementation. + Their lower 1MB needs to remain accessible for the vectors, but + the remainder of userspace will become appropriately inaccessible. + +config HW_PERF_EVENTS + def_bool y + depends on ARM_PMU config SYS_SUPPORTS_HUGETLBFS def_bool y diff --git a/arch/arm/boot/dts/ste-dbx5x0.dtsi b/arch/arm/boot/dts/ste-dbx5x0.dtsi index c5fbde3afcf6fc..50f5e9d0920382 100644 --- a/arch/arm/boot/dts/ste-dbx5x0.dtsi +++ b/arch/arm/boot/dts/ste-dbx5x0.dtsi @@ -294,7 +294,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <0>; - + gpio-ranges = <&pinctrl 0 0 32>; clocks = <&prcc_pclk 1 9>; }; @@ -309,7 +309,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <1>; - + gpio-ranges = <&pinctrl 0 32 5>; clocks = <&prcc_pclk 1 9>; }; @@ -324,7 +324,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <2>; - + gpio-ranges = <&pinctrl 0 64 32>; clocks = <&prcc_pclk 3 8>; }; @@ -339,7 +339,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <3>; - + gpio-ranges = <&pinctrl 0 96 2>; clocks = <&prcc_pclk 3 8>; }; @@ -354,7 +354,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <4>; - + gpio-ranges = <&pinctrl 0 128 32>; clocks = <&prcc_pclk 3 8>; }; @@ -369,7 +369,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <5>; - + gpio-ranges = <&pinctrl 0 160 12>; clocks = <&prcc_pclk 3 8>; }; @@ -384,7 +384,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <6>; - + gpio-ranges = <&pinctrl 0 192 32>; clocks = <&prcc_pclk 2 11>; }; @@ -399,7 +399,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <7>; - + gpio-ranges = <&pinctrl 0 224 7>; clocks = <&prcc_pclk 2 11>; }; @@ -414,12 +414,15 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <8>; - + gpio-ranges = <&pinctrl 0 256 12>; clocks = <&prcc_pclk 5 1>; }; - pinctrl { + pinctrl: pinctrl { compatible = "stericsson,db8500-pinctrl"; + nomadik-gpio-chips = <&gpio0>, <&gpio1>, <&gpio2>, <&gpio3>, + <&gpio4>, <&gpio5>, <&gpio6>, <&gpio7>, + <&gpio8>; prcm = <&prcmu>; }; diff --git a/arch/arm/boot/dts/ste-nomadik-stn8815.dtsi b/arch/arm/boot/dts/ste-nomadik-stn8815.dtsi index 176e332fc0bd6b..314f59c1216208 100644 --- a/arch/arm/boot/dts/ste-nomadik-stn8815.dtsi +++ b/arch/arm/boot/dts/ste-nomadik-stn8815.dtsi @@ -59,6 +59,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <0>; + gpio-ranges = <&pinctrl 0 0 32>; clocks = <&pclk>; }; @@ -72,6 +73,7 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <1>; + gpio-ranges = <&pinctrl 0 32 32>; clocks = <&pclk>; }; @@ -85,12 +87,14 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <2>; + gpio-ranges = <&pinctrl 0 64 32>; clocks = <&pclk>; }; gpio3: gpio@101e7000 { compatible = "st,nomadik-gpio"; reg = <0x101e7000 0x80>; + ngpio = <28>; interrupt-parent = <&vica>; interrupts = <9>; interrupt-controller; @@ -98,11 +102,13 @@ gpio-controller; #gpio-cells = <2>; gpio-bank = <3>; + gpio-ranges = <&pinctrl 0 96 28>; clocks = <&pclk>; }; - pinctrl { + pinctrl: pinctrl { compatible = "stericsson,stn8815-pinctrl"; + nomadik-gpio-chips = <&gpio0>, <&gpio1>, <&gpio2>, <&gpio3>; /* Pin configurations */ uart1 { uart1_default_mux: uart1_mux { diff --git a/arch/arm/common/mcpm_platsmp.c b/arch/arm/common/mcpm_platsmp.c index 92e54d7c6f468e..2b25b6038f6694 100644 --- a/arch/arm/common/mcpm_platsmp.c +++ b/arch/arm/common/mcpm_platsmp.c @@ -65,14 +65,10 @@ static int mcpm_cpu_kill(unsigned int cpu) return !mcpm_wait_for_cpu_powerdown(pcpu, pcluster); } -static int mcpm_cpu_disable(unsigned int cpu) +static bool mcpm_cpu_can_disable(unsigned int cpu) { - /* - * We assume all CPUs may be shut down. - * This would be the hook to use for eventual Secure - * OS migration requests as described in the PSCI spec. - */ - return 0; + /* We assume all CPUs may be shut down. */ + return true; } static void mcpm_cpu_die(unsigned int cpu) @@ -92,7 +88,7 @@ static struct smp_operations __initdata mcpm_smp_ops = { .smp_secondary_init = mcpm_secondary_init, #ifdef CONFIG_HOTPLUG_CPU .cpu_kill = mcpm_cpu_kill, - .cpu_disable = mcpm_cpu_disable, + .cpu_can_disable = mcpm_cpu_can_disable, .cpu_die = mcpm_cpu_die, #endif }; diff --git a/arch/arm/include/asm/Kbuild b/arch/arm/include/asm/Kbuild index 30b3bc1666d218..be648eb47cd9d8 100644 --- a/arch/arm/include/asm/Kbuild +++ b/arch/arm/include/asm/Kbuild @@ -12,7 +12,6 @@ generic-y += irq_regs.h generic-y += kdebug.h generic-y += local.h generic-y += local64.h -generic-y += mcs_spinlock.h generic-y += mm-arch-hooks.h generic-y += msgbuf.h generic-y += param.h diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h index 4abe57279c66f0..7bbf325a4f31f1 100644 --- a/arch/arm/include/asm/assembler.h +++ b/arch/arm/include/asm/assembler.h @@ -108,33 +108,37 @@ .endm #endif - .macro asm_trace_hardirqs_off + .macro asm_trace_hardirqs_off, save=1 #if defined(CONFIG_TRACE_IRQFLAGS) + .if \save stmdb sp!, {r0-r3, ip, lr} + .endif bl trace_hardirqs_off + .if \save ldmia sp!, {r0-r3, ip, lr} + .endif #endif .endm - .macro asm_trace_hardirqs_on_cond, cond + .macro asm_trace_hardirqs_on, cond=al, save=1 #if defined(CONFIG_TRACE_IRQFLAGS) /* * actually the registers should be pushed and pop'd conditionally, but * after bl the flags are certainly clobbered */ + .if \save stmdb sp!, {r0-r3, ip, lr} + .endif bl\cond trace_hardirqs_on + .if \save ldmia sp!, {r0-r3, ip, lr} + .endif #endif .endm - .macro asm_trace_hardirqs_on - asm_trace_hardirqs_on_cond al - .endm - - .macro disable_irq + .macro disable_irq, save=1 disable_irq_notrace - asm_trace_hardirqs_off + asm_trace_hardirqs_off \save .endm .macro enable_irq @@ -173,7 +177,7 @@ .macro restore_irqs, oldcpsr tst \oldcpsr, #PSR_I_BIT - asm_trace_hardirqs_on_cond eq + asm_trace_hardirqs_on cond=eq restore_irqs_notrace \oldcpsr .endm @@ -445,6 +449,53 @@ THUMB( orr \reg , \reg , #PSR_T_BIT ) #endif .endm + .macro uaccess_disable, tmp, isb=1 +#ifdef CONFIG_CPU_SW_DOMAIN_PAN + /* + * Whenever we re-enter userspace, the domains should always be + * set appropriately. + */ + mov \tmp, #DACR_UACCESS_DISABLE + mcr p15, 0, \tmp, c3, c0, 0 @ Set domain register + .if \isb + instr_sync + .endif +#endif + .endm + + .macro uaccess_enable, tmp, isb=1 +#ifdef CONFIG_CPU_SW_DOMAIN_PAN + /* + * Whenever we re-enter userspace, the domains should always be + * set appropriately. + */ + mov \tmp, #DACR_UACCESS_ENABLE + mcr p15, 0, \tmp, c3, c0, 0 + .if \isb + instr_sync + .endif +#endif + .endm + + .macro uaccess_save, tmp +#ifdef CONFIG_CPU_SW_DOMAIN_PAN + mrc p15, 0, \tmp, c3, c0, 0 + str \tmp, [sp, #S_FRAME_SIZE] +#endif + .endm + + .macro uaccess_restore +#ifdef CONFIG_CPU_SW_DOMAIN_PAN + ldr r0, [sp, #S_FRAME_SIZE] + mcr p15, 0, r0, c3, c0, 0 +#endif + .endm + + .macro uaccess_save_and_disable, tmp + uaccess_save \tmp + uaccess_disable \tmp + .endm + .irp c,,eq,ne,cs,cc,mi,pl,vs,vc,hi,ls,ge,lt,gt,le,hs,lo .macro ret\c, reg #if __LINUX_ARM_ARCH__ < 6 diff --git a/arch/arm/include/asm/atomic.h b/arch/arm/include/asm/atomic.h index e22c11970b7bd2..fe3ef397f5a407 100644 --- a/arch/arm/include/asm/atomic.h +++ b/arch/arm/include/asm/atomic.h @@ -57,12 +57,11 @@ static inline void atomic_##op(int i, atomic_t *v) \ } \ #define ATOMIC_OP_RETURN(op, c_op, asm_op) \ -static inline int atomic_##op##_return(int i, atomic_t *v) \ +static inline int atomic_##op##_return_relaxed(int i, atomic_t *v) \ { \ unsigned long tmp; \ int result; \ \ - smp_mb(); \ prefetchw(&v->counter); \ \ __asm__ __volatile__("@ atomic_" #op "_return\n" \ @@ -75,17 +74,17 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ : "r" (&v->counter), "Ir" (i) \ : "cc"); \ \ - smp_mb(); \ - \ return result; \ } -static inline int atomic_cmpxchg(atomic_t *ptr, int old, int new) +#define atomic_add_return_relaxed atomic_add_return_relaxed +#define atomic_sub_return_relaxed atomic_sub_return_relaxed + +static inline int atomic_cmpxchg_relaxed(atomic_t *ptr, int old, int new) { int oldval; unsigned long res; - smp_mb(); prefetchw(&ptr->counter); do { @@ -99,10 +98,9 @@ static inline int atomic_cmpxchg(atomic_t *ptr, int old, int new) : "cc"); } while (res); - smp_mb(); - return oldval; } +#define atomic_cmpxchg_relaxed atomic_cmpxchg_relaxed static inline int __atomic_add_unless(atomic_t *v, int a, int u) { @@ -194,6 +192,13 @@ static inline int __atomic_add_unless(atomic_t *v, int a, int u) ATOMIC_OPS(add, +=, add) ATOMIC_OPS(sub, -=, sub) +#define atomic_andnot atomic_andnot + +ATOMIC_OP(and, &=, and) +ATOMIC_OP(andnot, &= ~, bic) +ATOMIC_OP(or, |=, orr) +ATOMIC_OP(xor, ^=, eor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP @@ -290,12 +295,12 @@ static inline void atomic64_##op(long long i, atomic64_t *v) \ } \ #define ATOMIC64_OP_RETURN(op, op1, op2) \ -static inline long long atomic64_##op##_return(long long i, atomic64_t *v) \ +static inline long long \ +atomic64_##op##_return_relaxed(long long i, atomic64_t *v) \ { \ long long result; \ unsigned long tmp; \ \ - smp_mb(); \ prefetchw(&v->counter); \ \ __asm__ __volatile__("@ atomic64_" #op "_return\n" \ @@ -309,8 +314,6 @@ static inline long long atomic64_##op##_return(long long i, atomic64_t *v) \ : "r" (&v->counter), "r" (i) \ : "cc"); \ \ - smp_mb(); \ - \ return result; \ } @@ -321,17 +324,26 @@ static inline long long atomic64_##op##_return(long long i, atomic64_t *v) \ ATOMIC64_OPS(add, adds, adc) ATOMIC64_OPS(sub, subs, sbc) +#define atomic64_add_return_relaxed atomic64_add_return_relaxed +#define atomic64_sub_return_relaxed atomic64_sub_return_relaxed + +#define atomic64_andnot atomic64_andnot + +ATOMIC64_OP(and, and, and) +ATOMIC64_OP(andnot, bic, bic) +ATOMIC64_OP(or, orr, orr) +ATOMIC64_OP(xor, eor, eor) + #undef ATOMIC64_OPS #undef ATOMIC64_OP_RETURN #undef ATOMIC64_OP -static inline long long atomic64_cmpxchg(atomic64_t *ptr, long long old, - long long new) +static inline long long +atomic64_cmpxchg_relaxed(atomic64_t *ptr, long long old, long long new) { long long oldval; unsigned long res; - smp_mb(); prefetchw(&ptr->counter); do { @@ -346,17 +358,15 @@ static inline long long atomic64_cmpxchg(atomic64_t *ptr, long long old, : "cc"); } while (res); - smp_mb(); - return oldval; } +#define atomic64_cmpxchg_relaxed atomic64_cmpxchg_relaxed -static inline long long atomic64_xchg(atomic64_t *ptr, long long new) +static inline long long atomic64_xchg_relaxed(atomic64_t *ptr, long long new) { long long result; unsigned long tmp; - smp_mb(); prefetchw(&ptr->counter); __asm__ __volatile__("@ atomic64_xchg\n" @@ -368,10 +378,9 @@ static inline long long atomic64_xchg(atomic64_t *ptr, long long new) : "r" (&ptr->counter), "r" (new) : "cc"); - smp_mb(); - return result; } +#define atomic64_xchg_relaxed atomic64_xchg_relaxed static inline long long atomic64_dec_if_positive(atomic64_t *v) { diff --git a/arch/arm/include/asm/barrier.h b/arch/arm/include/asm/barrier.h index 6c2327e1c7323d..3ff5642d978849 100644 --- a/arch/arm/include/asm/barrier.h +++ b/arch/arm/include/asm/barrier.h @@ -2,7 +2,6 @@ #define __ASM_BARRIER_H #ifndef __ASSEMBLY__ -#include #define nop() __asm__ __volatile__("mov\tr0,r0\t@ nop\n\t"); @@ -37,12 +36,20 @@ #define dmb(x) __asm__ __volatile__ ("" : : : "memory") #endif +#ifdef CONFIG_ARM_HEAVY_MB +extern void (*soc_mb)(void); +extern void arm_heavy_mb(void); +#define __arm_heavy_mb(x...) do { dsb(x); arm_heavy_mb(); } while (0) +#else +#define __arm_heavy_mb(x...) dsb(x) +#endif + #ifdef CONFIG_ARCH_HAS_BARRIERS #include #elif defined(CONFIG_ARM_DMA_MEM_BUFFERABLE) || defined(CONFIG_SMP) -#define mb() do { dsb(); outer_sync(); } while (0) +#define mb() __arm_heavy_mb() #define rmb() dsb() -#define wmb() do { dsb(st); outer_sync(); } while (0) +#define wmb() __arm_heavy_mb(st) #define dma_rmb() dmb(osh) #define dma_wmb() dmb(oshst) #else @@ -67,12 +74,12 @@ do { \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ ___p1; \ diff --git a/arch/arm/include/asm/bitops.h b/arch/arm/include/asm/bitops.h index 56380995f4c383..e943e6cee25450 100644 --- a/arch/arm/include/asm/bitops.h +++ b/arch/arm/include/asm/bitops.h @@ -35,9 +35,9 @@ static inline void ____atomic_set_bit(unsigned int bit, volatile unsigned long *p) { unsigned long flags; - unsigned long mask = 1UL << (bit & 31); + unsigned long mask = BIT_MASK(bit); - p += bit >> 5; + p += BIT_WORD(bit); raw_local_irq_save(flags); *p |= mask; @@ -47,9 +47,9 @@ static inline void ____atomic_set_bit(unsigned int bit, volatile unsigned long * static inline void ____atomic_clear_bit(unsigned int bit, volatile unsigned long *p) { unsigned long flags; - unsigned long mask = 1UL << (bit & 31); + unsigned long mask = BIT_MASK(bit); - p += bit >> 5; + p += BIT_WORD(bit); raw_local_irq_save(flags); *p &= ~mask; @@ -59,9 +59,9 @@ static inline void ____atomic_clear_bit(unsigned int bit, volatile unsigned long static inline void ____atomic_change_bit(unsigned int bit, volatile unsigned long *p) { unsigned long flags; - unsigned long mask = 1UL << (bit & 31); + unsigned long mask = BIT_MASK(bit); - p += bit >> 5; + p += BIT_WORD(bit); raw_local_irq_save(flags); *p ^= mask; @@ -73,9 +73,9 @@ ____atomic_test_and_set_bit(unsigned int bit, volatile unsigned long *p) { unsigned long flags; unsigned int res; - unsigned long mask = 1UL << (bit & 31); + unsigned long mask = BIT_MASK(bit); - p += bit >> 5; + p += BIT_WORD(bit); raw_local_irq_save(flags); res = *p; @@ -90,9 +90,9 @@ ____atomic_test_and_clear_bit(unsigned int bit, volatile unsigned long *p) { unsigned long flags; unsigned int res; - unsigned long mask = 1UL << (bit & 31); + unsigned long mask = BIT_MASK(bit); - p += bit >> 5; + p += BIT_WORD(bit); raw_local_irq_save(flags); res = *p; @@ -107,9 +107,9 @@ ____atomic_test_and_change_bit(unsigned int bit, volatile unsigned long *p) { unsigned long flags; unsigned int res; - unsigned long mask = 1UL << (bit & 31); + unsigned long mask = BIT_MASK(bit); - p += bit >> 5; + p += BIT_WORD(bit); raw_local_irq_save(flags); res = *p; diff --git a/arch/arm/include/asm/cacheflush.h b/arch/arm/include/asm/cacheflush.h index 4812cda8fd1759..d5525bfc7e3e61 100644 --- a/arch/arm/include/asm/cacheflush.h +++ b/arch/arm/include/asm/cacheflush.h @@ -140,8 +140,6 @@ extern struct cpu_cache_fns cpu_cache; * is visible to DMA, or data written by DMA to system memory is * visible to the CPU. */ -#define dmac_map_area cpu_cache.dma_map_area -#define dmac_unmap_area cpu_cache.dma_unmap_area #define dmac_flush_range cpu_cache.dma_flush_range #else @@ -161,8 +159,6 @@ extern void __cpuc_flush_dcache_area(void *, size_t); * is visible to DMA, or data written by DMA to system memory is * visible to the CPU. */ -extern void dmac_map_area(const void *, size_t, int); -extern void dmac_unmap_area(const void *, size_t, int); extern void dmac_flush_range(const void *, const void *); #endif @@ -506,4 +502,21 @@ static inline void set_kernel_text_ro(void) { } void flush_uprobe_xol_access(struct page *page, unsigned long uaddr, void *kaddr, unsigned long len); +/** + * secure_flush_area - ensure coherency across the secure boundary + * @addr: virtual address + * @size: size of region + * + * Ensure that the specified area of memory is coherent across the secure + * boundary from the non-secure side. This is used when calling secure + * firmware where the secure firmware does not ensure coherency. + */ +static inline void secure_flush_area(const void *addr, size_t size) +{ + phys_addr_t phys = __pa(addr); + + __cpuc_flush_dcache_area((void *)addr, size); + outer_flush_range(phys, phys + size); +} + #endif diff --git a/arch/arm/include/asm/cmpxchg.h b/arch/arm/include/asm/cmpxchg.h index 1692a05d32074c..916a2744d5c66d 100644 --- a/arch/arm/include/asm/cmpxchg.h +++ b/arch/arm/include/asm/cmpxchg.h @@ -35,7 +35,6 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size unsigned int tmp; #endif - smp_mb(); prefetchw((const void *)ptr); switch (size) { @@ -98,12 +97,11 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size __bad_xchg(ptr, size), ret = 0; break; } - smp_mb(); return ret; } -#define xchg(ptr, x) ({ \ +#define xchg_relaxed(ptr, x) ({ \ (__typeof__(*(ptr)))__xchg((unsigned long)(x), (ptr), \ sizeof(*(ptr))); \ }) @@ -117,6 +115,8 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size #error "SMP is not supported on this platform" #endif +#define xchg xchg_relaxed + /* * cmpxchg_local and cmpxchg64_local are atomic wrt current CPU. Always make * them available. @@ -194,23 +194,11 @@ static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old, return oldval; } -static inline unsigned long __cmpxchg_mb(volatile void *ptr, unsigned long old, - unsigned long new, int size) -{ - unsigned long ret; - - smp_mb(); - ret = __cmpxchg(ptr, old, new, size); - smp_mb(); - - return ret; -} - -#define cmpxchg(ptr,o,n) ({ \ - (__typeof__(*(ptr)))__cmpxchg_mb((ptr), \ - (unsigned long)(o), \ - (unsigned long)(n), \ - sizeof(*(ptr))); \ +#define cmpxchg_relaxed(ptr,o,n) ({ \ + (__typeof__(*(ptr)))__cmpxchg((ptr), \ + (unsigned long)(o), \ + (unsigned long)(n), \ + sizeof(*(ptr))); \ }) static inline unsigned long __cmpxchg_local(volatile void *ptr, @@ -273,25 +261,6 @@ static inline unsigned long long __cmpxchg64(unsigned long long *ptr, #define cmpxchg64_local(ptr, o, n) cmpxchg64_relaxed((ptr), (o), (n)) -static inline unsigned long long __cmpxchg64_mb(unsigned long long *ptr, - unsigned long long old, - unsigned long long new) -{ - unsigned long long ret; - - smp_mb(); - ret = __cmpxchg64(ptr, old, new); - smp_mb(); - - return ret; -} - -#define cmpxchg64(ptr, o, n) ({ \ - (__typeof__(*(ptr)))__cmpxchg64_mb((ptr), \ - (unsigned long long)(o), \ - (unsigned long long)(n)); \ -}) - #endif /* __LINUX_ARM_ARCH__ >= 6 */ #endif /* __ASM_ARM_CMPXCHG_H */ diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h index b52101d37ec74d..a68b9d8a71fed8 100644 --- a/arch/arm/include/asm/dma-mapping.h +++ b/arch/arm/include/asm/dma-mapping.h @@ -14,7 +14,7 @@ #include #include -#define DMA_ERROR_CODE (~0) +#define DMA_ERROR_CODE (~(dma_addr_t)0x0) extern struct dma_map_ops arm_dma_ops; extern struct dma_map_ops arm_coherent_dma_ops; diff --git a/arch/arm/include/asm/domain.h b/arch/arm/include/asm/domain.h index 6ddbe446425e11..e878129f2fee5d 100644 --- a/arch/arm/include/asm/domain.h +++ b/arch/arm/include/asm/domain.h @@ -34,15 +34,14 @@ */ #ifndef CONFIG_IO_36 #define DOMAIN_KERNEL 0 -#define DOMAIN_TABLE 0 #define DOMAIN_USER 1 #define DOMAIN_IO 2 #else #define DOMAIN_KERNEL 2 -#define DOMAIN_TABLE 2 #define DOMAIN_USER 1 #define DOMAIN_IO 0 #endif +#define DOMAIN_VECTORS 3 /* * Domain types @@ -55,11 +54,46 @@ #define DOMAIN_MANAGER 1 #endif -#define domain_val(dom,type) ((type) << (2*(dom))) +#define domain_mask(dom) ((3) << (2 * (dom))) +#define domain_val(dom,type) ((type) << (2 * (dom))) + +#ifdef CONFIG_CPU_SW_DOMAIN_PAN +#define DACR_INIT \ + (domain_val(DOMAIN_USER, DOMAIN_NOACCESS) | \ + domain_val(DOMAIN_KERNEL, DOMAIN_MANAGER) | \ + domain_val(DOMAIN_IO, DOMAIN_CLIENT) | \ + domain_val(DOMAIN_VECTORS, DOMAIN_CLIENT)) +#else +#define DACR_INIT \ + (domain_val(DOMAIN_USER, DOMAIN_CLIENT) | \ + domain_val(DOMAIN_KERNEL, DOMAIN_MANAGER) | \ + domain_val(DOMAIN_IO, DOMAIN_CLIENT) | \ + domain_val(DOMAIN_VECTORS, DOMAIN_CLIENT)) +#endif + +#define __DACR_DEFAULT \ + domain_val(DOMAIN_KERNEL, DOMAIN_CLIENT) | \ + domain_val(DOMAIN_IO, DOMAIN_CLIENT) | \ + domain_val(DOMAIN_VECTORS, DOMAIN_CLIENT) + +#define DACR_UACCESS_DISABLE \ + (__DACR_DEFAULT | domain_val(DOMAIN_USER, DOMAIN_NOACCESS)) +#define DACR_UACCESS_ENABLE \ + (__DACR_DEFAULT | domain_val(DOMAIN_USER, DOMAIN_CLIENT)) #ifndef __ASSEMBLY__ -#ifdef CONFIG_CPU_USE_DOMAINS +static inline unsigned int get_domain(void) +{ + unsigned int domain; + + asm( + "mrc p15, 0, %0, c3, c0 @ get domain" + : "=r" (domain)); + + return domain; +} + static inline void set_domain(unsigned val) { asm volatile( @@ -68,17 +102,16 @@ static inline void set_domain(unsigned val) isb(); } +#ifdef CONFIG_CPU_USE_DOMAINS #define modify_domain(dom,type) \ do { \ - struct thread_info *thread = current_thread_info(); \ - unsigned int domain = thread->cpu_domain; \ - domain &= ~domain_val(dom, DOMAIN_MANAGER); \ - thread->cpu_domain = domain | domain_val(dom, type); \ - set_domain(thread->cpu_domain); \ + unsigned int domain = get_domain(); \ + domain &= ~domain_mask(dom); \ + domain = domain | domain_val(dom, type); \ + set_domain(domain); \ } while (0) #else -static inline void set_domain(unsigned val) { } static inline void modify_domain(unsigned dom, unsigned type) { } #endif diff --git a/arch/arm/include/asm/fixmap.h b/arch/arm/include/asm/fixmap.h index 0415eae1df2741..58cfe9f1a687e6 100644 --- a/arch/arm/include/asm/fixmap.h +++ b/arch/arm/include/asm/fixmap.h @@ -6,9 +6,13 @@ #define FIXADDR_TOP (FIXADDR_END - PAGE_SIZE) #include +#include enum fixed_addresses { - FIX_KMAP_BEGIN, + FIX_EARLYCON_MEM_BASE, + __end_of_permanent_fixed_addresses, + + FIX_KMAP_BEGIN = __end_of_permanent_fixed_addresses, FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_TYPE_NR * NR_CPUS) - 1, /* Support writing RO kernel text via kprobes, jump labels, etc. */ @@ -18,7 +22,16 @@ enum fixed_addresses { __end_of_fixed_addresses }; +#define FIXMAP_PAGE_COMMON (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN | L_PTE_DIRTY) + +#define FIXMAP_PAGE_NORMAL (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK) + +/* Used by set_fixmap_(io|nocache), both meant for mapping a device */ +#define FIXMAP_PAGE_IO (FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED | L_PTE_SHARED) +#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO + void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t prot); +void __init early_fixmap_init(void); #include diff --git a/arch/arm/include/asm/futex.h b/arch/arm/include/asm/futex.h index 5eed82809d82b7..6795368ad02380 100644 --- a/arch/arm/include/asm/futex.h +++ b/arch/arm/include/asm/futex.h @@ -22,8 +22,11 @@ #ifdef CONFIG_SMP #define __futex_atomic_op(insn, ret, oldval, tmp, uaddr, oparg) \ +({ \ + unsigned int __ua_flags; \ smp_mb(); \ prefetchw(uaddr); \ + __ua_flags = uaccess_save_and_enable(); \ __asm__ __volatile__( \ "1: ldrex %1, [%3]\n" \ " " insn "\n" \ @@ -34,12 +37,15 @@ __futex_atomic_ex_table("%5") \ : "=&r" (ret), "=&r" (oldval), "=&r" (tmp) \ : "r" (uaddr), "r" (oparg), "Ir" (-EFAULT) \ - : "cc", "memory") + : "cc", "memory"); \ + uaccess_restore(__ua_flags); \ +}) static inline int futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, u32 oldval, u32 newval) { + unsigned int __ua_flags; int ret; u32 val; @@ -49,6 +55,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, smp_mb(); /* Prefetching cannot fault */ prefetchw(uaddr); + __ua_flags = uaccess_save_and_enable(); __asm__ __volatile__("@futex_atomic_cmpxchg_inatomic\n" "1: ldrex %1, [%4]\n" " teq %1, %2\n" @@ -61,6 +68,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, : "=&r" (ret), "=&r" (val) : "r" (oldval), "r" (newval), "r" (uaddr), "Ir" (-EFAULT) : "cc", "memory"); + uaccess_restore(__ua_flags); smp_mb(); *uval = val; @@ -73,6 +81,8 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, #include #define __futex_atomic_op(insn, ret, oldval, tmp, uaddr, oparg) \ +({ \ + unsigned int __ua_flags = uaccess_save_and_enable(); \ __asm__ __volatile__( \ "1: " TUSER(ldr) " %1, [%3]\n" \ " " insn "\n" \ @@ -81,12 +91,15 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, __futex_atomic_ex_table("%5") \ : "=&r" (ret), "=&r" (oldval), "=&r" (tmp) \ : "r" (uaddr), "r" (oparg), "Ir" (-EFAULT) \ - : "cc", "memory") + : "cc", "memory"); \ + uaccess_restore(__ua_flags); \ +}) static inline int futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, u32 oldval, u32 newval) { + unsigned int __ua_flags; int ret = 0; u32 val; @@ -94,6 +107,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, return -EFAULT; preempt_disable(); + __ua_flags = uaccess_save_and_enable(); __asm__ __volatile__("@futex_atomic_cmpxchg_inatomic\n" "1: " TUSER(ldr) " %1, [%4]\n" " teq %1, %2\n" @@ -103,6 +117,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, : "+r" (ret), "=&r" (val) : "r" (oldval), "r" (newval), "r" (uaddr), "Ir" (-EFAULT) : "cc", "memory"); + uaccess_restore(__ua_flags); *uval = val; preempt_enable(); diff --git a/arch/arm/include/asm/glue-cache.h b/arch/arm/include/asm/glue-cache.h index a3c24cd5b7c8ee..cab07f69382dea 100644 --- a/arch/arm/include/asm/glue-cache.h +++ b/arch/arm/include/asm/glue-cache.h @@ -158,8 +158,6 @@ static inline void nop_dma_unmap_area(const void *s, size_t l, int f) { } #define __cpuc_coherent_user_range __glue(_CACHE,_coherent_user_range) #define __cpuc_flush_dcache_area __glue(_CACHE,_flush_kern_dcache_area) -#define dmac_map_area __glue(_CACHE,_dma_map_area) -#define dmac_unmap_area __glue(_CACHE,_dma_unmap_area) #define dmac_flush_range __glue(_CACHE,_dma_flush_range) #endif diff --git a/arch/arm/include/asm/jump_label.h b/arch/arm/include/asm/jump_label.h index 5f337dc5c1087f..34f7b6980d21dc 100644 --- a/arch/arm/include/asm/jump_label.h +++ b/arch/arm/include/asm/jump_label.h @@ -4,23 +4,32 @@ #ifndef __ASSEMBLY__ #include +#include #define JUMP_LABEL_NOP_SIZE 4 -#ifdef CONFIG_THUMB2_KERNEL -#define JUMP_LABEL_NOP "nop.w" -#else -#define JUMP_LABEL_NOP "nop" -#endif +static __always_inline bool arch_static_branch(struct static_key *key, bool branch) +{ + asm_volatile_goto("1:\n\t" + WASM(nop) "\n\t" + ".pushsection __jump_table, \"aw\"\n\t" + ".word 1b, %l[l_yes], %c0\n\t" + ".popsection\n\t" + : : "i" (&((char *)key)[branch]) : : l_yes); + + return false; +l_yes: + return true; +} -static __always_inline bool arch_static_branch(struct static_key *key) +static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch) { asm_volatile_goto("1:\n\t" - JUMP_LABEL_NOP "\n\t" + WASM(b) " %l[l_yes]\n\t" ".pushsection __jump_table, \"aw\"\n\t" ".word 1b, %l[l_yes], %c0\n\t" ".popsection\n\t" - : : "i" (key) : : l_yes); + : : "i" (&((char *)key)[branch]) : : l_yes); return false; l_yes: diff --git a/arch/arm/include/asm/outercache.h b/arch/arm/include/asm/outercache.h index 563b92fc2f41c3..c2bf24f40177dd 100644 --- a/arch/arm/include/asm/outercache.h +++ b/arch/arm/include/asm/outercache.h @@ -129,21 +129,4 @@ static inline void outer_resume(void) { } #endif -#ifdef CONFIG_OUTER_CACHE_SYNC -/** - * outer_sync - perform a sync point for outer cache - * - * Ensure that all outer cache operations are complete and any store - * buffers are drained. - */ -static inline void outer_sync(void) -{ - if (outer_cache.sync) - outer_cache.sync(); -} -#else -static inline void outer_sync(void) -{ } -#endif - #endif /* __ASM_OUTERCACHE_H */ diff --git a/arch/arm/include/asm/pgtable-2level-hwdef.h b/arch/arm/include/asm/pgtable-2level-hwdef.h index 5e68278e953e25..d0131ee6f6af92 100644 --- a/arch/arm/include/asm/pgtable-2level-hwdef.h +++ b/arch/arm/include/asm/pgtable-2level-hwdef.h @@ -23,6 +23,7 @@ #define PMD_PXNTABLE (_AT(pmdval_t, 1) << 2) /* v7 */ #define PMD_BIT4 (_AT(pmdval_t, 1) << 4) #define PMD_DOMAIN(x) (_AT(pmdval_t, (x)) << 5) +#define PMD_DOMAIN_MASK PMD_DOMAIN(0x0f) #define PMD_PROTECTION (_AT(pmdval_t, 1) << 9) /* v5 */ /* * - section diff --git a/arch/arm/include/asm/psci.h b/arch/arm/include/asm/psci.h index c25ef3ec6d1f85..68ee3ce17b820e 100644 --- a/arch/arm/include/asm/psci.h +++ b/arch/arm/include/asm/psci.h @@ -14,34 +14,11 @@ #ifndef __ASM_ARM_PSCI_H #define __ASM_ARM_PSCI_H -#define PSCI_POWER_STATE_TYPE_STANDBY 0 -#define PSCI_POWER_STATE_TYPE_POWER_DOWN 1 - -struct psci_power_state { - u16 id; - u8 type; - u8 affinity_level; -}; - -struct psci_operations { - int (*cpu_suspend)(struct psci_power_state state, - unsigned long entry_point); - int (*cpu_off)(struct psci_power_state state); - int (*cpu_on)(unsigned long cpuid, unsigned long entry_point); - int (*migrate)(unsigned long cpuid); - int (*affinity_info)(unsigned long target_affinity, - unsigned long lowest_affinity_level); - int (*migrate_info_type)(void); -}; - -extern struct psci_operations psci_ops; extern struct smp_operations psci_smp_ops; #ifdef CONFIG_ARM_PSCI -int psci_init(void); bool psci_smp_available(void); #else -static inline int psci_init(void) { return 0; } static inline bool psci_smp_available(void) { return false; } #endif diff --git a/arch/arm/include/asm/smp.h b/arch/arm/include/asm/smp.h index 2f3ac1ba6fb473..ef356659b4f43e 100644 --- a/arch/arm/include/asm/smp.h +++ b/arch/arm/include/asm/smp.h @@ -74,7 +74,6 @@ extern void secondary_startup_arm(void); extern int __cpu_disable(void); extern void __cpu_die(unsigned int cpu); -extern void cpu_die(void); extern void arch_send_call_function_single_ipi(int cpu); extern void arch_send_call_function_ipi_mask(const struct cpumask *mask); @@ -105,6 +104,7 @@ struct smp_operations { #ifdef CONFIG_HOTPLUG_CPU int (*cpu_kill)(unsigned int cpu); void (*cpu_die)(unsigned int cpu); + bool (*cpu_can_disable)(unsigned int cpu); int (*cpu_disable)(unsigned int cpu); #endif #endif diff --git a/arch/arm/include/asm/smp_plat.h b/arch/arm/include/asm/smp_plat.h index 993e5224d8f7ee..f9080717fc88c6 100644 --- a/arch/arm/include/asm/smp_plat.h +++ b/arch/arm/include/asm/smp_plat.h @@ -107,4 +107,13 @@ static inline u32 mpidr_hash_size(void) extern int platform_can_secondary_boot(void); extern int platform_can_cpu_hotplug(void); +#ifdef CONFIG_HOTPLUG_CPU +extern int platform_can_hotplug_cpu(unsigned int cpu); +#else +static inline int platform_can_hotplug_cpu(unsigned int cpu) +{ + return 0; +} +#endif + #endif diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index bd32eded3e5061..d0a1119dcaf38b 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -74,9 +74,6 @@ struct thread_info { .flags = 0, \ .preempt_count = INIT_PREEMPT_COUNT, \ .addr_limit = KERNEL_DS, \ - .cpu_domain = domain_val(DOMAIN_USER, DOMAIN_MANAGER) | \ - domain_val(DOMAIN_KERNEL, DOMAIN_MANAGER) | \ - domain_val(DOMAIN_IO, DOMAIN_CLIENT), \ } #define init_thread_info (init_thread_union.thread_info) @@ -136,22 +133,18 @@ extern int vfp_restore_user_hwstate(struct user_vfp __user *, /* * thread information flags: - * TIF_SYSCALL_TRACE - syscall trace active - * TIF_SYSCAL_AUDIT - syscall auditing active - * TIF_SIGPENDING - signal pending - * TIF_NEED_RESCHED - rescheduling necessary - * TIF_NOTIFY_RESUME - callback before returning to user * TIF_USEDFPU - FPU was used by this task this quantum (SMP) * TIF_POLLING_NRFLAG - true if poll_idle() is polling TIF_NEED_RESCHED */ -#define TIF_SIGPENDING 0 -#define TIF_NEED_RESCHED 1 +#define TIF_SIGPENDING 0 /* signal pending */ +#define TIF_NEED_RESCHED 1 /* rescheduling necessary */ #define TIF_NOTIFY_RESUME 2 /* callback before returning to user */ -#define TIF_UPROBE 7 -#define TIF_SYSCALL_TRACE 8 -#define TIF_SYSCALL_AUDIT 9 -#define TIF_SYSCALL_TRACEPOINT 10 -#define TIF_SECCOMP 11 /* seccomp syscall filtering active */ +#define TIF_UPROBE 3 /* breakpointed or singlestepping */ +#define TIF_SYSCALL_TRACE 4 /* syscall trace active */ +#define TIF_SYSCALL_AUDIT 5 /* syscall auditing active */ +#define TIF_SYSCALL_TRACEPOINT 6 /* syscall tracepoint instrumentation */ +#define TIF_SECCOMP 7 /* seccomp syscall filtering active */ + #define TIF_NOHZ 12 /* in adaptive nohz mode */ #define TIF_USING_IWMMXT 17 #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ diff --git a/arch/arm/include/asm/uaccess.h b/arch/arm/include/asm/uaccess.h index 74b17d09ef7aa5..8cc85a4ebec20a 100644 --- a/arch/arm/include/asm/uaccess.h +++ b/arch/arm/include/asm/uaccess.h @@ -49,6 +49,35 @@ struct exception_table_entry extern int fixup_exception(struct pt_regs *regs); +/* + * These two functions allow hooking accesses to userspace to increase + * system integrity by ensuring that the kernel can not inadvertantly + * perform such accesses (eg, via list poison values) which could then + * be exploited for priviledge escalation. + */ +static inline unsigned int uaccess_save_and_enable(void) +{ +#ifdef CONFIG_CPU_SW_DOMAIN_PAN + unsigned int old_domain = get_domain(); + + /* Set the current domain access to permit user accesses */ + set_domain((old_domain & ~domain_mask(DOMAIN_USER)) | + domain_val(DOMAIN_USER, DOMAIN_CLIENT)); + + return old_domain; +#else + return 0; +#endif +} + +static inline void uaccess_restore(unsigned int flags) +{ +#ifdef CONFIG_CPU_SW_DOMAIN_PAN + /* Restore the user access mask */ + set_domain(flags); +#endif +} + /* * These two are intentionally not defined anywhere - if the kernel * code generates any references to them, that's a bug. @@ -165,6 +194,7 @@ extern int __get_user_64t_4(void *); register typeof(x) __r2 asm("r2"); \ register unsigned long __l asm("r1") = __limit; \ register int __e asm("r0"); \ + unsigned int __ua_flags = uaccess_save_and_enable(); \ switch (sizeof(*(__p))) { \ case 1: \ if (sizeof((x)) >= 8) \ @@ -192,6 +222,7 @@ extern int __get_user_64t_4(void *); break; \ default: __e = __get_user_bad(); break; \ } \ + uaccess_restore(__ua_flags); \ x = (typeof(*(p))) __r2; \ __e; \ }) @@ -224,6 +255,7 @@ extern int __put_user_8(void *, unsigned long long); register const typeof(*(p)) __user *__p asm("r0") = __tmp_p; \ register unsigned long __l asm("r1") = __limit; \ register int __e asm("r0"); \ + unsigned int __ua_flags = uaccess_save_and_enable(); \ switch (sizeof(*(__p))) { \ case 1: \ __put_user_x(__r2, __p, __e, __l, 1); \ @@ -239,6 +271,7 @@ extern int __put_user_8(void *, unsigned long long); break; \ default: __e = __put_user_bad(); break; \ } \ + uaccess_restore(__ua_flags); \ __e; \ }) @@ -300,20 +333,23 @@ static inline void set_fs(mm_segment_t fs) do { \ unsigned long __gu_addr = (unsigned long)(ptr); \ unsigned long __gu_val; \ + unsigned int __ua_flags; \ __chk_user_ptr(ptr); \ might_fault(); \ + __ua_flags = uaccess_save_and_enable(); \ switch (sizeof(*(ptr))) { \ case 1: __get_user_asm_byte(__gu_val, __gu_addr, err); break; \ case 2: __get_user_asm_half(__gu_val, __gu_addr, err); break; \ case 4: __get_user_asm_word(__gu_val, __gu_addr, err); break; \ default: (__gu_val) = __get_user_bad(); \ } \ + uaccess_restore(__ua_flags); \ (x) = (__typeof__(*(ptr)))__gu_val; \ } while (0) -#define __get_user_asm_byte(x, addr, err) \ +#define __get_user_asm(x, addr, err, instr) \ __asm__ __volatile__( \ - "1: " TUSER(ldrb) " %1,[%2],#0\n" \ + "1: " TUSER(instr) " %1, [%2], #0\n" \ "2:\n" \ " .pushsection .text.fixup,\"ax\"\n" \ " .align 2\n" \ @@ -329,6 +365,9 @@ do { \ : "r" (addr), "i" (-EFAULT) \ : "cc") +#define __get_user_asm_byte(x, addr, err) \ + __get_user_asm(x, addr, err, ldrb) + #ifndef __ARMEB__ #define __get_user_asm_half(x, __gu_addr, err) \ ({ \ @@ -348,22 +387,7 @@ do { \ #endif #define __get_user_asm_word(x, addr, err) \ - __asm__ __volatile__( \ - "1: " TUSER(ldr) " %1,[%2],#0\n" \ - "2:\n" \ - " .pushsection .text.fixup,\"ax\"\n" \ - " .align 2\n" \ - "3: mov %0, %3\n" \ - " mov %1, #0\n" \ - " b 2b\n" \ - " .popsection\n" \ - " .pushsection __ex_table,\"a\"\n" \ - " .align 3\n" \ - " .long 1b, 3b\n" \ - " .popsection" \ - : "+r" (err), "=&r" (x) \ - : "r" (addr), "i" (-EFAULT) \ - : "cc") + __get_user_asm(x, addr, err, ldr) #define __put_user(x, ptr) \ ({ \ @@ -381,9 +405,11 @@ do { \ #define __put_user_err(x, ptr, err) \ do { \ unsigned long __pu_addr = (unsigned long)(ptr); \ + unsigned int __ua_flags; \ __typeof__(*(ptr)) __pu_val = (x); \ __chk_user_ptr(ptr); \ might_fault(); \ + __ua_flags = uaccess_save_and_enable(); \ switch (sizeof(*(ptr))) { \ case 1: __put_user_asm_byte(__pu_val, __pu_addr, err); break; \ case 2: __put_user_asm_half(__pu_val, __pu_addr, err); break; \ @@ -391,11 +417,12 @@ do { \ case 8: __put_user_asm_dword(__pu_val, __pu_addr, err); break; \ default: __put_user_bad(); \ } \ + uaccess_restore(__ua_flags); \ } while (0) -#define __put_user_asm_byte(x, __pu_addr, err) \ +#define __put_user_asm(x, __pu_addr, err, instr) \ __asm__ __volatile__( \ - "1: " TUSER(strb) " %1,[%2],#0\n" \ + "1: " TUSER(instr) " %1, [%2], #0\n" \ "2:\n" \ " .pushsection .text.fixup,\"ax\"\n" \ " .align 2\n" \ @@ -410,6 +437,9 @@ do { \ : "r" (x), "r" (__pu_addr), "i" (-EFAULT) \ : "cc") +#define __put_user_asm_byte(x, __pu_addr, err) \ + __put_user_asm(x, __pu_addr, err, strb) + #ifndef __ARMEB__ #define __put_user_asm_half(x, __pu_addr, err) \ ({ \ @@ -427,21 +457,7 @@ do { \ #endif #define __put_user_asm_word(x, __pu_addr, err) \ - __asm__ __volatile__( \ - "1: " TUSER(str) " %1,[%2],#0\n" \ - "2:\n" \ - " .pushsection .text.fixup,\"ax\"\n" \ - " .align 2\n" \ - "3: mov %0, %3\n" \ - " b 2b\n" \ - " .popsection\n" \ - " .pushsection __ex_table,\"a\"\n" \ - " .align 3\n" \ - " .long 1b, 3b\n" \ - " .popsection" \ - : "+r" (err) \ - : "r" (x), "r" (__pu_addr), "i" (-EFAULT) \ - : "cc") + __put_user_asm(x, __pu_addr, err, str) #ifndef __ARMEB__ #define __reg_oper0 "%R2" @@ -474,11 +490,46 @@ do { \ #ifdef CONFIG_MMU -extern unsigned long __must_check __copy_from_user(void *to, const void __user *from, unsigned long n); -extern unsigned long __must_check __copy_to_user(void __user *to, const void *from, unsigned long n); -extern unsigned long __must_check __copy_to_user_std(void __user *to, const void *from, unsigned long n); -extern unsigned long __must_check __clear_user(void __user *addr, unsigned long n); -extern unsigned long __must_check __clear_user_std(void __user *addr, unsigned long n); +extern unsigned long __must_check +arm_copy_from_user(void *to, const void __user *from, unsigned long n); + +static inline unsigned long __must_check +__copy_from_user(void *to, const void __user *from, unsigned long n) +{ + unsigned int __ua_flags = uaccess_save_and_enable(); + n = arm_copy_from_user(to, from, n); + uaccess_restore(__ua_flags); + return n; +} + +extern unsigned long __must_check +arm_copy_to_user(void __user *to, const void *from, unsigned long n); +extern unsigned long __must_check +__copy_to_user_std(void __user *to, const void *from, unsigned long n); + +static inline unsigned long __must_check +__copy_to_user(void __user *to, const void *from, unsigned long n) +{ + unsigned int __ua_flags = uaccess_save_and_enable(); + n = arm_copy_to_user(to, from, n); + uaccess_restore(__ua_flags); + return n; +} + +extern unsigned long __must_check +arm_clear_user(void __user *addr, unsigned long n); +extern unsigned long __must_check +__clear_user_std(void __user *addr, unsigned long n); + +static inline unsigned long __must_check +__clear_user(void __user *addr, unsigned long n) +{ + unsigned int __ua_flags = uaccess_save_and_enable(); + n = arm_clear_user(addr, n); + uaccess_restore(__ua_flags); + return n; +} + #else #define __copy_from_user(to, from, n) (memcpy(to, (void __force *)from, n), 0) #define __copy_to_user(to, from, n) (memcpy((void __force *)to, from, n), 0) @@ -511,6 +562,7 @@ static inline unsigned long __must_check clear_user(void __user *to, unsigned lo return n; } +/* These are from lib/ code, and use __get_user() and friends */ extern long strncpy_from_user(char *dest, const char __user *src, long count); extern __must_check long strlen_user(const char __user *str); diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index e69f7a19735d04..af9e59bf3831b9 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -71,8 +71,7 @@ obj-$(CONFIG_CPU_PJ4) += pj4-cp0.o obj-$(CONFIG_CPU_PJ4B) += pj4-cp0.o obj-$(CONFIG_IWMMXT) += iwmmxt.o obj-$(CONFIG_PERF_EVENTS) += perf_regs.o perf_callchain.o -obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o \ - perf_event_xscale.o perf_event_v6.o \ +obj-$(CONFIG_HW_PERF_EVENTS) += perf_event_xscale.o perf_event_v6.o \ perf_event_v7.o CFLAGS_pj4-cp0.o := -marm AFLAGS_iwmmxt.o := -Wa,-mcpu=iwmmxt @@ -89,7 +88,7 @@ obj-$(CONFIG_EARLY_PRINTK) += early_printk.o obj-$(CONFIG_ARM_VIRT_EXT) += hyp-stub.o ifeq ($(CONFIG_ARM_PSCI),y) -obj-y += psci.o psci-call.o +obj-y += psci-call.o obj-$(CONFIG_SMP) += psci_smp.o endif diff --git a/arch/arm/kernel/armksyms.c b/arch/arm/kernel/armksyms.c index 5e5a51a99e68ec..f89811fb9a55f3 100644 --- a/arch/arm/kernel/armksyms.c +++ b/arch/arm/kernel/armksyms.c @@ -97,9 +97,9 @@ EXPORT_SYMBOL(mmiocpy); #ifdef CONFIG_MMU EXPORT_SYMBOL(copy_page); -EXPORT_SYMBOL(__copy_from_user); -EXPORT_SYMBOL(__copy_to_user); -EXPORT_SYMBOL(__clear_user); +EXPORT_SYMBOL(arm_copy_from_user); +EXPORT_SYMBOL(arm_copy_to_user); +EXPORT_SYMBOL(arm_clear_user); EXPORT_SYMBOL(__get_user_1); EXPORT_SYMBOL(__get_user_2); diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index cb4fb1e6977860..3e1c26eb32b43e 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -149,10 +149,10 @@ ENDPROC(__und_invalid) #define SPFIX(code...) #endif - .macro svc_entry, stack_hole=0, trace=1 + .macro svc_entry, stack_hole=0, trace=1, uaccess=1 UNWIND(.fnstart ) UNWIND(.save {r0 - pc} ) - sub sp, sp, #(S_FRAME_SIZE + \stack_hole - 4) + sub sp, sp, #(S_FRAME_SIZE + 8 + \stack_hole - 4) #ifdef CONFIG_THUMB2_KERNEL SPFIX( str r0, [sp] ) @ temporarily saved SPFIX( mov r0, sp ) @@ -167,7 +167,7 @@ ENDPROC(__und_invalid) ldmia r0, {r3 - r5} add r7, sp, #S_SP - 4 @ here for interlock avoidance mov r6, #-1 @ "" "" "" "" - add r2, sp, #(S_FRAME_SIZE + \stack_hole - 4) + add r2, sp, #(S_FRAME_SIZE + 8 + \stack_hole - 4) SPFIX( addeq r2, r2, #4 ) str r3, [sp, #-4]! @ save the "real" r0 copied @ from the exception stack @@ -185,6 +185,11 @@ ENDPROC(__und_invalid) @ stmia r7, {r2 - r6} + uaccess_save r0 + .if \uaccess + uaccess_disable r0 + .endif + .if \trace #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off @@ -194,7 +199,7 @@ ENDPROC(__und_invalid) .align 5 __dabt_svc: - svc_entry + svc_entry uaccess=0 mov r2, sp dabt_helper THUMB( ldr r5, [sp, #S_PSR] ) @ potentially updated CPSR @@ -368,7 +373,7 @@ ENDPROC(__fiq_abt) #error "sizeof(struct pt_regs) must be a multiple of 8" #endif - .macro usr_entry, trace=1 + .macro usr_entry, trace=1, uaccess=1 UNWIND(.fnstart ) UNWIND(.cantunwind ) @ don't unwind the user space sub sp, sp, #S_FRAME_SIZE @@ -400,6 +405,10 @@ ENDPROC(__fiq_abt) ARM( stmdb r0, {sp, lr}^ ) THUMB( store_user_sp_lr r0, r1, S_SP - S_PC ) + .if \uaccess + uaccess_disable ip + .endif + @ Enable the alignment trap while in kernel mode ATRAP( teq r8, r7) ATRAP( mcrne p15, 0, r8, c1, c0, 0) @@ -435,7 +444,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: - usr_entry + usr_entry uaccess=0 kuser_cmpxchg_check mov r2, sp dabt_helper @@ -458,7 +467,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: - usr_entry + usr_entry uaccess=0 mov r2, r4 mov r3, r5 @@ -484,6 +493,8 @@ __und_usr: 1: ldrt r0, [r4] ARM_BE8(rev r0, r0) @ little endian instruction + uaccess_disable ip + @ r0 = 32-bit ARM instruction which caused the exception @ r2 = PC value for the following instruction (:= regs->ARM_pc) @ r4 = PC value for the faulting instruction @@ -518,9 +529,10 @@ __und_usr_thumb: 2: ldrht r5, [r4] ARM_BE8(rev16 r5, r5) @ little endian instruction cmp r5, #0xe800 @ 32bit instruction if xx != 0 - blo __und_usr_fault_16 @ 16bit undefined instruction + blo __und_usr_fault_16_pan @ 16bit undefined instruction 3: ldrht r0, [r2] ARM_BE8(rev16 r0, r0) @ little endian instruction + uaccess_disable ip add r2, r2, #2 @ r2 is PC + 2, make it PC + 4 str r2, [sp, #S_PC] @ it's a 2x16bit instr, update orr r0, r0, r5, lsl #16 @@ -715,6 +727,8 @@ ENDPROC(no_fp) __und_usr_fault_32: mov r1, #4 b 1f +__und_usr_fault_16_pan: + uaccess_disable ip __und_usr_fault_16: mov r1, #2 1: mov r0, sp @@ -770,6 +784,8 @@ ENTRY(__switch_to) ldr r4, [r2, #TI_TP_VALUE] ldr r5, [r2, #TI_TP_VALUE + 4] #ifdef CONFIG_CPU_USE_DOMAINS + mrc p15, 0, r6, c3, c0, 0 @ Get domain register + str r6, [r1, #TI_CPU_DOMAIN] @ Save old domain register ldr r6, [r2, #TI_CPU_DOMAIN] #endif switch_tls r1, r4, r5, r3, r7 diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index b48dd4f37f8067..30a7228eaceba7 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -24,35 +24,55 @@ .align 5 +#if !(IS_ENABLED(CONFIG_TRACE_IRQFLAGS) || IS_ENABLED(CONFIG_CONTEXT_TRACKING)) /* - * This is the fast syscall return path. We do as little as - * possible here, and this includes saving r0 back into the SVC - * stack. + * This is the fast syscall return path. We do as little as possible here, + * such as avoiding writing r0 to the stack. We only use this path if we + * have tracing and context tracking disabled - the overheads from those + * features make this path too inefficient. */ ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) - disable_irq @ disable interrupts + disable_irq_notrace @ disable interrupts ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing - tst r1, #_TIF_SYSCALL_WORK - bne __sys_trace_return - tst r1, #_TIF_WORK_MASK + tst r1, #_TIF_SYSCALL_WORK | _TIF_WORK_MASK bne fast_work_pending - asm_trace_hardirqs_on /* perform architecture specific actions before user return */ arch_ret_to_user r1, lr - ct_user_enter restore_user_regs fast = 1, offset = S_OFF UNWIND(.fnend ) +ENDPROC(ret_fast_syscall) -/* - * Ok, we need to do extra processing, enter the slow path. - */ + /* Ok, we need to do extra processing, enter the slow path. */ fast_work_pending: str r0, [sp, #S_R0+S_OFF]! @ returned r0 -work_pending: + /* fall through to work_pending */ +#else +/* + * The "replacement" ret_fast_syscall for when tracing or context tracking + * is enabled. As we will need to call out to some C functions, we save + * r0 first to avoid needing to save registers around each C function call. + */ +ret_fast_syscall: + UNWIND(.fnstart ) + UNWIND(.cantunwind ) + str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 + disable_irq_notrace @ disable interrupts + ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing + tst r1, #_TIF_SYSCALL_WORK | _TIF_WORK_MASK + beq no_work_pending + UNWIND(.fnend ) +ENDPROC(ret_fast_syscall) + + /* Slower path - fall through to work_pending */ +#endif + + tst r1, #_TIF_SYSCALL_WORK + bne __sys_trace_return_nosave +slow_work_pending: mov r0, sp @ 'regs' mov r2, why @ 'syscall' bl do_work_pending @@ -65,16 +85,19 @@ ENDPROC(ret_fast_syscall) /* * "slow" syscall return path. "why" tells us if this was a real syscall. + * IRQs may be enabled here, so always disable them. Note that we use the + * "notrace" version to avoid calling into the tracing code unnecessarily. + * do_work_pending() will update this state if necessary. */ ENTRY(ret_to_user) ret_slow_syscall: - disable_irq @ disable interrupts + disable_irq_notrace @ disable interrupts ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] tst r1, #_TIF_WORK_MASK - bne work_pending + bne slow_work_pending no_work_pending: - asm_trace_hardirqs_on + asm_trace_hardirqs_on save = 0 /* perform architecture specific actions before user return */ arch_ret_to_user r1, lr @@ -174,6 +197,8 @@ ENTRY(vector_swi) USER( ldr scno, [lr, #-4] ) @ get SWI instruction #endif + uaccess_disable tbl + adr tbl, sys_call_table @ load syscall table pointer #if defined(CONFIG_OABI_COMPAT) @@ -252,6 +277,12 @@ __sys_trace_return: bl syscall_trace_exit b ret_slow_syscall +__sys_trace_return_nosave: + enable_irq_notrace + mov r0, sp + bl syscall_trace_exit + b ret_slow_syscall + .align 5 #ifdef CONFIG_ALIGNMENT_TRAP .type __cr_alignment, #object diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 1a0045abead756..0d22ad206d5230 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -196,7 +196,7 @@ msr cpsr_c, \rtemp @ switch back to the SVC mode .endm -#ifndef CONFIG_THUMB2_KERNEL + .macro svc_exit, rpsr, irq = 0 .if \irq != 0 @ IRQs already off @@ -215,6 +215,10 @@ blne trace_hardirqs_off #endif .endif + uaccess_restore + +#ifndef CONFIG_THUMB2_KERNEL + @ ARM mode SVC restore msr spsr_cxsf, \rpsr #if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_32v6K) @ We must avoid clrex due to Cortex-A15 erratum #830321 @@ -222,6 +226,20 @@ strex r1, r2, [r0] @ clear the exclusive monitor #endif ldmia sp, {r0 - pc}^ @ load r0 - pc, cpsr +#else + @ Thumb mode SVC restore + ldr lr, [sp, #S_SP] @ top of the stack + ldrd r0, r1, [sp, #S_LR] @ calling lr and pc + + @ We must avoid clrex due to Cortex-A15 erratum #830321 + strex r2, r1, [sp, #S_LR] @ clear the exclusive monitor + + stmdb lr!, {r0, r1, \rpsr} @ calling lr and rfe context + ldmia sp, {r0 - r12} + mov sp, lr + ldr lr, [sp], #4 + rfeia sp! +#endif .endm @ @@ -241,6 +259,9 @@ @ on the stack remains correct). @ .macro svc_exit_via_fiq + uaccess_restore +#ifndef CONFIG_THUMB2_KERNEL + @ ARM mode restore mov r0, sp ldmib r0, {r1 - r14} @ abort is deadly from here onward (it will @ clobber state restored below) @@ -250,9 +271,27 @@ msr spsr_cxsf, r9 ldr r0, [r0, #S_R0] ldmia r8, {pc}^ +#else + @ Thumb mode restore + add r0, sp, #S_R2 + ldr lr, [sp, #S_LR] + ldr sp, [sp, #S_SP] @ abort is deadly from here onward (it will + @ clobber state restored below) + ldmia r0, {r2 - r12} + mov r1, #FIQ_MODE | PSR_I_BIT | PSR_F_BIT + msr cpsr_c, r1 + sub r0, #S_R2 + add r8, r0, #S_PC + ldmia r0, {r0 - r1} + rfeia r8 +#endif .endm + .macro restore_user_regs, fast = 0, offset = 0 + uaccess_enable r1, isb=0 +#ifndef CONFIG_THUMB2_KERNEL + @ ARM mode restore mov r2, sp ldr r1, [r2, #\offset + S_PSR] @ get calling cpsr ldr lr, [r2, #\offset + S_PC]! @ get pc @@ -270,72 +309,16 @@ @ after ldm {}^ add sp, sp, #\offset + S_FRAME_SIZE movs pc, lr @ return & move spsr_svc into cpsr - .endm - -#else /* CONFIG_THUMB2_KERNEL */ - .macro svc_exit, rpsr, irq = 0 - .if \irq != 0 - @ IRQs already off -#ifdef CONFIG_TRACE_IRQFLAGS - @ The parent context IRQs must have been enabled to get here in - @ the first place, so there's no point checking the PSR I bit. - bl trace_hardirqs_on -#endif - .else - @ IRQs off again before pulling preserved data off the stack - disable_irq_notrace -#ifdef CONFIG_TRACE_IRQFLAGS - tst \rpsr, #PSR_I_BIT - bleq trace_hardirqs_on - tst \rpsr, #PSR_I_BIT - blne trace_hardirqs_off -#endif - .endif - ldr lr, [sp, #S_SP] @ top of the stack - ldrd r0, r1, [sp, #S_LR] @ calling lr and pc - - @ We must avoid clrex due to Cortex-A15 erratum #830321 - strex r2, r1, [sp, #S_LR] @ clear the exclusive monitor - - stmdb lr!, {r0, r1, \rpsr} @ calling lr and rfe context - ldmia sp, {r0 - r12} - mov sp, lr - ldr lr, [sp], #4 - rfeia sp! - .endm - - @ - @ svc_exit_via_fiq - like svc_exit but switches to FIQ mode before exit - @ - @ For full details see non-Thumb implementation above. - @ - .macro svc_exit_via_fiq - add r0, sp, #S_R2 - ldr lr, [sp, #S_LR] - ldr sp, [sp, #S_SP] @ abort is deadly from here onward (it will - @ clobber state restored below) - ldmia r0, {r2 - r12} - mov r1, #FIQ_MODE | PSR_I_BIT | PSR_F_BIT - msr cpsr_c, r1 - sub r0, #S_R2 - add r8, r0, #S_PC - ldmia r0, {r0 - r1} - rfeia r8 - .endm - -#ifdef CONFIG_CPU_V7M - /* - * Note we don't need to do clrex here as clearing the local monitor is - * part of each exception entry and exit sequence. - */ - .macro restore_user_regs, fast = 0, offset = 0 +#elif defined(CONFIG_CPU_V7M) + @ V7M restore. + @ Note that we don't need to do clrex here as clearing the local + @ monitor is part of the exception entry and exit sequence. .if \offset add sp, #\offset .endif v7m_exception_slow_exit ret_r0 = \fast - .endm -#else /* ifdef CONFIG_CPU_V7M */ - .macro restore_user_regs, fast = 0, offset = 0 +#else + @ Thumb mode restore mov r2, sp load_user_sp_lr r2, r3, \offset + S_SP @ calling sp, lr ldr r1, [sp, #\offset + S_PSR] @ get calling cpsr @@ -353,9 +336,8 @@ .endif add sp, sp, #S_FRAME_SIZE - S_SP movs pc, lr @ return & move spsr_svc into cpsr - .endm -#endif /* ifdef CONFIG_CPU_V7M / else */ #endif /* !CONFIG_THUMB2_KERNEL */ + .endm /* * Context tracking subsystem. Used to instrument transitions diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S index 29e2991465cb27..04286fd9e09ce7 100644 --- a/arch/arm/kernel/head.S +++ b/arch/arm/kernel/head.S @@ -464,10 +464,7 @@ __enable_mmu: #ifdef CONFIG_ARM_LPAE mcrr p15, 0, r4, r5, c2 @ load TTBR0 #else - mov r5, #(domain_val(DOMAIN_USER, DOMAIN_MANAGER) | \ - domain_val(DOMAIN_KERNEL, DOMAIN_MANAGER) | \ - domain_val(DOMAIN_TABLE, DOMAIN_MANAGER) | \ - domain_val(DOMAIN_IO, DOMAIN_CLIENT)) + mov r5, #DACR_INIT mcr p15, 0, r5, c3, c0, 0 @ load domain access register mcr p15, 0, r4, c2, c0, 0 @ load page table pointer #endif diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c index baf8edebe26f73..5ff4826cb154b0 100644 --- a/arch/arm/kernel/irq.c +++ b/arch/arm/kernel/irq.c @@ -39,6 +39,7 @@ #include #include +#include #include #include #include diff --git a/arch/arm/kernel/jump_label.c b/arch/arm/kernel/jump_label.c index e39cbf488cfebc..845a5dd9c42ba3 100644 --- a/arch/arm/kernel/jump_label.c +++ b/arch/arm/kernel/jump_label.c @@ -12,7 +12,7 @@ static void __arch_jump_label_transform(struct jump_entry *entry, void *addr = (void *)entry->code; unsigned int insn; - if (type == JUMP_LABEL_ENABLE) + if (type == JUMP_LABEL_JMP) insn = arm_gen_branch(entry->code, entry->target); else insn = arm_gen_nop(); diff --git a/arch/arm/kernel/perf_event_v6.c b/arch/arm/kernel/perf_event_v6.c index 09f83e414a7262..09413e7b49aa62 100644 --- a/arch/arm/kernel/perf_event_v6.c +++ b/arch/arm/kernel/perf_event_v6.c @@ -34,9 +34,9 @@ #include #include -#include #include +#include #include enum armv6_perf_types { diff --git a/arch/arm/kernel/perf_event_v7.c b/arch/arm/kernel/perf_event_v7.c index f9b37f876e2077..126dc679b2308a 100644 --- a/arch/arm/kernel/perf_event_v7.c +++ b/arch/arm/kernel/perf_event_v7.c @@ -21,11 +21,11 @@ #include #include #include -#include #include #include "../vfp/vfpinstr.h" #include +#include #include /* diff --git a/arch/arm/kernel/perf_event_xscale.c b/arch/arm/kernel/perf_event_xscale.c index 304d056d5b252c..aa0499e2eef7b3 100644 --- a/arch/arm/kernel/perf_event_xscale.c +++ b/arch/arm/kernel/perf_event_xscale.c @@ -16,9 +16,9 @@ #include #include -#include #include +#include #include enum xscale_perf_types { diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c index f192a2a4171935..a3089bacb8d822 100644 --- a/arch/arm/kernel/process.c +++ b/arch/arm/kernel/process.c @@ -91,13 +91,6 @@ void arch_cpu_idle_exit(void) ledtrig_cpu(CPU_LED_IDLE_END); } -#ifdef CONFIG_HOTPLUG_CPU -void arch_cpu_idle_dead(void) -{ - cpu_die(); -} -#endif - void __show_regs(struct pt_regs *regs) { unsigned long flags; @@ -129,12 +122,36 @@ void __show_regs(struct pt_regs *regs) buf[4] = '\0'; #ifndef CONFIG_CPU_V7M - printk("Flags: %s IRQs o%s FIQs o%s Mode %s ISA %s Segment %s\n", - buf, interrupts_enabled(regs) ? "n" : "ff", - fast_interrupts_enabled(regs) ? "n" : "ff", - processor_modes[processor_mode(regs)], - isa_modes[isa_mode(regs)], - get_fs() == get_ds() ? "kernel" : "user"); + { + unsigned int domain = get_domain(); + const char *segment; + +#ifdef CONFIG_CPU_SW_DOMAIN_PAN + /* + * Get the domain register for the parent context. In user + * mode, we don't save the DACR, so lets use what it should + * be. For other modes, we place it after the pt_regs struct. + */ + if (user_mode(regs)) + domain = DACR_UACCESS_ENABLE; + else + domain = *(unsigned int *)(regs + 1); +#endif + + if ((domain & domain_mask(DOMAIN_USER)) == + domain_val(DOMAIN_USER, DOMAIN_NOACCESS)) + segment = "none"; + else if (get_fs() == get_ds()) + segment = "kernel"; + else + segment = "user"; + + printk("Flags: %s IRQs o%s FIQs o%s Mode %s ISA %s Segment %s\n", + buf, interrupts_enabled(regs) ? "n" : "ff", + fast_interrupts_enabled(regs) ? "n" : "ff", + processor_modes[processor_mode(regs)], + isa_modes[isa_mode(regs)], segment); + } #else printk("xPSR: %08lx\n", regs->ARM_cpsr); #endif @@ -146,10 +163,9 @@ void __show_regs(struct pt_regs *regs) buf[0] = '\0'; #ifdef CONFIG_CPU_CP15_MMU { - unsigned int transbase, dac; + unsigned int transbase, dac = get_domain(); asm("mrc p15, 0, %0, c2, c0\n\t" - "mrc p15, 0, %1, c3, c0\n" - : "=r" (transbase), "=r" (dac)); + : "=r" (transbase)); snprintf(buf, sizeof(buf), " Table: %08x DAC: %08x", transbase, dac); } @@ -210,6 +226,14 @@ copy_thread(unsigned long clone_flags, unsigned long stack_start, memset(&thread->cpu_context, 0, sizeof(struct cpu_context_save)); + /* + * Copy the initial value of the domain access control register + * from the current thread: thread->addr_limit will have been + * copied from the current thread via setup_thread_stack() in + * kernel/fork.c + */ + thread->cpu_domain = get_domain(); + if (likely(!(p->flags & PF_KTHREAD))) { *childregs = *current_pt_regs(); childregs->ARM_r0 = 0; diff --git a/arch/arm/kernel/psci.c b/arch/arm/kernel/psci.c deleted file mode 100644 index 2e602433479029..00000000000000 --- a/arch/arm/kernel/psci.c +++ /dev/null @@ -1,299 +0,0 @@ -/* - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * Copyright (C) 2012 ARM Limited - * - * Author: Will Deacon - */ - -#define pr_fmt(fmt) "psci: " fmt - -#include -#include -#include -#include -#include - -#include -#include -#include -#include - -struct psci_operations psci_ops; - -static int (*invoke_psci_fn)(u32, u32, u32, u32); -typedef int (*psci_initcall_t)(const struct device_node *); - -asmlinkage int __invoke_psci_fn_hvc(u32, u32, u32, u32); -asmlinkage int __invoke_psci_fn_smc(u32, u32, u32, u32); - -enum psci_function { - PSCI_FN_CPU_SUSPEND, - PSCI_FN_CPU_ON, - PSCI_FN_CPU_OFF, - PSCI_FN_MIGRATE, - PSCI_FN_AFFINITY_INFO, - PSCI_FN_MIGRATE_INFO_TYPE, - PSCI_FN_MAX, -}; - -static u32 psci_function_id[PSCI_FN_MAX]; - -static int psci_to_linux_errno(int errno) -{ - switch (errno) { - case PSCI_RET_SUCCESS: - return 0; - case PSCI_RET_NOT_SUPPORTED: - return -EOPNOTSUPP; - case PSCI_RET_INVALID_PARAMS: - return -EINVAL; - case PSCI_RET_DENIED: - return -EPERM; - }; - - return -EINVAL; -} - -static u32 psci_power_state_pack(struct psci_power_state state) -{ - return ((state.id << PSCI_0_2_POWER_STATE_ID_SHIFT) - & PSCI_0_2_POWER_STATE_ID_MASK) | - ((state.type << PSCI_0_2_POWER_STATE_TYPE_SHIFT) - & PSCI_0_2_POWER_STATE_TYPE_MASK) | - ((state.affinity_level << PSCI_0_2_POWER_STATE_AFFL_SHIFT) - & PSCI_0_2_POWER_STATE_AFFL_MASK); -} - -static int psci_get_version(void) -{ - int err; - - err = invoke_psci_fn(PSCI_0_2_FN_PSCI_VERSION, 0, 0, 0); - return err; -} - -static int psci_cpu_suspend(struct psci_power_state state, - unsigned long entry_point) -{ - int err; - u32 fn, power_state; - - fn = psci_function_id[PSCI_FN_CPU_SUSPEND]; - power_state = psci_power_state_pack(state); - err = invoke_psci_fn(fn, power_state, entry_point, 0); - return psci_to_linux_errno(err); -} - -static int psci_cpu_off(struct psci_power_state state) -{ - int err; - u32 fn, power_state; - - fn = psci_function_id[PSCI_FN_CPU_OFF]; - power_state = psci_power_state_pack(state); - err = invoke_psci_fn(fn, power_state, 0, 0); - return psci_to_linux_errno(err); -} - -static int psci_cpu_on(unsigned long cpuid, unsigned long entry_point) -{ - int err; - u32 fn; - - fn = psci_function_id[PSCI_FN_CPU_ON]; - err = invoke_psci_fn(fn, cpuid, entry_point, 0); - return psci_to_linux_errno(err); -} - -static int psci_migrate(unsigned long cpuid) -{ - int err; - u32 fn; - - fn = psci_function_id[PSCI_FN_MIGRATE]; - err = invoke_psci_fn(fn, cpuid, 0, 0); - return psci_to_linux_errno(err); -} - -static int psci_affinity_info(unsigned long target_affinity, - unsigned long lowest_affinity_level) -{ - int err; - u32 fn; - - fn = psci_function_id[PSCI_FN_AFFINITY_INFO]; - err = invoke_psci_fn(fn, target_affinity, lowest_affinity_level, 0); - return err; -} - -static int psci_migrate_info_type(void) -{ - int err; - u32 fn; - - fn = psci_function_id[PSCI_FN_MIGRATE_INFO_TYPE]; - err = invoke_psci_fn(fn, 0, 0, 0); - return err; -} - -static int get_set_conduit_method(struct device_node *np) -{ - const char *method; - - pr_info("probing for conduit method from DT.\n"); - - if (of_property_read_string(np, "method", &method)) { - pr_warn("missing \"method\" property\n"); - return -ENXIO; - } - - if (!strcmp("hvc", method)) { - invoke_psci_fn = __invoke_psci_fn_hvc; - } else if (!strcmp("smc", method)) { - invoke_psci_fn = __invoke_psci_fn_smc; - } else { - pr_warn("invalid \"method\" property: %s\n", method); - return -EINVAL; - } - return 0; -} - -static void psci_sys_reset(enum reboot_mode reboot_mode, const char *cmd) -{ - invoke_psci_fn(PSCI_0_2_FN_SYSTEM_RESET, 0, 0, 0); -} - -static void psci_sys_poweroff(void) -{ - invoke_psci_fn(PSCI_0_2_FN_SYSTEM_OFF, 0, 0, 0); -} - -/* - * PSCI Function IDs for v0.2+ are well defined so use - * standard values. - */ -static int psci_0_2_init(struct device_node *np) -{ - int err, ver; - - err = get_set_conduit_method(np); - - if (err) - goto out_put_node; - - ver = psci_get_version(); - - if (ver == PSCI_RET_NOT_SUPPORTED) { - /* PSCI v0.2 mandates implementation of PSCI_ID_VERSION. */ - pr_err("PSCI firmware does not comply with the v0.2 spec.\n"); - err = -EOPNOTSUPP; - goto out_put_node; - } else { - pr_info("PSCIv%d.%d detected in firmware.\n", - PSCI_VERSION_MAJOR(ver), - PSCI_VERSION_MINOR(ver)); - - if (PSCI_VERSION_MAJOR(ver) == 0 && - PSCI_VERSION_MINOR(ver) < 2) { - err = -EINVAL; - pr_err("Conflicting PSCI version detected.\n"); - goto out_put_node; - } - } - - pr_info("Using standard PSCI v0.2 function IDs\n"); - psci_function_id[PSCI_FN_CPU_SUSPEND] = PSCI_0_2_FN_CPU_SUSPEND; - psci_ops.cpu_suspend = psci_cpu_suspend; - - psci_function_id[PSCI_FN_CPU_OFF] = PSCI_0_2_FN_CPU_OFF; - psci_ops.cpu_off = psci_cpu_off; - - psci_function_id[PSCI_FN_CPU_ON] = PSCI_0_2_FN_CPU_ON; - psci_ops.cpu_on = psci_cpu_on; - - psci_function_id[PSCI_FN_MIGRATE] = PSCI_0_2_FN_MIGRATE; - psci_ops.migrate = psci_migrate; - - psci_function_id[PSCI_FN_AFFINITY_INFO] = PSCI_0_2_FN_AFFINITY_INFO; - psci_ops.affinity_info = psci_affinity_info; - - psci_function_id[PSCI_FN_MIGRATE_INFO_TYPE] = - PSCI_0_2_FN_MIGRATE_INFO_TYPE; - psci_ops.migrate_info_type = psci_migrate_info_type; - - arm_pm_restart = psci_sys_reset; - - pm_power_off = psci_sys_poweroff; - -out_put_node: - of_node_put(np); - return err; -} - -/* - * PSCI < v0.2 get PSCI Function IDs via DT. - */ -static int psci_0_1_init(struct device_node *np) -{ - u32 id; - int err; - - err = get_set_conduit_method(np); - - if (err) - goto out_put_node; - - pr_info("Using PSCI v0.1 Function IDs from DT\n"); - - if (!of_property_read_u32(np, "cpu_suspend", &id)) { - psci_function_id[PSCI_FN_CPU_SUSPEND] = id; - psci_ops.cpu_suspend = psci_cpu_suspend; - } - - if (!of_property_read_u32(np, "cpu_off", &id)) { - psci_function_id[PSCI_FN_CPU_OFF] = id; - psci_ops.cpu_off = psci_cpu_off; - } - - if (!of_property_read_u32(np, "cpu_on", &id)) { - psci_function_id[PSCI_FN_CPU_ON] = id; - psci_ops.cpu_on = psci_cpu_on; - } - - if (!of_property_read_u32(np, "migrate", &id)) { - psci_function_id[PSCI_FN_MIGRATE] = id; - psci_ops.migrate = psci_migrate; - } - -out_put_node: - of_node_put(np); - return err; -} - -static const struct of_device_id const psci_of_match[] __initconst = { - { .compatible = "arm,psci", .data = psci_0_1_init}, - { .compatible = "arm,psci-0.2", .data = psci_0_2_init}, - {}, -}; - -int __init psci_init(void) -{ - struct device_node *np; - const struct of_device_id *matched_np; - psci_initcall_t init_fn; - - np = of_find_matching_node_and_match(NULL, psci_of_match, &matched_np); - if (!np) - return -ENODEV; - - init_fn = (psci_initcall_t)matched_np->data; - return init_fn(np); -} diff --git a/arch/arm/kernel/psci_smp.c b/arch/arm/kernel/psci_smp.c index 28a1db4da70428..61c04b02faebb0 100644 --- a/arch/arm/kernel/psci_smp.c +++ b/arch/arm/kernel/psci_smp.c @@ -17,6 +17,8 @@ #include #include #include +#include + #include #include @@ -51,22 +53,34 @@ static int psci_boot_secondary(unsigned int cpu, struct task_struct *idle) { if (psci_ops.cpu_on) return psci_ops.cpu_on(cpu_logical_map(cpu), - __pa(secondary_startup)); + virt_to_idmap(&secondary_startup)); return -ENODEV; } #ifdef CONFIG_HOTPLUG_CPU +int psci_cpu_disable(unsigned int cpu) +{ + /* Fail early if we don't have CPU_OFF support */ + if (!psci_ops.cpu_off) + return -EOPNOTSUPP; + + /* Trusted OS will deny CPU_OFF */ + if (psci_tos_resident_on(cpu)) + return -EPERM; + + return 0; +} + void __ref psci_cpu_die(unsigned int cpu) { - const struct psci_power_state ps = { - .type = PSCI_POWER_STATE_TYPE_POWER_DOWN, - }; + u32 state = PSCI_POWER_STATE_TYPE_POWER_DOWN << + PSCI_0_2_POWER_STATE_TYPE_SHIFT; - if (psci_ops.cpu_off) - psci_ops.cpu_off(ps); + if (psci_ops.cpu_off) + psci_ops.cpu_off(state); - /* We should never return */ - panic("psci: cpu %d failed to shutdown\n", cpu); + /* We should never return */ + panic("psci: cpu %d failed to shutdown\n", cpu); } int __ref psci_cpu_kill(unsigned int cpu) @@ -109,6 +123,7 @@ bool __init psci_smp_available(void) struct smp_operations __initdata psci_smp_ops = { .smp_boot_secondary = psci_boot_secondary, #ifdef CONFIG_HOTPLUG_CPU + .cpu_disable = psci_cpu_disable, .cpu_die = psci_cpu_die, .cpu_kill = psci_cpu_kill, #endif diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c index 36c18b73c1f463..20edd349d379f2 100644 --- a/arch/arm/kernel/setup.c +++ b/arch/arm/kernel/setup.c @@ -31,12 +31,14 @@ #include #include #include +#include #include #include #include #include #include +#include #include #include #include @@ -954,6 +956,9 @@ void __init setup_arch(char **cmdline_p) strlcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE); *cmdline_p = cmd_line; + if (IS_ENABLED(CONFIG_FIX_EARLYCON_MEM)) + early_fixmap_init(); + parse_early_param(); #ifdef CONFIG_MMU @@ -972,7 +977,7 @@ void __init setup_arch(char **cmdline_p) unflatten_device_tree(); arm_dt_init_cpu_maps(); - psci_init(); + psci_dt_init(); xen_early_init(); #ifdef CONFIG_SMP if (is_smp()) { @@ -1015,7 +1020,7 @@ static int __init topology_init(void) for_each_possible_cpu(cpu) { struct cpuinfo_arm *cpuinfo = &per_cpu(cpu_data, cpu); - cpuinfo->cpu.hotpluggable = 1; + cpuinfo->cpu.hotpluggable = platform_can_hotplug_cpu(cpu); register_cpu(&cpuinfo->cpu, cpu); } diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 423663e23791e1..b6cda06b455fc6 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -562,6 +562,12 @@ static int do_signal(struct pt_regs *regs, int syscall) asmlinkage int do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) { + /* + * The assembly code enters us with IRQs off, but it hasn't + * informed the tracing code of that for efficiency reasons. + * Update the trace code with the current status. + */ + trace_hardirqs_off(); do { if (likely(thread_flags & _TIF_NEED_RESCHED)) { schedule(); diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index 3d6b7821cff8c9..ba0063c539c3fc 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -175,13 +175,26 @@ static int platform_cpu_disable(unsigned int cpu) if (smp_ops.cpu_disable) return smp_ops.cpu_disable(cpu); + return 0; +} + +int platform_can_hotplug_cpu(unsigned int cpu) +{ + /* cpu_die must be specified to support hotplug */ + if (!smp_ops.cpu_die) + return 0; + + if (smp_ops.cpu_can_disable) + return smp_ops.cpu_can_disable(cpu); + /* * By default, allow disabling all CPUs except the first one, * since this is special on a lot of platforms, e.g. because * of clock tick interrupts. */ - return cpu == 0 ? -EPERM : 0; + return cpu != 0; } + /* * __cpu_disable runs on the processor to be shutdown. */ @@ -253,7 +266,7 @@ void __cpu_die(unsigned int cpu) * of the other hotplug-cpu capable cores, so presumably coming * out of idle fixes this. */ -void __ref cpu_die(void) +void arch_cpu_idle_dead(void) { unsigned int cpu = smp_processor_id(); diff --git a/arch/arm/kernel/swp_emulate.c b/arch/arm/kernel/swp_emulate.c index 1361756782c73b..5b26e7efa9ea41 100644 --- a/arch/arm/kernel/swp_emulate.c +++ b/arch/arm/kernel/swp_emulate.c @@ -141,11 +141,14 @@ static int emulate_swpX(unsigned int address, unsigned int *data, while (1) { unsigned long temp; + unsigned int __ua_flags; + __ua_flags = uaccess_save_and_enable(); if (type == TYPE_SWPB) __user_swpb_asm(*data, address, res, temp); else __user_swp_asm(*data, address, res, temp); + uaccess_restore(__ua_flags); if (likely(res != -EAGAIN) || signal_pending(current)) break; diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index d358226236f295..969f9d9e665f4d 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -870,7 +870,6 @@ void __init early_trap_init(void *vectors_base) kuser_init(vectors_base); flush_icache_range(vectors, vectors + PAGE_SIZE * 2); - modify_domain(DOMAIN_USER, DOMAIN_CLIENT); #else /* ifndef CONFIG_CPU_V7M */ /* * on V7-M there is no need to copy the vector table to a dedicated diff --git a/arch/arm/lib/clear_user.S b/arch/arm/lib/clear_user.S index 1710fd7db2d57d..970d6c0437743c 100644 --- a/arch/arm/lib/clear_user.S +++ b/arch/arm/lib/clear_user.S @@ -12,14 +12,14 @@ .text -/* Prototype: int __clear_user(void *addr, size_t sz) +/* Prototype: unsigned long arm_clear_user(void *addr, size_t sz) * Purpose : clear some user memory * Params : addr - user memory address to clear * : sz - number of bytes to clear * Returns : number of bytes NOT cleared */ ENTRY(__clear_user_std) -WEAK(__clear_user) +WEAK(arm_clear_user) stmfd sp!, {r1, lr} mov r2, #0 cmp r1, #4 @@ -44,7 +44,7 @@ WEAK(__clear_user) USER( strnebt r2, [r0]) mov r0, #0 ldmfd sp!, {r1, pc} -ENDPROC(__clear_user) +ENDPROC(arm_clear_user) ENDPROC(__clear_user_std) .pushsection .text.fixup,"ax" diff --git a/arch/arm/lib/copy_from_user.S b/arch/arm/lib/copy_from_user.S index 7a235b9952be04..1512bebfbf1b18 100644 --- a/arch/arm/lib/copy_from_user.S +++ b/arch/arm/lib/copy_from_user.S @@ -17,7 +17,7 @@ /* * Prototype: * - * size_t __copy_from_user(void *to, const void *from, size_t n) + * size_t arm_copy_from_user(void *to, const void *from, size_t n) * * Purpose: * @@ -89,11 +89,11 @@ .text -ENTRY(__copy_from_user) +ENTRY(arm_copy_from_user) #include "copy_template.S" -ENDPROC(__copy_from_user) +ENDPROC(arm_copy_from_user) .pushsection .fixup,"ax" .align 0 diff --git a/arch/arm/lib/copy_to_user.S b/arch/arm/lib/copy_to_user.S index 9648b0675a3efc..caf5019d8161e2 100644 --- a/arch/arm/lib/copy_to_user.S +++ b/arch/arm/lib/copy_to_user.S @@ -17,7 +17,7 @@ /* * Prototype: * - * size_t __copy_to_user(void *to, const void *from, size_t n) + * size_t arm_copy_to_user(void *to, const void *from, size_t n) * * Purpose: * @@ -93,11 +93,11 @@ .text ENTRY(__copy_to_user_std) -WEAK(__copy_to_user) +WEAK(arm_copy_to_user) #include "copy_template.S" -ENDPROC(__copy_to_user) +ENDPROC(arm_copy_to_user) ENDPROC(__copy_to_user_std) .pushsection .text.fixup,"ax" diff --git a/arch/arm/lib/csumpartialcopyuser.S b/arch/arm/lib/csumpartialcopyuser.S index 1d0957e61f898a..1712f132b80d24 100644 --- a/arch/arm/lib/csumpartialcopyuser.S +++ b/arch/arm/lib/csumpartialcopyuser.S @@ -17,6 +17,19 @@ .text +#ifdef CONFIG_CPU_SW_DOMAIN_PAN + .macro save_regs + mrc p15, 0, ip, c3, c0, 0 + stmfd sp!, {r1, r2, r4 - r8, ip, lr} + uaccess_enable ip + .endm + + .macro load_regs + ldmfd sp!, {r1, r2, r4 - r8, ip, lr} + mcr p15, 0, ip, c3, c0, 0 + ret lr + .endm +#else .macro save_regs stmfd sp!, {r1, r2, r4 - r8, lr} .endm @@ -24,6 +37,7 @@ .macro load_regs ldmfd sp!, {r1, r2, r4 - r8, pc} .endm +#endif .macro load1b, reg1 ldrusr \reg1, r0, 1 diff --git a/arch/arm/lib/uaccess_with_memcpy.c b/arch/arm/lib/uaccess_with_memcpy.c index 4b39af2dfda996..d72b9090513248 100644 --- a/arch/arm/lib/uaccess_with_memcpy.c +++ b/arch/arm/lib/uaccess_with_memcpy.c @@ -136,7 +136,7 @@ __copy_to_user_memcpy(void __user *to, const void *from, unsigned long n) } unsigned long -__copy_to_user(void __user *to, const void *from, unsigned long n) +arm_copy_to_user(void __user *to, const void *from, unsigned long n) { /* * This test is stubbed out of the main function above to keep @@ -190,7 +190,7 @@ __clear_user_memset(void __user *addr, unsigned long n) return n; } -unsigned long __clear_user(void __user *addr, unsigned long n) +unsigned long arm_clear_user(void __user *addr, unsigned long n) { /* See rational for this in __copy_to_user() above. */ if (n < 64) diff --git a/arch/arm/mach-highbank/highbank.c b/arch/arm/mach-highbank/highbank.c index 231fba0d03e513..6050a14faee6f3 100644 --- a/arch/arm/mach-highbank/highbank.c +++ b/arch/arm/mach-highbank/highbank.c @@ -28,8 +28,8 @@ #include #include #include +#include -#include #include #include #include diff --git a/arch/arm/mach-highbank/pm.c b/arch/arm/mach-highbank/pm.c index 7f2bd85eb9350d..40031169554813 100644 --- a/arch/arm/mach-highbank/pm.c +++ b/arch/arm/mach-highbank/pm.c @@ -16,19 +16,21 @@ #include #include +#include #include #include -#include + +#include + +#define HIGHBANK_SUSPEND_PARAM \ + ((0 << PSCI_0_2_POWER_STATE_ID_SHIFT) | \ + (1 << PSCI_0_2_POWER_STATE_AFFL_SHIFT) | \ + (PSCI_POWER_STATE_TYPE_POWER_DOWN << PSCI_0_2_POWER_STATE_TYPE_SHIFT)) static int highbank_suspend_finish(unsigned long val) { - const struct psci_power_state ps = { - .type = PSCI_POWER_STATE_TYPE_POWER_DOWN, - .affinity_level = 1, - }; - - return psci_ops.cpu_suspend(ps, __pa(cpu_resume)); + return psci_ops.cpu_suspend(HIGHBANK_SUSPEND_PARAM, __pa(cpu_resume)); } static int highbank_pm_enter(suspend_state_t state) diff --git a/arch/arm/mach-mmp/pm-pxa910.c b/arch/arm/mach-mmp/pm-pxa910.c index 04c9daf9f8d767..7db5870d127fc0 100644 --- a/arch/arm/mach-mmp/pm-pxa910.c +++ b/arch/arm/mach-mmp/pm-pxa910.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include diff --git a/arch/arm/mach-omap2/Kconfig b/arch/arm/mach-omap2/Kconfig index 9e2a68456b8153..07d2e100caabf9 100644 --- a/arch/arm/mach-omap2/Kconfig +++ b/arch/arm/mach-omap2/Kconfig @@ -29,6 +29,7 @@ config ARCH_OMAP4 select HAVE_ARM_SCU if SMP select HAVE_ARM_TWD if SMP select OMAP_INTERCONNECT + select OMAP_INTERCONNECT_BARRIER select PL310_ERRATA_588369 if CACHE_L2X0 select PL310_ERRATA_727915 if CACHE_L2X0 select PM_OPP if PM @@ -46,6 +47,7 @@ config SOC_OMAP5 select HAVE_ARM_TWD if SMP select HAVE_ARM_ARCH_TIMER select ARM_ERRATA_798181 if SMP + select OMAP_INTERCONNECT_BARRIER config SOC_AM33XX bool "TI AM33XX" @@ -71,6 +73,7 @@ config SOC_DRA7XX select HAVE_ARM_ARCH_TIMER select IRQ_CROSSBAR select ARM_ERRATA_798181 if SMP + select OMAP_INTERCONNECT_BARRIER config ARCH_OMAP2PLUS bool @@ -92,6 +95,10 @@ config ARCH_OMAP2PLUS help Systems based on OMAP2, OMAP3, OMAP4 or OMAP5 +config OMAP_INTERCONNECT_BARRIER + bool + select ARM_HEAVY_MB + if ARCH_OMAP2PLUS diff --git a/arch/arm/mach-omap2/common.c b/arch/arm/mach-omap2/common.c index eae6a0e87c90d3..484cdadfb18785 100644 --- a/arch/arm/mach-omap2/common.c +++ b/arch/arm/mach-omap2/common.c @@ -30,4 +30,5 @@ int __weak omap_secure_ram_reserve_memblock(void) void __init omap_reserve(void) { omap_secure_ram_reserve_memblock(); + omap_barrier_reserve_memblock(); } diff --git a/arch/arm/mach-omap2/common.h b/arch/arm/mach-omap2/common.h index 749d50bb4ca59d..92e92cfc2775f4 100644 --- a/arch/arm/mach-omap2/common.h +++ b/arch/arm/mach-omap2/common.h @@ -189,6 +189,15 @@ static inline void omap44xx_restart(enum reboot_mode mode, const char *cmd) } #endif +#ifdef CONFIG_OMAP_INTERCONNECT_BARRIER +void omap_barrier_reserve_memblock(void); +void omap_barriers_init(void); +#else +static inline void omap_barrier_reserve_memblock(void) +{ +} +#endif + /* This gets called from mach-omap2/io.c, do not call this */ void __init omap2_set_globals_tap(u32 class, void __iomem *tap); diff --git a/arch/arm/mach-omap2/include/mach/barriers.h b/arch/arm/mach-omap2/include/mach/barriers.h deleted file mode 100644 index 1c582a8592b936..00000000000000 --- a/arch/arm/mach-omap2/include/mach/barriers.h +++ /dev/null @@ -1,33 +0,0 @@ -/* - * OMAP memory barrier header. - * - * Copyright (C) 2011 Texas Instruments, Inc. - * Santosh Shilimkar - * Richard Woodruff - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write to the Free Software - * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - */ - -#ifndef __MACH_BARRIERS_H -#define __MACH_BARRIERS_H - -#include - -extern void omap_bus_sync(void); - -#define rmb() dsb() -#define wmb() do { dsb(); outer_sync(); omap_bus_sync(); } while (0) -#define mb() wmb() - -#endif /* __MACH_BARRIERS_H */ diff --git a/arch/arm/mach-omap2/io.c b/arch/arm/mach-omap2/io.c index 6a4822dbb4ea65..980c9372e6fd6e 100644 --- a/arch/arm/mach-omap2/io.c +++ b/arch/arm/mach-omap2/io.c @@ -352,6 +352,7 @@ void __init am33xx_map_io(void) void __init omap4_map_io(void) { iotable_init(omap44xx_io_desc, ARRAY_SIZE(omap44xx_io_desc)); + omap_barriers_init(); } #endif @@ -359,6 +360,7 @@ void __init omap4_map_io(void) void __init omap5_map_io(void) { iotable_init(omap54xx_io_desc, ARRAY_SIZE(omap54xx_io_desc)); + omap_barriers_init(); } #endif diff --git a/arch/arm/mach-omap2/omap4-common.c b/arch/arm/mach-omap2/omap4-common.c index 16350eefa66c89..949696b6f17b69 100644 --- a/arch/arm/mach-omap2/omap4-common.c +++ b/arch/arm/mach-omap2/omap4-common.c @@ -51,6 +51,127 @@ static void __iomem *twd_base; #define IRQ_LOCALTIMER 29 +#ifdef CONFIG_OMAP_INTERCONNECT_BARRIER + +/* Used to implement memory barrier on DRAM path */ +#define OMAP4_DRAM_BARRIER_VA 0xfe600000 + +static void __iomem *dram_sync, *sram_sync; +static phys_addr_t dram_sync_paddr; +static u32 dram_sync_size; + +/* + * The OMAP4 bus structure contains asynchrnous bridges which can buffer + * data writes from the MPU. These asynchronous bridges can be found on + * paths between the MPU to EMIF, and the MPU to L3 interconnects. + * + * We need to be careful about re-ordering which can happen as a result + * of different accesses being performed via different paths, and + * therefore different asynchronous bridges. + */ + +/* + * OMAP4 interconnect barrier which is called for each mb() and wmb(). + * This is to ensure that normal paths to DRAM (normal memory, cacheable + * accesses) are properly synchronised with writes to DMA coherent memory + * (normal memory, uncacheable) and device writes. + * + * The mb() and wmb() barriers only operate only on the MPU->MA->EMIF + * path, as we need to ensure that data is visible to other system + * masters prior to writes to those system masters being seen. + * + * Note: the SRAM path is not synchronised via mb() and wmb(). + */ +static void omap4_mb(void) +{ + if (dram_sync) + writel_relaxed(0, dram_sync); +} + +/* + * OMAP4 Errata i688 - asynchronous bridge corruption when entering WFI. + * + * If a data is stalled inside asynchronous bridge because of back + * pressure, it may be accepted multiple times, creating pointer + * misalignment that will corrupt next transfers on that data path until + * next reset of the system. No recovery procedure once the issue is hit, + * the path remains consistently broken. + * + * Async bridges can be found on paths between MPU to EMIF and MPU to L3 + * interconnects. + * + * This situation can happen only when the idle is initiated by a Master + * Request Disconnection (which is trigged by software when executing WFI + * on the CPU). + * + * The work-around for this errata needs all the initiators connected + * through an async bridge to ensure that data path is properly drained + * before issuing WFI. This condition will be met if one Strongly ordered + * access is performed to the target right before executing the WFI. + * + * In MPU case, L3 T2ASYNC FIFO and DDR T2ASYNC FIFO needs to be drained. + * IO barrier ensure that there is no synchronisation loss on initiators + * operating on both interconnect port simultaneously. + * + * This is a stronger version of the OMAP4 memory barrier below, and + * operates on both the MPU->MA->EMIF path but also the MPU->OCP path + * as well, and is necessary prior to executing a WFI. + */ +void omap_interconnect_sync(void) +{ + if (dram_sync && sram_sync) { + writel_relaxed(readl_relaxed(dram_sync), dram_sync); + writel_relaxed(readl_relaxed(sram_sync), sram_sync); + isb(); + } +} + +static int __init omap4_sram_init(void) +{ + struct device_node *np; + struct gen_pool *sram_pool; + + np = of_find_compatible_node(NULL, NULL, "ti,omap4-mpu"); + if (!np) + pr_warn("%s:Unable to allocate sram needed to handle errata I688\n", + __func__); + sram_pool = of_gen_pool_get(np, "sram", 0); + if (!sram_pool) + pr_warn("%s:Unable to get sram pool needed to handle errata I688\n", + __func__); + else + sram_sync = (void *)gen_pool_alloc(sram_pool, PAGE_SIZE); + + return 0; +} +omap_arch_initcall(omap4_sram_init); + +/* Steal one page physical memory for barrier implementation */ +void __init omap_barrier_reserve_memblock(void) +{ + dram_sync_size = ALIGN(PAGE_SIZE, SZ_1M); + dram_sync_paddr = arm_memblock_steal(dram_sync_size, SZ_1M); +} + +void __init omap_barriers_init(void) +{ + struct map_desc dram_io_desc[1]; + + dram_io_desc[0].virtual = OMAP4_DRAM_BARRIER_VA; + dram_io_desc[0].pfn = __phys_to_pfn(dram_sync_paddr); + dram_io_desc[0].length = dram_sync_size; + dram_io_desc[0].type = MT_MEMORY_RW_SO; + iotable_init(dram_io_desc, ARRAY_SIZE(dram_io_desc)); + dram_sync = (void __iomem *) dram_io_desc[0].virtual; + + pr_info("OMAP4: Map %pa to %p for dram barrier\n", + &dram_sync_paddr, dram_sync); + + soc_mb = omap4_mb; +} + +#endif + void gic_dist_disable(void) { if (gic_dist_base_addr) diff --git a/arch/arm/mach-omap2/sleep44xx.S b/arch/arm/mach-omap2/sleep44xx.S index ad1bb9431e941c..9b09d85d811a1c 100644 --- a/arch/arm/mach-omap2/sleep44xx.S +++ b/arch/arm/mach-omap2/sleep44xx.S @@ -333,14 +333,12 @@ ENDPROC(omap4_cpu_resume) #endif /* defined(CONFIG_SMP) && defined(CONFIG_PM) */ -ENTRY(omap_bus_sync) - ret lr -ENDPROC(omap_bus_sync) - ENTRY(omap_do_wfi) stmfd sp!, {lr} +#ifdef CONFIG_OMAP_INTERCONNECT_BARRIER /* Drain interconnect write buffers. */ - bl omap_bus_sync + bl omap_interconnect_sync +#endif /* * Execute an ISB instruction to ensure that all of the diff --git a/arch/arm/mach-prima2/pm.c b/arch/arm/mach-prima2/pm.c index d99d08eeb9664c..83e94c95e31441 100644 --- a/arch/arm/mach-prima2/pm.c +++ b/arch/arm/mach-prima2/pm.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include diff --git a/arch/arm/mach-shmobile/common.h b/arch/arm/mach-shmobile/common.h index 476092b86c6e42..8d27ec546a35f4 100644 --- a/arch/arm/mach-shmobile/common.h +++ b/arch/arm/mach-shmobile/common.h @@ -13,7 +13,7 @@ extern void shmobile_smp_boot(void); extern void shmobile_smp_sleep(void); extern void shmobile_smp_hook(unsigned int cpu, unsigned long fn, unsigned long arg); -extern int shmobile_smp_cpu_disable(unsigned int cpu); +extern bool shmobile_smp_cpu_can_disable(unsigned int cpu); extern void shmobile_boot_scu(void); extern void shmobile_smp_scu_prepare_cpus(unsigned int max_cpus); extern void shmobile_smp_scu_cpu_die(unsigned int cpu); diff --git a/arch/arm/mach-shmobile/platsmp.c b/arch/arm/mach-shmobile/platsmp.c index 3923e09e966d50..b23378f3d7e172 100644 --- a/arch/arm/mach-shmobile/platsmp.c +++ b/arch/arm/mach-shmobile/platsmp.c @@ -31,8 +31,8 @@ void shmobile_smp_hook(unsigned int cpu, unsigned long fn, unsigned long arg) } #ifdef CONFIG_HOTPLUG_CPU -int shmobile_smp_cpu_disable(unsigned int cpu) +bool shmobile_smp_cpu_can_disable(unsigned int cpu) { - return 0; /* Hotplug of any CPU is supported */ + return true; /* Hotplug of any CPU is supported */ } #endif diff --git a/arch/arm/mach-shmobile/smp-r8a7790.c b/arch/arm/mach-shmobile/smp-r8a7790.c index 2ef0054ce934f0..4b33d432a36408 100644 --- a/arch/arm/mach-shmobile/smp-r8a7790.c +++ b/arch/arm/mach-shmobile/smp-r8a7790.c @@ -64,7 +64,7 @@ struct smp_operations r8a7790_smp_ops __initdata = { .smp_prepare_cpus = r8a7790_smp_prepare_cpus, .smp_boot_secondary = shmobile_smp_apmu_boot_secondary, #ifdef CONFIG_HOTPLUG_CPU - .cpu_disable = shmobile_smp_cpu_disable, + .cpu_can_disable = shmobile_smp_cpu_can_disable, .cpu_die = shmobile_smp_apmu_cpu_die, .cpu_kill = shmobile_smp_apmu_cpu_kill, #endif diff --git a/arch/arm/mach-shmobile/smp-r8a7791.c b/arch/arm/mach-shmobile/smp-r8a7791.c index 5e2d1db79afa53..b2508c0d276b46 100644 --- a/arch/arm/mach-shmobile/smp-r8a7791.c +++ b/arch/arm/mach-shmobile/smp-r8a7791.c @@ -58,7 +58,7 @@ struct smp_operations r8a7791_smp_ops __initdata = { .smp_prepare_cpus = r8a7791_smp_prepare_cpus, .smp_boot_secondary = r8a7791_smp_boot_secondary, #ifdef CONFIG_HOTPLUG_CPU - .cpu_disable = shmobile_smp_cpu_disable, + .cpu_can_disable = shmobile_smp_cpu_can_disable, .cpu_die = shmobile_smp_apmu_cpu_die, .cpu_kill = shmobile_smp_apmu_cpu_kill, #endif diff --git a/arch/arm/mach-shmobile/smp-sh73a0.c b/arch/arm/mach-shmobile/smp-sh73a0.c index d03aa11fb46dad..bc2824a036e129 100644 --- a/arch/arm/mach-shmobile/smp-sh73a0.c +++ b/arch/arm/mach-shmobile/smp-sh73a0.c @@ -60,7 +60,7 @@ struct smp_operations sh73a0_smp_ops __initdata = { .smp_prepare_cpus = sh73a0_smp_prepare_cpus, .smp_boot_secondary = sh73a0_boot_secondary, #ifdef CONFIG_HOTPLUG_CPU - .cpu_disable = shmobile_smp_cpu_disable, + .cpu_can_disable = shmobile_smp_cpu_can_disable, .cpu_die = shmobile_smp_scu_cpu_die, .cpu_kill = shmobile_smp_scu_cpu_kill, #endif diff --git a/arch/arm/mach-ux500/cache-l2x0.c b/arch/arm/mach-ux500/cache-l2x0.c index 7557bede7ae677..780bd13cd7e3d9 100644 --- a/arch/arm/mach-ux500/cache-l2x0.c +++ b/arch/arm/mach-ux500/cache-l2x0.c @@ -8,6 +8,7 @@ #include #include +#include #include #include "db8500-regs.h" diff --git a/arch/arm/mach-ux500/cpu-db8500.c b/arch/arm/mach-ux500/cpu-db8500.c index ba708ce08616a4..f80560318c58ca 100644 --- a/arch/arm/mach-ux500/cpu-db8500.c +++ b/arch/arm/mach-ux500/cpu-db8500.c @@ -20,10 +20,10 @@ #include #include #include +#include #include #include -#include #include #include "setup.h" diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig index 7c6b976ab8d39d..df7537f12469a1 100644 --- a/arch/arm/mm/Kconfig +++ b/arch/arm/mm/Kconfig @@ -883,6 +883,7 @@ config OUTER_CACHE config OUTER_CACHE_SYNC bool + select ARM_HEAVY_MB help The outer cache has a outer_cache_fns.sync function pointer that can be used to drain the write buffer of the outer cache. @@ -1031,6 +1032,9 @@ config ARCH_HAS_BARRIERS This option allows the use of custom mandatory barriers included via the mach/barriers.h file. +config ARM_HEAVY_MB + bool + config ARCH_SUPPORTS_BIG_ENDIAN bool help diff --git a/arch/arm/mm/abort-ev4.S b/arch/arm/mm/abort-ev4.S index 54473cd4aba951..b3b31e30cadd20 100644 --- a/arch/arm/mm/abort-ev4.S +++ b/arch/arm/mm/abort-ev4.S @@ -19,6 +19,7 @@ ENTRY(v4_early_abort) mrc p15, 0, r1, c5, c0, 0 @ get FSR mrc p15, 0, r0, c6, c0, 0 @ get FAR ldr r3, [r4] @ read aborted ARM instruction + uaccess_disable ip @ disable userspace access bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r3, #1 << 20 @ L = 1 -> write? orreq r1, r1, #1 << 11 @ yes. diff --git a/arch/arm/mm/abort-ev5t.S b/arch/arm/mm/abort-ev5t.S index a0908d4653a34a..a6a381a6caa5a3 100644 --- a/arch/arm/mm/abort-ev5t.S +++ b/arch/arm/mm/abort-ev5t.S @@ -21,8 +21,10 @@ ENTRY(v5t_early_abort) mrc p15, 0, r0, c6, c0, 0 @ get FAR do_thumb_abort fsr=r1, pc=r4, psr=r5, tmp=r3 ldreq r3, [r4] @ read aborted ARM instruction + uaccess_disable ip @ disable user access bic r1, r1, #1 << 11 @ clear bits 11 of FSR - do_ldrd_abort tmp=ip, insn=r3 + teq_ldrd tmp=ip, insn=r3 @ insn was LDRD? + beq do_DataAbort @ yes tst r3, #1 << 20 @ check write orreq r1, r1, #1 << 11 b do_DataAbort diff --git a/arch/arm/mm/abort-ev5tj.S b/arch/arm/mm/abort-ev5tj.S index 4006b7a612642b..00ab011bef5848 100644 --- a/arch/arm/mm/abort-ev5tj.S +++ b/arch/arm/mm/abort-ev5tj.S @@ -24,7 +24,9 @@ ENTRY(v5tj_early_abort) bne do_DataAbort do_thumb_abort fsr=r1, pc=r4, psr=r5, tmp=r3 ldreq r3, [r4] @ read aborted ARM instruction - do_ldrd_abort tmp=ip, insn=r3 + uaccess_disable ip @ disable userspace access + teq_ldrd tmp=ip, insn=r3 @ insn was LDRD? + beq do_DataAbort @ yes tst r3, #1 << 20 @ L = 0 -> write orreq r1, r1, #1 << 11 @ yes. b do_DataAbort diff --git a/arch/arm/mm/abort-ev6.S b/arch/arm/mm/abort-ev6.S index 8c48c5c22a331a..8801a15aa10595 100644 --- a/arch/arm/mm/abort-ev6.S +++ b/arch/arm/mm/abort-ev6.S @@ -26,16 +26,18 @@ ENTRY(v6_early_abort) ldr ip, =0x4107b36 mrc p15, 0, r3, c0, c0, 0 @ get processor id teq ip, r3, lsr #4 @ r0 ARM1136? - bne do_DataAbort + bne 1f tst r5, #PSR_J_BIT @ Java? tsteq r5, #PSR_T_BIT @ Thumb? - bne do_DataAbort + bne 1f bic r1, r1, #1 << 11 @ clear bit 11 of FSR ldr r3, [r4] @ read aborted ARM instruction ARM_BE8(rev r3, r3) - do_ldrd_abort tmp=ip, insn=r3 + teq_ldrd tmp=ip, insn=r3 @ insn was LDRD? + beq 1f @ yes tst r3, #1 << 20 @ L = 0 -> write orreq r1, r1, #1 << 11 @ yes. #endif +1: uaccess_disable ip @ disable userspace access b do_DataAbort diff --git a/arch/arm/mm/abort-ev7.S b/arch/arm/mm/abort-ev7.S index 4812ad05421457..e8d0e08c227fc5 100644 --- a/arch/arm/mm/abort-ev7.S +++ b/arch/arm/mm/abort-ev7.S @@ -15,6 +15,7 @@ ENTRY(v7_early_abort) mrc p15, 0, r1, c5, c0, 0 @ get FSR mrc p15, 0, r0, c6, c0, 0 @ get FAR + uaccess_disable ip @ disable userspace access /* * V6 code adjusts the returned DFSR. diff --git a/arch/arm/mm/abort-lv4t.S b/arch/arm/mm/abort-lv4t.S index f3982580c27305..6d8e8e3365d173 100644 --- a/arch/arm/mm/abort-lv4t.S +++ b/arch/arm/mm/abort-lv4t.S @@ -26,6 +26,7 @@ ENTRY(v4t_late_abort) #endif bne .data_thumb_abort ldr r8, [r4] @ read arm instruction + uaccess_disable ip @ disable userspace access tst r8, #1 << 20 @ L = 1 -> write? orreq r1, r1, #1 << 11 @ yes. and r7, r8, #15 << 24 @@ -155,6 +156,7 @@ ENTRY(v4t_late_abort) .data_thumb_abort: ldrh r8, [r4] @ read instruction + uaccess_disable ip @ disable userspace access tst r8, #1 << 11 @ L = 1 -> write? orreq r1, r1, #1 << 8 @ yes and r7, r8, #15 << 12 diff --git a/arch/arm/mm/abort-macro.S b/arch/arm/mm/abort-macro.S index 2cbf68ef0e8321..4509bee4e081ce 100644 --- a/arch/arm/mm/abort-macro.S +++ b/arch/arm/mm/abort-macro.S @@ -13,6 +13,7 @@ tst \psr, #PSR_T_BIT beq not_thumb ldrh \tmp, [\pc] @ Read aborted Thumb instruction + uaccess_disable ip @ disable userspace access and \tmp, \tmp, # 0xfe00 @ Mask opcode field cmp \tmp, # 0x5600 @ Is it ldrsb? orreq \tmp, \tmp, #1 << 11 @ Set L-bit if yes @@ -29,12 +30,9 @@ not_thumb: * [7:4] == 1101 * [20] == 0 */ - .macro do_ldrd_abort, tmp, insn - tst \insn, #0x0e100000 @ [27:25,20] == 0 - bne not_ldrd - and \tmp, \insn, #0x000000f0 @ [7:4] == 1101 - cmp \tmp, #0x000000d0 - beq do_DataAbort -not_ldrd: + .macro teq_ldrd, tmp, insn + mov \tmp, #0x0e100000 + orr \tmp, #0x000000f0 + and \tmp, \insn, \tmp + teq \tmp, #0x000000d0 .endm - diff --git a/arch/arm/mm/cache-feroceon-l2.c b/arch/arm/mm/cache-feroceon-l2.c index 097181e08c25f7..5c1b7a7b9af630 100644 --- a/arch/arm/mm/cache-feroceon-l2.c +++ b/arch/arm/mm/cache-feroceon-l2.c @@ -368,7 +368,6 @@ int __init feroceon_of_init(void) struct device_node *node; void __iomem *base; bool l2_wt_override = false; - struct resource res; #if defined(CONFIG_CACHE_FEROCEON_L2_WRITETHROUGH) l2_wt_override = true; @@ -376,10 +375,7 @@ int __init feroceon_of_init(void) node = of_find_matching_node(NULL, feroceon_ids); if (node && of_device_is_compatible(node, "marvell,kirkwood-cache")) { - if (of_address_to_resource(node, 0, &res)) - return -ENODEV; - - base = ioremap(res.start, resource_size(&res)); + base = of_iomap(node, 0); if (!base) return -ENOMEM; diff --git a/arch/arm/mm/cache-l2x0.c b/arch/arm/mm/cache-l2x0.c index 71b3d330902449..493692d838c679 100644 --- a/arch/arm/mm/cache-l2x0.c +++ b/arch/arm/mm/cache-l2x0.c @@ -1171,6 +1171,11 @@ static void __init l2c310_of_parse(const struct device_node *np, } } + if (of_property_read_bool(np, "arm,shared-override")) { + *aux_val |= L2C_AUX_CTRL_SHARED_OVERRIDE; + *aux_mask &= ~L2C_AUX_CTRL_SHARED_OVERRIDE; + } + prefetch = l2x0_saved_regs.prefetch_ctrl; ret = of_property_read_u32(np, "arm,double-linefill", &val); diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c index 3d3d6aa60c8741..bf35abcc7d598c 100644 --- a/arch/arm/mm/dma-mapping.c +++ b/arch/arm/mm/dma-mapping.c @@ -39,6 +39,7 @@ #include #include +#include "dma.h" #include "mm.h" /* @@ -648,14 +649,18 @@ static void *__dma_alloc(struct device *dev, size_t size, dma_addr_t *handle, size = PAGE_ALIGN(size); want_vaddr = !dma_get_attr(DMA_ATTR_NO_KERNEL_MAPPING, attrs); - if (is_coherent || nommu()) + if (nommu()) + addr = __alloc_simple_buffer(dev, size, gfp, &page); + else if (dev_get_cma_area(dev) && (gfp & __GFP_WAIT)) + addr = __alloc_from_contiguous(dev, size, prot, &page, + caller, want_vaddr); + else if (is_coherent) addr = __alloc_simple_buffer(dev, size, gfp, &page); else if (!(gfp & __GFP_WAIT)) addr = __alloc_from_pool(size, &page); - else if (!dev_get_cma_area(dev)) - addr = __alloc_remap_buffer(dev, size, gfp, prot, &page, caller, want_vaddr); else - addr = __alloc_from_contiguous(dev, size, prot, &page, caller, want_vaddr); + addr = __alloc_remap_buffer(dev, size, gfp, prot, &page, + caller, want_vaddr); if (page) *handle = pfn_to_dma(dev, page_to_pfn(page)); @@ -683,13 +688,12 @@ void *arm_dma_alloc(struct device *dev, size_t size, dma_addr_t *handle, static void *arm_coherent_dma_alloc(struct device *dev, size_t size, dma_addr_t *handle, gfp_t gfp, struct dma_attrs *attrs) { - pgprot_t prot = __get_dma_pgprot(attrs, PAGE_KERNEL); void *memory; if (dma_alloc_from_coherent(dev, size, handle, &memory)) return memory; - return __dma_alloc(dev, size, handle, gfp, prot, true, + return __dma_alloc(dev, size, handle, gfp, PAGE_KERNEL, true, attrs, __builtin_return_address(0)); } @@ -753,12 +757,12 @@ static void __arm_dma_free(struct device *dev, size_t size, void *cpu_addr, size = PAGE_ALIGN(size); - if (is_coherent || nommu()) { + if (nommu()) { __dma_free_buffer(page, size); - } else if (__free_from_pool(cpu_addr, size)) { + } else if (!is_coherent && __free_from_pool(cpu_addr, size)) { return; } else if (!dev_get_cma_area(dev)) { - if (want_vaddr) + if (want_vaddr && !is_coherent) __dma_free_remap(cpu_addr, size); __dma_free_buffer(page, size); } else { diff --git a/arch/arm/mm/dma.h b/arch/arm/mm/dma.h new file mode 100644 index 00000000000000..70ea6852f94e41 --- /dev/null +++ b/arch/arm/mm/dma.h @@ -0,0 +1,32 @@ +#ifndef DMA_H +#define DMA_H + +#include + +#ifndef MULTI_CACHE +#define dmac_map_area __glue(_CACHE,_dma_map_area) +#define dmac_unmap_area __glue(_CACHE,_dma_unmap_area) + +/* + * These are private to the dma-mapping API. Do not use directly. + * Their sole purpose is to ensure that data held in the cache + * is visible to DMA, or data written by DMA to system memory is + * visible to the CPU. + */ +extern void dmac_map_area(const void *, size_t, int); +extern void dmac_unmap_area(const void *, size_t, int); + +#else + +/* + * These are private to the dma-mapping API. Do not use directly. + * Their sole purpose is to ensure that data held in the cache + * is visible to DMA, or data written by DMA to system memory is + * visible to the CPU. + */ +#define dmac_map_area cpu_cache.dma_map_area +#define dmac_unmap_area cpu_cache.dma_unmap_area + +#endif + +#endif diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c index 34b66af516ea9e..1ec8e7590fc682 100644 --- a/arch/arm/mm/flush.c +++ b/arch/arm/mm/flush.c @@ -21,6 +21,21 @@ #include "mm.h" +#ifdef CONFIG_ARM_HEAVY_MB +void (*soc_mb)(void); + +void arm_heavy_mb(void) +{ +#ifdef CONFIG_OUTER_CACHE_SYNC + if (outer_cache.sync) + outer_cache.sync(); +#endif + if (soc_mb) + soc_mb(); +} +EXPORT_SYMBOL(arm_heavy_mb); +#endif + #ifdef CONFIG_CPU_CACHE_VIPT static void flush_pfn_alias(unsigned long pfn, unsigned long vaddr) diff --git a/arch/arm/mm/highmem.c b/arch/arm/mm/highmem.c index ee8dfa79398978..9df5f09585ca49 100644 --- a/arch/arm/mm/highmem.c +++ b/arch/arm/mm/highmem.c @@ -79,7 +79,7 @@ void *kmap_atomic(struct page *page) type = kmap_atomic_idx_push(); - idx = type + KM_TYPE_NR * smp_processor_id(); + idx = FIX_KMAP_BEGIN + type + KM_TYPE_NR * smp_processor_id(); vaddr = __fix_to_virt(idx); #ifdef CONFIG_DEBUG_HIGHMEM /* @@ -106,7 +106,7 @@ void __kunmap_atomic(void *kvaddr) if (kvaddr >= (void *)FIXADDR_START) { type = kmap_atomic_idx(); - idx = type + KM_TYPE_NR * smp_processor_id(); + idx = FIX_KMAP_BEGIN + type + KM_TYPE_NR * smp_processor_id(); if (cache_is_vivt()) __cpuc_flush_dcache_area((void *)vaddr, PAGE_SIZE); @@ -138,7 +138,7 @@ void *kmap_atomic_pfn(unsigned long pfn) return page_address(page); type = kmap_atomic_idx_push(); - idx = type + KM_TYPE_NR * smp_processor_id(); + idx = FIX_KMAP_BEGIN + type + KM_TYPE_NR * smp_processor_id(); vaddr = __fix_to_virt(idx); #ifdef CONFIG_DEBUG_HIGHMEM BUG_ON(!pte_none(get_fixmap_pte(vaddr))); diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c index 870838a46d5241..7cd15143a50774 100644 --- a/arch/arm/mm/mmu.c +++ b/arch/arm/mm/mmu.c @@ -291,13 +291,13 @@ static struct mem_type mem_types[] = { .prot_pte = L_PTE_PRESENT | L_PTE_YOUNG | L_PTE_DIRTY | L_PTE_RDONLY, .prot_l1 = PMD_TYPE_TABLE, - .domain = DOMAIN_USER, + .domain = DOMAIN_VECTORS, }, [MT_HIGH_VECTORS] = { .prot_pte = L_PTE_PRESENT | L_PTE_YOUNG | L_PTE_DIRTY | L_PTE_USER | L_PTE_RDONLY, .prot_l1 = PMD_TYPE_TABLE, - .domain = DOMAIN_USER, + .domain = DOMAIN_VECTORS, }, [MT_MEMORY_RWX] = { .prot_pte = L_PTE_PRESENT | L_PTE_YOUNG | L_PTE_DIRTY, @@ -357,6 +357,47 @@ const struct mem_type *get_mem_type(unsigned int type) } EXPORT_SYMBOL(get_mem_type); +static pte_t *(*pte_offset_fixmap)(pmd_t *dir, unsigned long addr); + +static pte_t bm_pte[PTRS_PER_PTE + PTE_HWTABLE_PTRS] + __aligned(PTE_HWTABLE_OFF + PTE_HWTABLE_SIZE) __initdata; + +static pte_t * __init pte_offset_early_fixmap(pmd_t *dir, unsigned long addr) +{ + return &bm_pte[pte_index(addr)]; +} + +static pte_t *pte_offset_late_fixmap(pmd_t *dir, unsigned long addr) +{ + return pte_offset_kernel(dir, addr); +} + +static inline pmd_t * __init fixmap_pmd(unsigned long addr) +{ + pgd_t *pgd = pgd_offset_k(addr); + pud_t *pud = pud_offset(pgd, addr); + pmd_t *pmd = pmd_offset(pud, addr); + + return pmd; +} + +void __init early_fixmap_init(void) +{ + pmd_t *pmd; + + /* + * The early fixmap range spans multiple pmds, for which + * we are not prepared: + */ + BUILD_BUG_ON((__fix_to_virt(__end_of_permanent_fixed_addresses) >> PMD_SHIFT) + != FIXADDR_TOP >> PMD_SHIFT); + + pmd = fixmap_pmd(FIXADDR_TOP); + pmd_populate_kernel(&init_mm, pmd, bm_pte); + + pte_offset_fixmap = pte_offset_early_fixmap; +} + /* * To avoid TLB flush broadcasts, this uses local_flush_tlb_kernel_range(). * As a result, this can only be called with preemption disabled, as under @@ -365,7 +406,7 @@ EXPORT_SYMBOL(get_mem_type); void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t prot) { unsigned long vaddr = __fix_to_virt(idx); - pte_t *pte = pte_offset_kernel(pmd_off_k(vaddr), vaddr); + pte_t *pte = pte_offset_fixmap(pmd_off_k(vaddr), vaddr); /* Make sure fixmap region does not exceed available allocation. */ BUILD_BUG_ON(FIXADDR_START + (__end_of_fixed_addresses * PAGE_SIZE) > @@ -855,7 +896,7 @@ static void __init create_mapping(struct map_desc *md) } if ((md->type == MT_DEVICE || md->type == MT_ROM) && - md->virtual >= PAGE_OFFSET && + md->virtual >= PAGE_OFFSET && md->virtual < FIXADDR_START && (md->virtual < VMALLOC_START || md->virtual >= VMALLOC_END)) { pr_warn("BUG: mapping for 0x%08llx at 0x%08lx out of vmalloc space\n", (long long)__pfn_to_phys((u64)md->pfn), md->virtual); @@ -1219,10 +1260,10 @@ void __init arm_mm_memblock_reserve(void) /* * Set up the device mappings. Since we clear out the page tables for all - * mappings above VMALLOC_START, we will remove any debug device mappings. - * This means you have to be careful how you debug this function, or any - * called function. This means you can't use any function or debugging - * method which may touch any device, otherwise the kernel _will_ crash. + * mappings above VMALLOC_START, except early fixmap, we might remove debug + * device mappings. This means earlycon can be used to debug this function + * Any other function or debugging method which may touch any device _will_ + * crash the kernel. */ static void __init devicemaps_init(const struct machine_desc *mdesc) { @@ -1237,7 +1278,10 @@ static void __init devicemaps_init(const struct machine_desc *mdesc) early_trap_init(vectors); - for (addr = VMALLOC_START; addr; addr += PMD_SIZE) + /* + * Clear page table except top pmd used by early fixmaps + */ + for (addr = VMALLOC_START; addr < (FIXADDR_TOP & PMD_MASK); addr += PMD_SIZE) pmd_clear(pmd_off_k(addr)); /* @@ -1489,6 +1533,35 @@ void __init early_paging_init(const struct machine_desc *mdesc) #endif +static void __init early_fixmap_shutdown(void) +{ + int i; + unsigned long va = fix_to_virt(__end_of_permanent_fixed_addresses - 1); + + pte_offset_fixmap = pte_offset_late_fixmap; + pmd_clear(fixmap_pmd(va)); + local_flush_tlb_kernel_page(va); + + for (i = 0; i < __end_of_permanent_fixed_addresses; i++) { + pte_t *pte; + struct map_desc map; + + map.virtual = fix_to_virt(i); + pte = pte_offset_early_fixmap(pmd_off_k(map.virtual), map.virtual); + + /* Only i/o device mappings are supported ATM */ + if (pte_none(*pte) || + (pte_val(*pte) & L_PTE_MT_MASK) != L_PTE_MT_DEV_SHARED) + continue; + + map.pfn = pte_pfn(*pte); + map.type = MT_DEVICE; + map.length = PAGE_SIZE; + + create_mapping(&map); + } +} + /* * paging_init() sets up the page tables, initialises the zone memory * maps, and sets up the zero page, bad page and bad page tables. @@ -1502,6 +1575,7 @@ void __init paging_init(const struct machine_desc *mdesc) map_lowmem(); memblock_set_current_limit(arm_lowmem_limit); dma_contiguous_remap(); + early_fixmap_shutdown(); devicemaps_init(mdesc); kmap_init(); tcm_init(); diff --git a/arch/arm/mm/pgd.c b/arch/arm/mm/pgd.c index a3681f11dd9f12..e683db1b90a3f8 100644 --- a/arch/arm/mm/pgd.c +++ b/arch/arm/mm/pgd.c @@ -84,6 +84,16 @@ pgd_t *pgd_alloc(struct mm_struct *mm) if (!new_pte) goto no_pte; +#ifndef CONFIG_ARM_LPAE + /* + * Modify the PTE pointer to have the correct domain. This + * needs to be the vectors domain to avoid the low vectors + * being unmapped. + */ + pmd_val(*new_pmd) &= ~PMD_DOMAIN_MASK; + pmd_val(*new_pmd) |= PMD_DOMAIN(DOMAIN_VECTORS); +#endif + init_pud = pud_offset(init_pgd, 0); init_pmd = pmd_offset(init_pud, 0); init_pte = pte_offset_map(init_pmd, 0); diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b7b9ceaa684ad4..7d95663c0160bd 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -20,6 +20,7 @@ config ARM64 select ARM_GIC_V2M if PCI_MSI select ARM_GIC_V3 select ARM_GIC_V3_ITS if PCI_MSI + select ARM_PSCI_FW select BUILDTIME_EXTABLE_SORT select CLONE_BACKWARDS select COMMON_CLK @@ -28,7 +29,7 @@ config ARM64 select EDAC_SUPPORT select GENERIC_ALLOCATOR select GENERIC_CLOCKEVENTS - select GENERIC_CLOCKEVENTS_BROADCAST if SMP + select GENERIC_CLOCKEVENTS_BROADCAST select GENERIC_CPU_AUTOPROBE select GENERIC_EARLY_IOREMAP select GENERIC_IRQ_PROBE @@ -53,6 +54,7 @@ config ARM64 select HAVE_C_RECORDMCOUNT select HAVE_CC_STACKPROTECTOR select HAVE_CMPXCHG_DOUBLE + select HAVE_CMPXCHG_LOCAL select HAVE_DEBUG_BUGVERBOSE select HAVE_DEBUG_KMEMLEAK select HAVE_DMA_API_DEBUG @@ -104,6 +106,10 @@ config NO_IOPORT_MAP config STACKTRACE_SUPPORT def_bool y +config ILLEGAL_POINTER_VALUE + hex + default 0xdead000000000000 + config LOCKDEP_SUPPORT def_bool y @@ -113,6 +119,14 @@ config TRACE_IRQFLAGS_SUPPORT config RWSEM_XCHGADD_ALGORITHM def_bool y +config GENERIC_BUG + def_bool y + depends on BUG + +config GENERIC_BUG_RELATIVE_POINTERS + def_bool y + depends on GENERIC_BUG + config GENERIC_HWEIGHT def_bool y @@ -137,6 +151,9 @@ config NEED_DMA_MAP_STATE config NEED_SG_DMA_LENGTH def_bool y +config SMP + def_bool y + config SWIOTLB def_bool y @@ -371,22 +388,8 @@ config CPU_BIG_ENDIAN help Say Y if you plan on running a kernel in big-endian mode. -config SMP - bool "Symmetric Multi-Processing" - help - This enables support for systems with more than one CPU. If - you say N here, the kernel will run on single and - multiprocessor machines, but will use only one CPU of a - multiprocessor machine. If you say Y here, the kernel will run - on many, but not all, single processor machines. On a single - processor machine, the kernel will run faster if you say N - here. - - If you don't know what to do here, say N. - config SCHED_MC bool "Multi-core scheduler support" - depends on SMP help Multi-core scheduler support improves the CPU scheduler's decision making when dealing with multi-core CPU chips at a cost of slightly @@ -394,7 +397,6 @@ config SCHED_MC config SCHED_SMT bool "SMT scheduler support" - depends on SMP help Improves the CPU scheduler's decision making when dealing with MultiThreading at a cost of slightly increased overhead in some @@ -403,23 +405,17 @@ config SCHED_SMT config NR_CPUS int "Maximum number of CPUs (2-4096)" range 2 4096 - depends on SMP # These have to remain sorted largest to smallest default "64" config HOTPLUG_CPU bool "Support for hot-pluggable CPUs" - depends on SMP help Say Y here to experiment with turning CPUs off and on. CPUs can be controlled through /sys/devices/system/cpu. source kernel/Kconfig.preempt -config UP_LATE_INIT - def_bool y - depends on !SMP - config HZ int default 100 @@ -561,6 +557,53 @@ config SETEND_EMULATION If unsure, say Y endif +menu "ARMv8.1 architectural features" + +config ARM64_HW_AFDBM + bool "Support for hardware updates of the Access and Dirty page flags" + default y + help + The ARMv8.1 architecture extensions introduce support for + hardware updates of the access and dirty information in page + table entries. When enabled in TCR_EL1 (HA and HD bits) on + capable processors, accesses to pages with PTE_AF cleared will + set this bit instead of raising an access flag fault. + Similarly, writes to read-only pages with the DBM bit set will + clear the read-only bit (AP[2]) instead of raising a + permission fault. + + Kernels built with this configuration option enabled continue + to work on pre-ARMv8.1 hardware and the performance impact is + minimal. If unsure, say Y. + +config ARM64_PAN + bool "Enable support for Privileged Access Never (PAN)" + default y + help + Privileged Access Never (PAN; part of the ARMv8.1 Extensions) + prevents the kernel or hypervisor from accessing user-space (EL0) + memory directly. + + Choosing this option will cause any unprotected (not using + copy_to_user et al) memory access to fail with a permission fault. + + The feature is detected at runtime, and will remain as a 'nop' + instruction if the cpu does not implement the feature. + +config ARM64_LSE_ATOMICS + bool "Atomic instructions" + help + As part of the Large System Extensions, ARMv8.1 introduces new + atomic instructions that are designed specifically to scale in + very large systems. + + Say Y here to make use of these instructions for the in-kernel + atomic routines. This incurs a small overhead on CPUs that do + not support these instructions and requires the kernel to be + built with binutils >= 2.25. + +endmenu + endmenu menu "Boot options" diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 4d2a925998f925..15ff5b4156fd74 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -17,7 +17,18 @@ GZFLAGS :=-9 KBUILD_DEFCONFIG := defconfig -KBUILD_CFLAGS += -mgeneral-regs-only +# Check for binutils support for specific extensions +lseinstr := $(call as-instr,.arch_extension lse,-DCONFIG_AS_LSE=1) + +ifeq ($(CONFIG_ARM64_LSE_ATOMICS), y) + ifeq ($(lseinstr),) +$(warning LSE atomics not supported by binutils) + endif +endif + +KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) +KBUILD_AFLAGS += $(lseinstr) + ifeq ($(CONFIG_CPU_BIG_ENDIAN), y) KBUILD_CPPFLAGS += -mbig-endian AS += -EB @@ -58,7 +69,10 @@ all: $(KBUILD_IMAGE) $(KBUILD_DTBS) boot := arch/arm64/boot -Image Image.gz: vmlinux +Image: vmlinux + $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ + +Image.%: vmlinux $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ zinstall install: vmlinux diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile index 5a0e3ab854a574..abcbba2f01baad 100644 --- a/arch/arm64/boot/Makefile +++ b/arch/arm64/boot/Makefile @@ -19,9 +19,21 @@ targets := Image Image.gz $(obj)/Image: vmlinux FORCE $(call if_changed,objcopy) +$(obj)/Image.bz2: $(obj)/Image FORCE + $(call if_changed,bzip2) + $(obj)/Image.gz: $(obj)/Image FORCE $(call if_changed,gzip) +$(obj)/Image.lz4: $(obj)/Image FORCE + $(call if_changed,lz4) + +$(obj)/Image.lzma: $(obj)/Image FORCE + $(call if_changed,lzma) + +$(obj)/Image.lzo: $(obj)/Image FORCE + $(call if_changed,lzo) + install: $(obj)/Image $(CONFIG_SHELL) $(srctree)/$(src)/install.sh $(KERNELRELEASE) \ $(obj)/Image System.map "$(INSTALL_PATH)" diff --git a/arch/arm64/include/asm/acpi.h b/arch/arm64/include/asm/acpi.h index 406485ed110af9..208cec08a74fa0 100644 --- a/arch/arm64/include/asm/acpi.h +++ b/arch/arm64/include/asm/acpi.h @@ -12,11 +12,11 @@ #ifndef _ASM_ACPI_H #define _ASM_ACPI_H -#include #include +#include +#include #include -#include #include /* Macros for consistency checks of the GICC subtable of MADT */ diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h index c385a0c4057f03..d56ec07151570e 100644 --- a/arch/arm64/include/asm/alternative.h +++ b/arch/arm64/include/asm/alternative.h @@ -3,6 +3,8 @@ #ifndef __ASSEMBLY__ +#include +#include #include #include #include @@ -15,7 +17,7 @@ struct alt_instr { u8 alt_len; /* size of new instruction(s), <= orig_len */ }; -void apply_alternatives_all(void); +void __init apply_alternatives_all(void); void apply_alternatives(void *start, size_t length); void free_alternatives_memory(void); @@ -40,7 +42,8 @@ void free_alternatives_memory(void); * be fixed in a binutils release posterior to 2.25.51.0.2 (anything * containing commit 4e4d08cf7399b606 or c1baaddf8861). */ -#define ALTERNATIVE(oldinstr, newinstr, feature) \ +#define __ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg_enabled) \ + ".if "__stringify(cfg_enabled)" == 1\n" \ "661:\n\t" \ oldinstr "\n" \ "662:\n" \ @@ -53,7 +56,11 @@ void free_alternatives_memory(void); "664:\n\t" \ ".popsection\n\t" \ ".org . - (664b-663b) + (662b-661b)\n\t" \ - ".org . - (662b-661b) + (664b-663b)\n" + ".org . - (662b-661b) + (664b-663b)\n" \ + ".endif\n" + +#define _ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg, ...) \ + __ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg)) #else @@ -65,7 +72,8 @@ void free_alternatives_memory(void); .byte \alt_len .endm -.macro alternative_insn insn1 insn2 cap +.macro alternative_insn insn1, insn2, cap, enable = 1 + .if \enable 661: \insn1 662: .pushsection .altinstructions, "a" altinstruction_entry 661b, 663f, \cap, 662b-661b, 664f-663f @@ -75,8 +83,70 @@ void free_alternatives_memory(void); 664: .popsection .org . - (664b-663b) + (662b-661b) .org . - (662b-661b) + (664b-663b) + .endif +.endm + +/* + * Begin an alternative code sequence. + * + * The code that follows this macro will be assembled and linked as + * normal. There are no restrictions on this code. + */ +.macro alternative_if_not cap, enable = 1 + .if \enable + .pushsection .altinstructions, "a" + altinstruction_entry 661f, 663f, \cap, 662f-661f, 664f-663f + .popsection +661: + .endif +.endm + +/* + * Provide the alternative code sequence. + * + * The code that follows this macro is assembled into a special + * section to be used for dynamic patching. Code that follows this + * macro must: + * + * 1. Be exactly the same length (in bytes) as the default code + * sequence. + * + * 2. Not contain a branch target that is used outside of the + * alternative sequence it is defined in (branches into an + * alternative sequence are not fixed up). + */ +.macro alternative_else, enable = 1 + .if \enable +662: .pushsection .altinstr_replacement, "ax" +663: + .endif .endm +/* + * Complete an alternative code sequence. + */ +.macro alternative_endif, enable = 1 + .if \enable +664: .popsection + .org . - (664b-663b) + (662b-661b) + .org . - (662b-661b) + (664b-663b) + .endif +.endm + +#define _ALTERNATIVE_CFG(insn1, insn2, cap, cfg, ...) \ + alternative_insn insn1, insn2, cap, IS_ENABLED(cfg) + + #endif /* __ASSEMBLY__ */ +/* + * Usage: asm(ALTERNATIVE(oldinstr, newinstr, feature)); + * + * Usage: asm(ALTERNATIVE(oldinstr, newinstr, feature, CONFIG_FOO)); + * N.B. If CONFIG_FOO is specified, but not selected, the whole block + * will be omitted, including oldinstr. + */ +#define ALTERNATIVE(oldinstr, newinstr, ...) \ + _ALTERNATIVE_CFG(oldinstr, newinstr, __VA_ARGS__, 1) + #endif /* __ASM_ALTERNATIVE_H */ diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 144b64ad96c33b..b51f2cc22ca997 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -49,18 +49,6 @@ msr daifclr, #2 .endm -/* - * Save/disable and restore interrupts. - */ - .macro save_and_disable_irqs, olddaif - mrs \olddaif, daif - disable_irq - .endm - - .macro restore_irqs, olddaif - msr daif, \olddaif - .endm - /* * Enable and disable debug exceptions. */ @@ -103,9 +91,7 @@ * SMP data memory barrier */ .macro smp_dmb, opt -#ifdef CONFIG_SMP dmb \opt -#endif .endm #define USER(l, x...) \ diff --git a/arch/arm64/include/asm/atomic.h b/arch/arm64/include/asm/atomic.h index 7047051ded40e3..35a67783cfa088 100644 --- a/arch/arm64/include/asm/atomic.h +++ b/arch/arm64/include/asm/atomic.h @@ -24,233 +24,72 @@ #include #include -#include - -#define ATOMIC_INIT(i) { (i) } +#include #ifdef __KERNEL__ -/* - * On ARM, ordinary assignment (str instruction) doesn't clear the local - * strex/ldrex monitor on some implementations. The reason we can use it for - * atomic_set() is the clrex or dummy strex done on every exception return. - */ -#define atomic_read(v) ACCESS_ONCE((v)->counter) -#define atomic_set(v,i) (((v)->counter) = (i)) - -/* - * AArch64 UP and SMP safe atomic ops. We use load exclusive and - * store exclusive to ensure that these are atomic. We may loop - * to ensure that the update happens. - */ - -#define ATOMIC_OP(op, asm_op) \ -static inline void atomic_##op(int i, atomic_t *v) \ -{ \ - unsigned long tmp; \ - int result; \ - \ - asm volatile("// atomic_" #op "\n" \ -"1: ldxr %w0, %2\n" \ -" " #asm_op " %w0, %w0, %w3\n" \ -" stxr %w1, %w0, %2\n" \ -" cbnz %w1, 1b" \ - : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \ - : "Ir" (i)); \ -} \ +#define __ARM64_IN_ATOMIC_IMPL -#define ATOMIC_OP_RETURN(op, asm_op) \ -static inline int atomic_##op##_return(int i, atomic_t *v) \ -{ \ - unsigned long tmp; \ - int result; \ - \ - asm volatile("// atomic_" #op "_return\n" \ -"1: ldxr %w0, %2\n" \ -" " #asm_op " %w0, %w0, %w3\n" \ -" stlxr %w1, %w0, %2\n" \ -" cbnz %w1, 1b" \ - : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \ - : "Ir" (i) \ - : "memory"); \ - \ - smp_mb(); \ - return result; \ -} - -#define ATOMIC_OPS(op, asm_op) \ - ATOMIC_OP(op, asm_op) \ - ATOMIC_OP_RETURN(op, asm_op) - -ATOMIC_OPS(add, add) -ATOMIC_OPS(sub, sub) - -#undef ATOMIC_OPS -#undef ATOMIC_OP_RETURN -#undef ATOMIC_OP - -static inline int atomic_cmpxchg(atomic_t *ptr, int old, int new) -{ - unsigned long tmp; - int oldval; - - smp_mb(); - - asm volatile("// atomic_cmpxchg\n" -"1: ldxr %w1, %2\n" -" cmp %w1, %w3\n" -" b.ne 2f\n" -" stxr %w0, %w4, %2\n" -" cbnz %w0, 1b\n" -"2:" - : "=&r" (tmp), "=&r" (oldval), "+Q" (ptr->counter) - : "Ir" (old), "r" (new) - : "cc"); - - smp_mb(); - return oldval; -} - -#define atomic_xchg(v, new) (xchg(&((v)->counter), new)) +#if defined(CONFIG_ARM64_LSE_ATOMICS) && defined(CONFIG_AS_LSE) +#include +#else +#include +#endif -static inline int __atomic_add_unless(atomic_t *v, int a, int u) -{ - int c, old; +#undef __ARM64_IN_ATOMIC_IMPL - c = atomic_read(v); - while (c != u && (old = atomic_cmpxchg((v), c, c + a)) != c) - c = old; - return c; -} +#include -#define atomic_inc(v) atomic_add(1, v) -#define atomic_dec(v) atomic_sub(1, v) +#define ___atomic_add_unless(v, a, u, sfx) \ +({ \ + typeof((v)->counter) c, old; \ + \ + c = atomic##sfx##_read(v); \ + while (c != (u) && \ + (old = atomic##sfx##_cmpxchg((v), c, c + (a))) != c) \ + c = old; \ + c; \ + }) -#define atomic_inc_and_test(v) (atomic_add_return(1, v) == 0) -#define atomic_dec_and_test(v) (atomic_sub_return(1, v) == 0) -#define atomic_inc_return(v) (atomic_add_return(1, v)) -#define atomic_dec_return(v) (atomic_sub_return(1, v)) -#define atomic_sub_and_test(i, v) (atomic_sub_return(i, v) == 0) +#define ATOMIC_INIT(i) { (i) } -#define atomic_add_negative(i,v) (atomic_add_return(i, v) < 0) +#define atomic_read(v) READ_ONCE((v)->counter) +#define atomic_set(v, i) (((v)->counter) = (i)) +#define atomic_xchg(v, new) xchg(&((v)->counter), (new)) +#define atomic_cmpxchg(v, old, new) cmpxchg(&((v)->counter), (old), (new)) + +#define atomic_inc(v) atomic_add(1, (v)) +#define atomic_dec(v) atomic_sub(1, (v)) +#define atomic_inc_return(v) atomic_add_return(1, (v)) +#define atomic_dec_return(v) atomic_sub_return(1, (v)) +#define atomic_inc_and_test(v) (atomic_inc_return(v) == 0) +#define atomic_dec_and_test(v) (atomic_dec_return(v) == 0) +#define atomic_sub_and_test(i, v) (atomic_sub_return((i), (v)) == 0) +#define atomic_add_negative(i, v) (atomic_add_return((i), (v)) < 0) +#define __atomic_add_unless(v, a, u) ___atomic_add_unless(v, a, u,) +#define atomic_andnot atomic_andnot /* * 64-bit atomic operations. */ -#define ATOMIC64_INIT(i) { (i) } - -#define atomic64_read(v) ACCESS_ONCE((v)->counter) -#define atomic64_set(v,i) (((v)->counter) = (i)) - -#define ATOMIC64_OP(op, asm_op) \ -static inline void atomic64_##op(long i, atomic64_t *v) \ -{ \ - long result; \ - unsigned long tmp; \ - \ - asm volatile("// atomic64_" #op "\n" \ -"1: ldxr %0, %2\n" \ -" " #asm_op " %0, %0, %3\n" \ -" stxr %w1, %0, %2\n" \ -" cbnz %w1, 1b" \ - : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \ - : "Ir" (i)); \ -} \ - -#define ATOMIC64_OP_RETURN(op, asm_op) \ -static inline long atomic64_##op##_return(long i, atomic64_t *v) \ -{ \ - long result; \ - unsigned long tmp; \ - \ - asm volatile("// atomic64_" #op "_return\n" \ -"1: ldxr %0, %2\n" \ -" " #asm_op " %0, %0, %3\n" \ -" stlxr %w1, %0, %2\n" \ -" cbnz %w1, 1b" \ - : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \ - : "Ir" (i) \ - : "memory"); \ - \ - smp_mb(); \ - return result; \ -} - -#define ATOMIC64_OPS(op, asm_op) \ - ATOMIC64_OP(op, asm_op) \ - ATOMIC64_OP_RETURN(op, asm_op) - -ATOMIC64_OPS(add, add) -ATOMIC64_OPS(sub, sub) - -#undef ATOMIC64_OPS -#undef ATOMIC64_OP_RETURN -#undef ATOMIC64_OP - -static inline long atomic64_cmpxchg(atomic64_t *ptr, long old, long new) -{ - long oldval; - unsigned long res; - - smp_mb(); - - asm volatile("// atomic64_cmpxchg\n" -"1: ldxr %1, %2\n" -" cmp %1, %3\n" -" b.ne 2f\n" -" stxr %w0, %4, %2\n" -" cbnz %w0, 1b\n" -"2:" - : "=&r" (res), "=&r" (oldval), "+Q" (ptr->counter) - : "Ir" (old), "r" (new) - : "cc"); - - smp_mb(); - return oldval; -} - -#define atomic64_xchg(v, new) (xchg(&((v)->counter), new)) - -static inline long atomic64_dec_if_positive(atomic64_t *v) -{ - long result; - unsigned long tmp; - - asm volatile("// atomic64_dec_if_positive\n" -"1: ldxr %0, %2\n" -" subs %0, %0, #1\n" -" b.mi 2f\n" -" stlxr %w1, %0, %2\n" -" cbnz %w1, 1b\n" -" dmb ish\n" -"2:" - : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) - : - : "cc", "memory"); - - return result; -} - -static inline int atomic64_add_unless(atomic64_t *v, long a, long u) -{ - long c, old; - - c = atomic64_read(v); - while (c != u && (old = atomic64_cmpxchg((v), c, c + a)) != c) - c = old; - - return c != u; -} - -#define atomic64_add_negative(a, v) (atomic64_add_return((a), (v)) < 0) -#define atomic64_inc(v) atomic64_add(1LL, (v)) -#define atomic64_inc_return(v) atomic64_add_return(1LL, (v)) +#define ATOMIC64_INIT ATOMIC_INIT +#define atomic64_read atomic_read +#define atomic64_set atomic_set +#define atomic64_xchg atomic_xchg +#define atomic64_cmpxchg atomic_cmpxchg + +#define atomic64_inc(v) atomic64_add(1, (v)) +#define atomic64_dec(v) atomic64_sub(1, (v)) +#define atomic64_inc_return(v) atomic64_add_return(1, (v)) +#define atomic64_dec_return(v) atomic64_sub_return(1, (v)) #define atomic64_inc_and_test(v) (atomic64_inc_return(v) == 0) -#define atomic64_sub_and_test(a, v) (atomic64_sub_return((a), (v)) == 0) -#define atomic64_dec(v) atomic64_sub(1LL, (v)) -#define atomic64_dec_return(v) atomic64_sub_return(1LL, (v)) -#define atomic64_dec_and_test(v) (atomic64_dec_return((v)) == 0) -#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1LL, 0LL) +#define atomic64_dec_and_test(v) (atomic64_dec_return(v) == 0) +#define atomic64_sub_and_test(i, v) (atomic64_sub_return((i), (v)) == 0) +#define atomic64_add_negative(i, v) (atomic64_add_return((i), (v)) < 0) +#define atomic64_add_unless(v, a, u) (___atomic_add_unless(v, a, u, 64) != u) +#define atomic64_andnot atomic64_andnot + +#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1, 0) #endif #endif diff --git a/arch/arm64/include/asm/atomic_ll_sc.h b/arch/arm64/include/asm/atomic_ll_sc.h new file mode 100644 index 00000000000000..b3b5c4ae3800b0 --- /dev/null +++ b/arch/arm64/include/asm/atomic_ll_sc.h @@ -0,0 +1,247 @@ +/* + * Based on arch/arm/include/asm/atomic.h + * + * Copyright (C) 1996 Russell King. + * Copyright (C) 2002 Deep Blue Solutions Ltd. + * Copyright (C) 2012 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#ifndef __ASM_ATOMIC_LL_SC_H +#define __ASM_ATOMIC_LL_SC_H + +#ifndef __ARM64_IN_ATOMIC_IMPL +#error "please don't include this file directly" +#endif + +/* + * AArch64 UP and SMP safe atomic ops. We use load exclusive and + * store exclusive to ensure that these are atomic. We may loop + * to ensure that the update happens. + * + * NOTE: these functions do *not* follow the PCS and must explicitly + * save any clobbered registers other than x0 (regardless of return + * value). This is achieved through -fcall-saved-* compiler flags for + * this file, which unfortunately don't work on a per-function basis + * (the optimize attribute silently ignores these options). + */ + +#define ATOMIC_OP(op, asm_op) \ +__LL_SC_INLINE void \ +__LL_SC_PREFIX(atomic_##op(int i, atomic_t *v)) \ +{ \ + unsigned long tmp; \ + int result; \ + \ + asm volatile("// atomic_" #op "\n" \ +" prfm pstl1strm, %2\n" \ +"1: ldxr %w0, %2\n" \ +" " #asm_op " %w0, %w0, %w3\n" \ +" stxr %w1, %w0, %2\n" \ +" cbnz %w1, 1b" \ + : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \ + : "Ir" (i)); \ +} \ +__LL_SC_EXPORT(atomic_##op); + +#define ATOMIC_OP_RETURN(op, asm_op) \ +__LL_SC_INLINE int \ +__LL_SC_PREFIX(atomic_##op##_return(int i, atomic_t *v)) \ +{ \ + unsigned long tmp; \ + int result; \ + \ + asm volatile("// atomic_" #op "_return\n" \ +" prfm pstl1strm, %2\n" \ +"1: ldxr %w0, %2\n" \ +" " #asm_op " %w0, %w0, %w3\n" \ +" stlxr %w1, %w0, %2\n" \ +" cbnz %w1, 1b" \ + : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \ + : "Ir" (i) \ + : "memory"); \ + \ + smp_mb(); \ + return result; \ +} \ +__LL_SC_EXPORT(atomic_##op##_return); + +#define ATOMIC_OPS(op, asm_op) \ + ATOMIC_OP(op, asm_op) \ + ATOMIC_OP_RETURN(op, asm_op) + +ATOMIC_OPS(add, add) +ATOMIC_OPS(sub, sub) + +ATOMIC_OP(and, and) +ATOMIC_OP(andnot, bic) +ATOMIC_OP(or, orr) +ATOMIC_OP(xor, eor) + +#undef ATOMIC_OPS +#undef ATOMIC_OP_RETURN +#undef ATOMIC_OP + +#define ATOMIC64_OP(op, asm_op) \ +__LL_SC_INLINE void \ +__LL_SC_PREFIX(atomic64_##op(long i, atomic64_t *v)) \ +{ \ + long result; \ + unsigned long tmp; \ + \ + asm volatile("// atomic64_" #op "\n" \ +" prfm pstl1strm, %2\n" \ +"1: ldxr %0, %2\n" \ +" " #asm_op " %0, %0, %3\n" \ +" stxr %w1, %0, %2\n" \ +" cbnz %w1, 1b" \ + : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \ + : "Ir" (i)); \ +} \ +__LL_SC_EXPORT(atomic64_##op); + +#define ATOMIC64_OP_RETURN(op, asm_op) \ +__LL_SC_INLINE long \ +__LL_SC_PREFIX(atomic64_##op##_return(long i, atomic64_t *v)) \ +{ \ + long result; \ + unsigned long tmp; \ + \ + asm volatile("// atomic64_" #op "_return\n" \ +" prfm pstl1strm, %2\n" \ +"1: ldxr %0, %2\n" \ +" " #asm_op " %0, %0, %3\n" \ +" stlxr %w1, %0, %2\n" \ +" cbnz %w1, 1b" \ + : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) \ + : "Ir" (i) \ + : "memory"); \ + \ + smp_mb(); \ + return result; \ +} \ +__LL_SC_EXPORT(atomic64_##op##_return); + +#define ATOMIC64_OPS(op, asm_op) \ + ATOMIC64_OP(op, asm_op) \ + ATOMIC64_OP_RETURN(op, asm_op) + +ATOMIC64_OPS(add, add) +ATOMIC64_OPS(sub, sub) + +ATOMIC64_OP(and, and) +ATOMIC64_OP(andnot, bic) +ATOMIC64_OP(or, orr) +ATOMIC64_OP(xor, eor) + +#undef ATOMIC64_OPS +#undef ATOMIC64_OP_RETURN +#undef ATOMIC64_OP + +__LL_SC_INLINE long +__LL_SC_PREFIX(atomic64_dec_if_positive(atomic64_t *v)) +{ + long result; + unsigned long tmp; + + asm volatile("// atomic64_dec_if_positive\n" +" prfm pstl1strm, %2\n" +"1: ldxr %0, %2\n" +" subs %0, %0, #1\n" +" b.lt 2f\n" +" stlxr %w1, %0, %2\n" +" cbnz %w1, 1b\n" +" dmb ish\n" +"2:" + : "=&r" (result), "=&r" (tmp), "+Q" (v->counter) + : + : "cc", "memory"); + + return result; +} +__LL_SC_EXPORT(atomic64_dec_if_positive); + +#define __CMPXCHG_CASE(w, sz, name, mb, rel, cl) \ +__LL_SC_INLINE unsigned long \ +__LL_SC_PREFIX(__cmpxchg_case_##name(volatile void *ptr, \ + unsigned long old, \ + unsigned long new)) \ +{ \ + unsigned long tmp, oldval; \ + \ + asm volatile( \ + " prfm pstl1strm, %[v]\n" \ + "1: ldxr" #sz "\t%" #w "[oldval], %[v]\n" \ + " eor %" #w "[tmp], %" #w "[oldval], %" #w "[old]\n" \ + " cbnz %" #w "[tmp], 2f\n" \ + " st" #rel "xr" #sz "\t%w[tmp], %" #w "[new], %[v]\n" \ + " cbnz %w[tmp], 1b\n" \ + " " #mb "\n" \ + " mov %" #w "[oldval], %" #w "[old]\n" \ + "2:" \ + : [tmp] "=&r" (tmp), [oldval] "=&r" (oldval), \ + [v] "+Q" (*(unsigned long *)ptr) \ + : [old] "Lr" (old), [new] "r" (new) \ + : cl); \ + \ + return oldval; \ +} \ +__LL_SC_EXPORT(__cmpxchg_case_##name); + +__CMPXCHG_CASE(w, b, 1, , , ) +__CMPXCHG_CASE(w, h, 2, , , ) +__CMPXCHG_CASE(w, , 4, , , ) +__CMPXCHG_CASE( , , 8, , , ) +__CMPXCHG_CASE(w, b, mb_1, dmb ish, l, "memory") +__CMPXCHG_CASE(w, h, mb_2, dmb ish, l, "memory") +__CMPXCHG_CASE(w, , mb_4, dmb ish, l, "memory") +__CMPXCHG_CASE( , , mb_8, dmb ish, l, "memory") + +#undef __CMPXCHG_CASE + +#define __CMPXCHG_DBL(name, mb, rel, cl) \ +__LL_SC_INLINE int \ +__LL_SC_PREFIX(__cmpxchg_double##name(unsigned long old1, \ + unsigned long old2, \ + unsigned long new1, \ + unsigned long new2, \ + volatile void *ptr)) \ +{ \ + unsigned long tmp, ret; \ + \ + asm volatile("// __cmpxchg_double" #name "\n" \ + " prfm pstl1strm, %2\n" \ + "1: ldxp %0, %1, %2\n" \ + " eor %0, %0, %3\n" \ + " eor %1, %1, %4\n" \ + " orr %1, %0, %1\n" \ + " cbnz %1, 2f\n" \ + " st" #rel "xp %w0, %5, %6, %2\n" \ + " cbnz %w0, 1b\n" \ + " " #mb "\n" \ + "2:" \ + : "=&r" (tmp), "=&r" (ret), "+Q" (*(unsigned long *)ptr) \ + : "r" (old1), "r" (old2), "r" (new1), "r" (new2) \ + : cl); \ + \ + return ret; \ +} \ +__LL_SC_EXPORT(__cmpxchg_double##name); + +__CMPXCHG_DBL( , , , ) +__CMPXCHG_DBL(_mb, dmb ish, l, "memory") + +#undef __CMPXCHG_DBL + +#endif /* __ASM_ATOMIC_LL_SC_H */ diff --git a/arch/arm64/include/asm/atomic_lse.h b/arch/arm64/include/asm/atomic_lse.h new file mode 100644 index 00000000000000..55d740e6345963 --- /dev/null +++ b/arch/arm64/include/asm/atomic_lse.h @@ -0,0 +1,391 @@ +/* + * Based on arch/arm/include/asm/atomic.h + * + * Copyright (C) 1996 Russell King. + * Copyright (C) 2002 Deep Blue Solutions Ltd. + * Copyright (C) 2012 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#ifndef __ASM_ATOMIC_LSE_H +#define __ASM_ATOMIC_LSE_H + +#ifndef __ARM64_IN_ATOMIC_IMPL +#error "please don't include this file directly" +#endif + +#define __LL_SC_ATOMIC(op) __LL_SC_CALL(atomic_##op) + +static inline void atomic_andnot(int i, atomic_t *v) +{ + register int w0 asm ("w0") = i; + register atomic_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN(__LL_SC_ATOMIC(andnot), + " stclr %w[i], %[v]\n") + : [i] "+r" (w0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline void atomic_or(int i, atomic_t *v) +{ + register int w0 asm ("w0") = i; + register atomic_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN(__LL_SC_ATOMIC(or), + " stset %w[i], %[v]\n") + : [i] "+r" (w0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline void atomic_xor(int i, atomic_t *v) +{ + register int w0 asm ("w0") = i; + register atomic_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN(__LL_SC_ATOMIC(xor), + " steor %w[i], %[v]\n") + : [i] "+r" (w0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline void atomic_add(int i, atomic_t *v) +{ + register int w0 asm ("w0") = i; + register atomic_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN(__LL_SC_ATOMIC(add), + " stadd %w[i], %[v]\n") + : [i] "+r" (w0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline int atomic_add_return(int i, atomic_t *v) +{ + register int w0 asm ("w0") = i; + register atomic_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " nop\n" + __LL_SC_ATOMIC(add_return), + /* LSE atomics */ + " ldaddal %w[i], w30, %[v]\n" + " add %w[i], %w[i], w30") + : [i] "+r" (w0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30", "memory"); + + return w0; +} + +static inline void atomic_and(int i, atomic_t *v) +{ + register int w0 asm ("w0") = i; + register atomic_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " nop\n" + __LL_SC_ATOMIC(and), + /* LSE atomics */ + " mvn %w[i], %w[i]\n" + " stclr %w[i], %[v]") + : [i] "+r" (w0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline void atomic_sub(int i, atomic_t *v) +{ + register int w0 asm ("w0") = i; + register atomic_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " nop\n" + __LL_SC_ATOMIC(sub), + /* LSE atomics */ + " neg %w[i], %w[i]\n" + " stadd %w[i], %[v]") + : [i] "+r" (w0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline int atomic_sub_return(int i, atomic_t *v) +{ + register int w0 asm ("w0") = i; + register atomic_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " nop\n" + __LL_SC_ATOMIC(sub_return) + " nop", + /* LSE atomics */ + " neg %w[i], %w[i]\n" + " ldaddal %w[i], w30, %[v]\n" + " add %w[i], %w[i], w30") + : [i] "+r" (w0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30", "memory"); + + return w0; +} + +#undef __LL_SC_ATOMIC + +#define __LL_SC_ATOMIC64(op) __LL_SC_CALL(atomic64_##op) + +static inline void atomic64_andnot(long i, atomic64_t *v) +{ + register long x0 asm ("x0") = i; + register atomic64_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN(__LL_SC_ATOMIC64(andnot), + " stclr %[i], %[v]\n") + : [i] "+r" (x0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline void atomic64_or(long i, atomic64_t *v) +{ + register long x0 asm ("x0") = i; + register atomic64_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN(__LL_SC_ATOMIC64(or), + " stset %[i], %[v]\n") + : [i] "+r" (x0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline void atomic64_xor(long i, atomic64_t *v) +{ + register long x0 asm ("x0") = i; + register atomic64_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN(__LL_SC_ATOMIC64(xor), + " steor %[i], %[v]\n") + : [i] "+r" (x0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline void atomic64_add(long i, atomic64_t *v) +{ + register long x0 asm ("x0") = i; + register atomic64_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN(__LL_SC_ATOMIC64(add), + " stadd %[i], %[v]\n") + : [i] "+r" (x0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline long atomic64_add_return(long i, atomic64_t *v) +{ + register long x0 asm ("x0") = i; + register atomic64_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " nop\n" + __LL_SC_ATOMIC64(add_return), + /* LSE atomics */ + " ldaddal %[i], x30, %[v]\n" + " add %[i], %[i], x30") + : [i] "+r" (x0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30", "memory"); + + return x0; +} + +static inline void atomic64_and(long i, atomic64_t *v) +{ + register long x0 asm ("x0") = i; + register atomic64_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " nop\n" + __LL_SC_ATOMIC64(and), + /* LSE atomics */ + " mvn %[i], %[i]\n" + " stclr %[i], %[v]") + : [i] "+r" (x0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline void atomic64_sub(long i, atomic64_t *v) +{ + register long x0 asm ("x0") = i; + register atomic64_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " nop\n" + __LL_SC_ATOMIC64(sub), + /* LSE atomics */ + " neg %[i], %[i]\n" + " stadd %[i], %[v]") + : [i] "+r" (x0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30"); +} + +static inline long atomic64_sub_return(long i, atomic64_t *v) +{ + register long x0 asm ("x0") = i; + register atomic64_t *x1 asm ("x1") = v; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " nop\n" + __LL_SC_ATOMIC64(sub_return) + " nop", + /* LSE atomics */ + " neg %[i], %[i]\n" + " ldaddal %[i], x30, %[v]\n" + " add %[i], %[i], x30") + : [i] "+r" (x0), [v] "+Q" (v->counter) + : "r" (x1) + : "x30", "memory"); + + return x0; +} + +static inline long atomic64_dec_if_positive(atomic64_t *v) +{ + register long x0 asm ("x0") = (long)v; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " nop\n" + __LL_SC_ATOMIC64(dec_if_positive) + " nop\n" + " nop\n" + " nop\n" + " nop\n" + " nop", + /* LSE atomics */ + "1: ldr x30, %[v]\n" + " subs %[ret], x30, #1\n" + " b.lt 2f\n" + " casal x30, %[ret], %[v]\n" + " sub x30, x30, #1\n" + " sub x30, x30, %[ret]\n" + " cbnz x30, 1b\n" + "2:") + : [ret] "+&r" (x0), [v] "+Q" (v->counter) + : + : "x30", "cc", "memory"); + + return x0; +} + +#undef __LL_SC_ATOMIC64 + +#define __LL_SC_CMPXCHG(op) __LL_SC_CALL(__cmpxchg_case_##op) + +#define __CMPXCHG_CASE(w, sz, name, mb, cl...) \ +static inline unsigned long __cmpxchg_case_##name(volatile void *ptr, \ + unsigned long old, \ + unsigned long new) \ +{ \ + register unsigned long x0 asm ("x0") = (unsigned long)ptr; \ + register unsigned long x1 asm ("x1") = old; \ + register unsigned long x2 asm ("x2") = new; \ + \ + asm volatile(ARM64_LSE_ATOMIC_INSN( \ + /* LL/SC */ \ + " nop\n" \ + __LL_SC_CMPXCHG(name) \ + " nop", \ + /* LSE atomics */ \ + " mov " #w "30, %" #w "[old]\n" \ + " cas" #mb #sz "\t" #w "30, %" #w "[new], %[v]\n" \ + " mov %" #w "[ret], " #w "30") \ + : [ret] "+r" (x0), [v] "+Q" (*(unsigned long *)ptr) \ + : [old] "r" (x1), [new] "r" (x2) \ + : "x30" , ##cl); \ + \ + return x0; \ +} + +__CMPXCHG_CASE(w, b, 1, ) +__CMPXCHG_CASE(w, h, 2, ) +__CMPXCHG_CASE(w, , 4, ) +__CMPXCHG_CASE(x, , 8, ) +__CMPXCHG_CASE(w, b, mb_1, al, "memory") +__CMPXCHG_CASE(w, h, mb_2, al, "memory") +__CMPXCHG_CASE(w, , mb_4, al, "memory") +__CMPXCHG_CASE(x, , mb_8, al, "memory") + +#undef __LL_SC_CMPXCHG +#undef __CMPXCHG_CASE + +#define __LL_SC_CMPXCHG_DBL(op) __LL_SC_CALL(__cmpxchg_double##op) + +#define __CMPXCHG_DBL(name, mb, cl...) \ +static inline int __cmpxchg_double##name(unsigned long old1, \ + unsigned long old2, \ + unsigned long new1, \ + unsigned long new2, \ + volatile void *ptr) \ +{ \ + unsigned long oldval1 = old1; \ + unsigned long oldval2 = old2; \ + register unsigned long x0 asm ("x0") = old1; \ + register unsigned long x1 asm ("x1") = old2; \ + register unsigned long x2 asm ("x2") = new1; \ + register unsigned long x3 asm ("x3") = new2; \ + register unsigned long x4 asm ("x4") = (unsigned long)ptr; \ + \ + asm volatile(ARM64_LSE_ATOMIC_INSN( \ + /* LL/SC */ \ + " nop\n" \ + " nop\n" \ + " nop\n" \ + __LL_SC_CMPXCHG_DBL(name), \ + /* LSE atomics */ \ + " casp" #mb "\t%[old1], %[old2], %[new1], %[new2], %[v]\n"\ + " eor %[old1], %[old1], %[oldval1]\n" \ + " eor %[old2], %[old2], %[oldval2]\n" \ + " orr %[old1], %[old1], %[old2]") \ + : [old1] "+r" (x0), [old2] "+r" (x1), \ + [v] "+Q" (*(unsigned long *)ptr) \ + : [new1] "r" (x2), [new2] "r" (x3), [ptr] "r" (x4), \ + [oldval1] "r" (oldval1), [oldval2] "r" (oldval2) \ + : "x30" , ##cl); \ + \ + return x0; \ +} + +__CMPXCHG_DBL( , ) +__CMPXCHG_DBL(_mb, al, "memory") + +#undef __LL_SC_CMPXCHG_DBL +#undef __CMPXCHG_DBL + +#endif /* __ASM_ATOMIC_LSE_H */ diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index 0fa47c4275cb56..624f9679f4b0ab 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -35,28 +35,6 @@ #define dma_rmb() dmb(oshld) #define dma_wmb() dmb(oshst) -#ifndef CONFIG_SMP -#define smp_mb() barrier() -#define smp_rmb() barrier() -#define smp_wmb() barrier() - -#define smp_store_release(p, v) \ -do { \ - compiletime_assert_atomic_type(*p); \ - barrier(); \ - ACCESS_ONCE(*p) = (v); \ -} while (0) - -#define smp_load_acquire(p) \ -({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ - compiletime_assert_atomic_type(*p); \ - barrier(); \ - ___p1; \ -}) - -#else - #define smp_mb() dmb(ish) #define smp_rmb() dmb(ishld) #define smp_wmb() dmb(ishst) @@ -109,8 +87,6 @@ do { \ ___p1; \ }) -#endif - #define read_barrier_depends() do { } while(0) #define smp_read_barrier_depends() do { } while(0) diff --git a/arch/arm64/include/asm/bug.h b/arch/arm64/include/asm/bug.h new file mode 100644 index 00000000000000..4a748ce9ba1a71 --- /dev/null +++ b/arch/arm64/include/asm/bug.h @@ -0,0 +1,64 @@ +/* + * Copyright (C) 2015 ARM Limited + * Author: Dave Martin + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#ifndef _ARCH_ARM64_ASM_BUG_H +#define _ARCH_ARM64_ASM_BUG_H + +#include + +#ifdef CONFIG_GENERIC_BUG +#define HAVE_ARCH_BUG + +#ifdef CONFIG_DEBUG_BUGVERBOSE +#define _BUGVERBOSE_LOCATION(file, line) __BUGVERBOSE_LOCATION(file, line) +#define __BUGVERBOSE_LOCATION(file, line) \ + ".pushsection .rodata.str,\"aMS\",@progbits,1\n" \ + "2: .string \"" file "\"\n\t" \ + ".popsection\n\t" \ + \ + ".long 2b - 0b\n\t" \ + ".short " #line "\n\t" +#else +#define _BUGVERBOSE_LOCATION(file, line) +#endif + +#define _BUG_FLAGS(flags) __BUG_FLAGS(flags) + +#define __BUG_FLAGS(flags) asm volatile ( \ + ".pushsection __bug_table,\"a\"\n\t" \ + ".align 2\n\t" \ + "0: .long 1f - 0b\n\t" \ +_BUGVERBOSE_LOCATION(__FILE__, __LINE__) \ + ".short " #flags "\n\t" \ + ".popsection\n" \ + \ + "1: brk %[imm]" \ + :: [imm] "i" (BUG_BRK_IMM) \ +) + +#define BUG() do { \ + _BUG_FLAGS(0); \ + unreachable(); \ +} while (0) + +#define __WARN_TAINT(taint) _BUG_FLAGS(BUGFLAG_TAINT(taint)) + +#endif /* ! CONFIG_GENERIC_BUG */ + +#include + +#endif /* ! _ARCH_ARM64_ASM_BUG_H */ diff --git a/arch/arm64/include/asm/cmpxchg.h b/arch/arm64/include/asm/cmpxchg.h index d8c25b7b18fbf4..899e9f1d19e486 100644 --- a/arch/arm64/include/asm/cmpxchg.h +++ b/arch/arm64/include/asm/cmpxchg.h @@ -21,7 +21,9 @@ #include #include +#include #include +#include static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size) { @@ -29,37 +31,73 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size switch (size) { case 1: - asm volatile("// __xchg1\n" + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " prfm pstl1strm, %2\n" "1: ldxrb %w0, %2\n" " stlxrb %w1, %w3, %2\n" " cbnz %w1, 1b\n" + " dmb ish", + /* LSE atomics */ + " nop\n" + " nop\n" + " swpalb %w3, %w0, %2\n" + " nop\n" + " nop") : "=&r" (ret), "=&r" (tmp), "+Q" (*(u8 *)ptr) : "r" (x) : "memory"); break; case 2: - asm volatile("// __xchg2\n" + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " prfm pstl1strm, %2\n" "1: ldxrh %w0, %2\n" " stlxrh %w1, %w3, %2\n" " cbnz %w1, 1b\n" + " dmb ish", + /* LSE atomics */ + " nop\n" + " nop\n" + " swpalh %w3, %w0, %2\n" + " nop\n" + " nop") : "=&r" (ret), "=&r" (tmp), "+Q" (*(u16 *)ptr) : "r" (x) : "memory"); break; case 4: - asm volatile("// __xchg4\n" + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " prfm pstl1strm, %2\n" "1: ldxr %w0, %2\n" " stlxr %w1, %w3, %2\n" " cbnz %w1, 1b\n" + " dmb ish", + /* LSE atomics */ + " nop\n" + " nop\n" + " swpal %w3, %w0, %2\n" + " nop\n" + " nop") : "=&r" (ret), "=&r" (tmp), "+Q" (*(u32 *)ptr) : "r" (x) : "memory"); break; case 8: - asm volatile("// __xchg8\n" + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " prfm pstl1strm, %2\n" "1: ldxr %0, %2\n" " stlxr %w1, %3, %2\n" " cbnz %w1, 1b\n" + " dmb ish", + /* LSE atomics */ + " nop\n" + " nop\n" + " swpal %3, %0, %2\n" + " nop\n" + " nop") : "=&r" (ret), "=&r" (tmp), "+Q" (*(u64 *)ptr) : "r" (x) : "memory"); @@ -68,7 +106,6 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size BUILD_BUG(); } - smp_mb(); return ret; } @@ -83,131 +120,39 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old, unsigned long new, int size) { - unsigned long oldval = 0, res; - switch (size) { case 1: - do { - asm volatile("// __cmpxchg1\n" - " ldxrb %w1, %2\n" - " mov %w0, #0\n" - " cmp %w1, %w3\n" - " b.ne 1f\n" - " stxrb %w0, %w4, %2\n" - "1:\n" - : "=&r" (res), "=&r" (oldval), "+Q" (*(u8 *)ptr) - : "Ir" (old), "r" (new) - : "cc"); - } while (res); - break; - + return __cmpxchg_case_1(ptr, (u8)old, new); case 2: - do { - asm volatile("// __cmpxchg2\n" - " ldxrh %w1, %2\n" - " mov %w0, #0\n" - " cmp %w1, %w3\n" - " b.ne 1f\n" - " stxrh %w0, %w4, %2\n" - "1:\n" - : "=&r" (res), "=&r" (oldval), "+Q" (*(u16 *)ptr) - : "Ir" (old), "r" (new) - : "cc"); - } while (res); - break; - + return __cmpxchg_case_2(ptr, (u16)old, new); case 4: - do { - asm volatile("// __cmpxchg4\n" - " ldxr %w1, %2\n" - " mov %w0, #0\n" - " cmp %w1, %w3\n" - " b.ne 1f\n" - " stxr %w0, %w4, %2\n" - "1:\n" - : "=&r" (res), "=&r" (oldval), "+Q" (*(u32 *)ptr) - : "Ir" (old), "r" (new) - : "cc"); - } while (res); - break; - + return __cmpxchg_case_4(ptr, old, new); case 8: - do { - asm volatile("// __cmpxchg8\n" - " ldxr %1, %2\n" - " mov %w0, #0\n" - " cmp %1, %3\n" - " b.ne 1f\n" - " stxr %w0, %4, %2\n" - "1:\n" - : "=&r" (res), "=&r" (oldval), "+Q" (*(u64 *)ptr) - : "Ir" (old), "r" (new) - : "cc"); - } while (res); - break; - + return __cmpxchg_case_8(ptr, old, new); default: BUILD_BUG(); } - return oldval; + unreachable(); } -#define system_has_cmpxchg_double() 1 - -static inline int __cmpxchg_double(volatile void *ptr1, volatile void *ptr2, - unsigned long old1, unsigned long old2, - unsigned long new1, unsigned long new2, int size) +static inline unsigned long __cmpxchg_mb(volatile void *ptr, unsigned long old, + unsigned long new, int size) { - unsigned long loop, lost; - switch (size) { + case 1: + return __cmpxchg_case_mb_1(ptr, (u8)old, new); + case 2: + return __cmpxchg_case_mb_2(ptr, (u16)old, new); + case 4: + return __cmpxchg_case_mb_4(ptr, old, new); case 8: - VM_BUG_ON((unsigned long *)ptr2 - (unsigned long *)ptr1 != 1); - do { - asm volatile("// __cmpxchg_double8\n" - " ldxp %0, %1, %2\n" - " eor %0, %0, %3\n" - " eor %1, %1, %4\n" - " orr %1, %0, %1\n" - " mov %w0, #0\n" - " cbnz %1, 1f\n" - " stxp %w0, %5, %6, %2\n" - "1:\n" - : "=&r"(loop), "=&r"(lost), "+Q" (*(u64 *)ptr1) - : "r" (old1), "r"(old2), "r"(new1), "r"(new2)); - } while (loop); - break; + return __cmpxchg_case_mb_8(ptr, old, new); default: BUILD_BUG(); } - return !lost; -} - -static inline int __cmpxchg_double_mb(volatile void *ptr1, volatile void *ptr2, - unsigned long old1, unsigned long old2, - unsigned long new1, unsigned long new2, int size) -{ - int ret; - - smp_mb(); - ret = __cmpxchg_double(ptr1, ptr2, old1, old2, new1, new2, size); - smp_mb(); - - return ret; -} - -static inline unsigned long __cmpxchg_mb(volatile void *ptr, unsigned long old, - unsigned long new, int size) -{ - unsigned long ret; - - smp_mb(); - ret = __cmpxchg(ptr, old, new, size); - smp_mb(); - - return ret; + unreachable(); } #define cmpxchg(ptr, o, n) \ @@ -228,21 +173,32 @@ static inline unsigned long __cmpxchg_mb(volatile void *ptr, unsigned long old, __ret; \ }) +#define system_has_cmpxchg_double() 1 + +#define __cmpxchg_double_check(ptr1, ptr2) \ +({ \ + if (sizeof(*(ptr1)) != 8) \ + BUILD_BUG(); \ + VM_BUG_ON((unsigned long *)(ptr2) - (unsigned long *)(ptr1) != 1); \ +}) + #define cmpxchg_double(ptr1, ptr2, o1, o2, n1, n2) \ ({\ int __ret;\ - __ret = __cmpxchg_double_mb((ptr1), (ptr2), (unsigned long)(o1), \ - (unsigned long)(o2), (unsigned long)(n1), \ - (unsigned long)(n2), sizeof(*(ptr1)));\ + __cmpxchg_double_check(ptr1, ptr2); \ + __ret = !__cmpxchg_double_mb((unsigned long)(o1), (unsigned long)(o2), \ + (unsigned long)(n1), (unsigned long)(n2), \ + ptr1); \ __ret; \ }) #define cmpxchg_double_local(ptr1, ptr2, o1, o2, n1, n2) \ ({\ int __ret;\ - __ret = __cmpxchg_double((ptr1), (ptr2), (unsigned long)(o1), \ - (unsigned long)(o2), (unsigned long)(n1), \ - (unsigned long)(n2), sizeof(*(ptr1)));\ + __cmpxchg_double_check(ptr1, ptr2); \ + __ret = !__cmpxchg_double((unsigned long)(o1), (unsigned long)(o2), \ + (unsigned long)(n1), (unsigned long)(n2), \ + ptr1); \ __ret; \ }) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index c1044218a63a15..171570702bb801 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -25,15 +25,20 @@ #define ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE 1 #define ARM64_WORKAROUND_845719 2 #define ARM64_HAS_SYSREG_GIC_CPUIF 3 +#define ARM64_HAS_PAN 4 +#define ARM64_HAS_LSE_ATOMICS 5 -#define ARM64_NCAPS 4 +#define ARM64_NCAPS 6 #ifndef __ASSEMBLY__ +#include + struct arm64_cpu_capabilities { const char *desc; u16 capability; bool (*matches)(const struct arm64_cpu_capabilities *); + void (*enable)(void); union { struct { /* To be used for erratum handling only */ u32 midr_model; @@ -41,8 +46,8 @@ struct arm64_cpu_capabilities { }; struct { /* Feature register checking */ - u64 register_mask; - u64 register_value; + int field_pos; + int min_field_value; }; }; }; @@ -70,6 +75,13 @@ static inline void cpus_set_cap(unsigned int num) __set_bit(num, cpu_hwcaps); } +static inline int __attribute_const__ cpuid_feature_extract_field(u64 features, + int field) +{ + return (s64)(features << (64 - 4 - field)) >> (64 - 4); +} + + void check_cpu_capabilities(const struct arm64_cpu_capabilities *caps, const char *info); void check_local_cpu_errata(void); diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h index a84ec605bed819..ee6403df9fe4c1 100644 --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -81,9 +81,6 @@ #define ID_AA64MMFR0_BIGEND(mmfr0) \ (((mmfr0) & ID_AA64MMFR0_BIGEND_MASK) >> ID_AA64MMFR0_BIGEND_SHIFT) -#define SCTLR_EL1_CP15BEN (0x1 << 5) -#define SCTLR_EL1_SED (0x1 << 8) - #ifndef __ASSEMBLY__ /* diff --git a/arch/arm64/include/asm/debug-monitors.h b/arch/arm64/include/asm/debug-monitors.h index 40ec68aa687017..279c85b5ec091e 100644 --- a/arch/arm64/include/asm/debug-monitors.h +++ b/arch/arm64/include/asm/debug-monitors.h @@ -18,6 +18,12 @@ #ifdef __KERNEL__ +#include +#include +#include +#include +#include + /* Low-level stepping controls. */ #define DBG_MDSCR_SS (1 << 0) #define DBG_SPSR_SS (1 << 21) @@ -38,12 +44,7 @@ /* * Break point instruction encoding */ -#define BREAK_INSTR_SIZE 4 - -/* - * ESR values expected for dynamic and compile time BRK instruction - */ -#define DBG_ESR_VAL_BRK(x) (0xf2000000 | ((x) & 0xfffff)) +#define BREAK_INSTR_SIZE AARCH64_INSN_SIZE /* * #imm16 values used for BRK instruction generation @@ -51,10 +52,12 @@ * 0x100: for triggering a fault on purpose (reserved) * 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction + * 0x800: kernel-mode BUG() and WARN() traps */ #define FAULT_BRK_IMM 0x100 #define KGDB_DYN_DBG_BRK_IMM 0x400 #define KGDB_COMPILED_DBG_BRK_IMM 0x401 +#define BUG_BRK_IMM 0x800 /* * BRK instruction encoding @@ -68,25 +71,10 @@ */ #define AARCH64_BREAK_FAULT (AARCH64_BREAK_MON | (FAULT_BRK_IMM << 5)) -/* - * Extract byte from BRK instruction - */ -#define KGDB_DYN_DBG_BRK_INS_BYTE(x) \ - ((((AARCH64_BREAK_MON) & 0xffe0001f) >> (x * 8)) & 0xff) - -/* - * Extract byte from BRK #imm16 - */ -#define KGBD_DYN_DBG_BRK_IMM_BYTE(x) \ - (((((KGDB_DYN_DBG_BRK_IMM) & 0xffff) << 5) >> (x * 8)) & 0xff) - -#define KGDB_DYN_DBG_BRK_BYTE(x) \ - (KGDB_DYN_DBG_BRK_INS_BYTE(x) | KGBD_DYN_DBG_BRK_IMM_BYTE(x)) - -#define KGDB_DYN_BRK_INS_BYTE0 KGDB_DYN_DBG_BRK_BYTE(0) -#define KGDB_DYN_BRK_INS_BYTE1 KGDB_DYN_DBG_BRK_BYTE(1) -#define KGDB_DYN_BRK_INS_BYTE2 KGDB_DYN_DBG_BRK_BYTE(2) -#define KGDB_DYN_BRK_INS_BYTE3 KGDB_DYN_DBG_BRK_BYTE(3) +#define AARCH64_BREAK_KGDB_DYN_DBG \ + (AARCH64_BREAK_MON | (KGDB_DYN_DBG_BRK_IMM << 5)) +#define KGDB_DYN_BRK_INS_BYTE(x) \ + ((AARCH64_BREAK_KGDB_DYN_DBG >> (8 * (x))) & 0xff) #define CACHE_FLUSH_IS_SAFE 1 @@ -127,13 +115,13 @@ void unregister_break_hook(struct break_hook *hook); u8 debug_monitors_arch(void); -enum debug_el { +enum dbg_active_el { DBG_ACTIVE_EL0 = 0, DBG_ACTIVE_EL1, }; -void enable_debug_monitors(enum debug_el el); -void disable_debug_monitors(enum debug_el el); +void enable_debug_monitors(enum dbg_active_el el); +void disable_debug_monitors(enum dbg_active_el el); void user_rewind_single_step(struct task_struct *task); void user_fastforward_single_step(struct task_struct *task); diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 70522450ca2342..77eeb2cc648fd7 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -18,6 +18,8 @@ #ifndef __ASM_ESR_H #define __ASM_ESR_H +#include + #define ESR_ELx_EC_UNKNOWN (0x00) #define ESR_ELx_EC_WFx (0x01) /* Unallocated EC: 0x02 */ @@ -99,6 +101,13 @@ #define ESR_ELx_WFx_ISS_WFE (UL(1) << 0) #define ESR_ELx_xVC_IMM_MASK ((1UL << 16) - 1) +/* ESR value templates for specific events */ + +/* BRK instruction trap from AArch64 state */ +#define ESR_ELx_VAL_BRK64(imm) \ + ((ESR_ELx_EC_BRK64 << ESR_ELx_EC_SHIFT) | ESR_ELx_IL | \ + ((imm) & 0xffff)) + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index 0303705fcad6a6..6cb7e1a6bc02a5 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -18,7 +18,13 @@ #ifndef __ASM_EXCEPTION_H #define __ASM_EXCEPTION_H +#include + #define __exception __attribute__((section(".exception.text"))) +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +#define __exception_irq_entry __irq_entry +#else #define __exception_irq_entry __exception +#endif #endif /* __ASM_EXCEPTION_H */ diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index c0739187a92043..8b9884c726adc7 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -8,7 +8,7 @@ * Copyright (C) 1998 Ingo Molnar * Copyright (C) 2013 Mark Salter * - * Adapted from arch/x86_64 version. + * Adapted from arch/x86 version. * */ diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h index 74069b3bd919c7..007a69fc4f408d 100644 --- a/arch/arm64/include/asm/futex.h +++ b/arch/arm64/include/asm/futex.h @@ -20,10 +20,17 @@ #include #include + +#include +#include #include +#include #define __futex_atomic_op(insn, ret, oldval, uaddr, tmp, oparg) \ asm volatile( \ + ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) \ +" prfm pstl1strm, %2\n" \ "1: ldxr %w1, %2\n" \ insn "\n" \ "2: stlxr %w3, %w0, %2\n" \ @@ -39,6 +46,8 @@ " .align 3\n" \ " .quad 1b, 4b, 2b, 4b\n" \ " .popsection\n" \ + ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) \ : "=&r" (ret), "=&r" (oldval), "+Q" (*uaddr), "=&r" (tmp) \ : "r" (oparg), "Ir" (-EFAULT) \ : "memory") @@ -112,6 +121,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, return -EFAULT; asm volatile("// futex_atomic_cmpxchg_inatomic\n" +" prfm pstl1strm, %2\n" "1: ldxr %w1, %2\n" " sub %w3, %w1, %w4\n" " cbnz %w3, 3f\n" diff --git a/arch/arm64/include/asm/hardirq.h b/arch/arm64/include/asm/hardirq.h index 6aae421f4d733d..2bb7009bdac721 100644 --- a/arch/arm64/include/asm/hardirq.h +++ b/arch/arm64/include/asm/hardirq.h @@ -24,9 +24,7 @@ typedef struct { unsigned int __softirq_pending; -#ifdef CONFIG_SMP unsigned int ipi_irqs[NR_IPI]; -#endif } ____cacheline_aligned irq_cpustat_t; #include /* Standard mappings for irq_cpustat_t above */ @@ -34,10 +32,8 @@ typedef struct { #define __inc_irq_stat(cpu, member) __IRQ_STAT(cpu, member)++ #define __get_irq_stat(cpu, member) __IRQ_STAT(cpu, member) -#ifdef CONFIG_SMP u64 smp_irq_stat_cpu(unsigned int cpu); #define arch_irq_stat_cpu smp_irq_stat_cpu -#endif #define __ARCH_IRQ_EXIT_IRQS_DISABLED 1 diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 2fd9b14ca2956e..bb4052e85dbac9 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -13,10 +13,6 @@ * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write to the Free Software - * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ #ifndef __ASM_HUGETLB_H diff --git a/arch/arm64/include/asm/irq_work.h b/arch/arm64/include/asm/irq_work.h index b4f6b19a8a685d..8e24ef3f7c82c7 100644 --- a/arch/arm64/include/asm/irq_work.h +++ b/arch/arm64/include/asm/irq_work.h @@ -1,8 +1,6 @@ #ifndef __ASM_IRQ_WORK_H #define __ASM_IRQ_WORK_H -#ifdef CONFIG_SMP - #include static inline bool arch_irq_work_has_interrupt(void) @@ -10,13 +8,4 @@ static inline bool arch_irq_work_has_interrupt(void) return !!__smp_cross_call; } -#else - -static inline bool arch_irq_work_has_interrupt(void) -{ - return false; -} - -#endif - #endif /* __ASM_IRQ_WORK_H */ diff --git a/arch/arm64/include/asm/jump_label.h b/arch/arm64/include/asm/jump_label.h index c0e5165c2f76d3..1b5e0e843c3af8 100644 --- a/arch/arm64/include/asm/jump_label.h +++ b/arch/arm64/include/asm/jump_label.h @@ -26,14 +26,28 @@ #define JUMP_LABEL_NOP_SIZE AARCH64_INSN_SIZE -static __always_inline bool arch_static_branch(struct static_key *key) +static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { asm goto("1: nop\n\t" ".pushsection __jump_table, \"aw\"\n\t" ".align 3\n\t" ".quad 1b, %l[l_yes], %c0\n\t" ".popsection\n\t" - : : "i"(key) : : l_yes); + : : "i"(&((char *)key)[branch]) : : l_yes); + + return false; +l_yes: + return true; +} + +static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch) +{ + asm goto("1: b %l[l_yes]\n\t" + ".pushsection __jump_table, \"aw\"\n\t" + ".align 3\n\t" + ".quad 1b, %l[l_yes], %c0\n\t" + ".popsection\n\t" + : : "i"(&((char *)key)[branch]) : : l_yes); return false; l_yes: diff --git a/arch/arm64/include/asm/lse.h b/arch/arm64/include/asm/lse.h new file mode 100644 index 00000000000000..3de42d68611df9 --- /dev/null +++ b/arch/arm64/include/asm/lse.h @@ -0,0 +1,53 @@ +#ifndef __ASM_LSE_H +#define __ASM_LSE_H + +#if defined(CONFIG_AS_LSE) && defined(CONFIG_ARM64_LSE_ATOMICS) + +#include +#include +#include + +#ifdef __ASSEMBLER__ + +.arch_extension lse + +.macro alt_lse, llsc, lse + alternative_insn "\llsc", "\lse", ARM64_HAS_LSE_ATOMICS +.endm + +#else /* __ASSEMBLER__ */ + +__asm__(".arch_extension lse"); + +/* Move the ll/sc atomics out-of-line */ +#define __LL_SC_INLINE +#define __LL_SC_PREFIX(x) __ll_sc_##x +#define __LL_SC_EXPORT(x) EXPORT_SYMBOL(__LL_SC_PREFIX(x)) + +/* Macro for constructing calls to out-of-line ll/sc atomics */ +#define __LL_SC_CALL(op) "bl\t" __stringify(__LL_SC_PREFIX(op)) "\n" + +/* In-line patching at runtime */ +#define ARM64_LSE_ATOMIC_INSN(llsc, lse) \ + ALTERNATIVE(llsc, lse, ARM64_HAS_LSE_ATOMICS) + +#endif /* __ASSEMBLER__ */ +#else /* CONFIG_AS_LSE && CONFIG_ARM64_LSE_ATOMICS */ + +#ifdef __ASSEMBLER__ + +.macro alt_lse, llsc, lse + \llsc +.endm + +#else /* __ASSEMBLER__ */ + +#define __LL_SC_INLINE static inline +#define __LL_SC_PREFIX(x) x +#define __LL_SC_EXPORT(x) + +#define ARM64_LSE_ATOMIC_INSN(llsc, lse) llsc + +#endif /* __ASSEMBLER__ */ +#endif /* CONFIG_AS_LSE && CONFIG_ARM64_LSE_ATOMICS */ +#endif /* __ASM_LSE_H */ diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index f800d45ea2265b..44a59c20e77358 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -113,6 +113,14 @@ extern phys_addr_t memstart_addr; /* PHYS_OFFSET - the physical address of the start of memory. */ #define PHYS_OFFSET ({ memstart_addr; }) +/* + * The maximum physical address that the linear direct mapping + * of system RAM can cover. (PAGE_OFFSET can be interpreted as + * a 2's complement signed quantity and negated to derive the + * maximum size of the linear mapping.) + */ +#define MAX_MEMBLOCK_ADDR ({ memstart_addr - PAGE_OFFSET - 1; }) + /* * PFNs are used to describe any physical page; this means * PFN 0 == physical address 0. diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 79fcfb04888440..030208767185bd 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -28,7 +28,6 @@ typedef struct { #define ASID(mm) ((mm)->context.id & 0xffff) extern void paging_init(void); -extern void setup_mm_for_reboot(void); extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt); extern void init_mem_pgprot(void); extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, diff --git a/arch/arm64/include/asm/percpu.h b/arch/arm64/include/asm/percpu.h index 4fde8c1df97ffb..0a456bef8c792d 100644 --- a/arch/arm64/include/asm/percpu.h +++ b/arch/arm64/include/asm/percpu.h @@ -16,8 +16,6 @@ #ifndef __ASM_PERCPU_H #define __ASM_PERCPU_H -#ifdef CONFIG_SMP - static inline void set_my_cpu_offset(unsigned long off) { asm volatile("msr tpidr_el1, %0" :: "r" (off) : "memory"); @@ -38,12 +36,6 @@ static inline unsigned long __my_cpu_offset(void) } #define __my_cpu_offset __my_cpu_offset() -#else /* !CONFIG_SMP */ - -#define set_my_cpu_offset(x) do { } while (0) - -#endif /* CONFIG_SMP */ - #define PERCPU_OP(op, asm_op) \ static inline unsigned long __percpu_##op(void *ptr, \ unsigned long val, int size) \ diff --git a/arch/arm64/include/asm/perf_event.h b/arch/arm64/include/asm/perf_event.h index 6471773db6fd8c..7bd3cdb533ea80 100644 --- a/arch/arm64/include/asm/perf_event.h +++ b/arch/arm64/include/asm/perf_event.h @@ -17,7 +17,7 @@ #ifndef __ASM_PERF_EVENT_H #define __ASM_PERF_EVENT_H -#ifdef CONFIG_HW_PERF_EVENTS +#ifdef CONFIG_PERF_EVENTS struct pt_regs; extern unsigned long perf_instruction_pointer(struct pt_regs *regs); extern unsigned long perf_misc_flags(struct pt_regs *regs); diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 59bfae75dc9800..24154b055835b0 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -104,6 +104,7 @@ #define PTE_SHARED (_AT(pteval_t, 3) << 8) /* SH[1:0], inner shareable */ #define PTE_AF (_AT(pteval_t, 1) << 10) /* Access Flag */ #define PTE_NG (_AT(pteval_t, 1) << 11) /* nG */ +#define PTE_DBM (_AT(pteval_t, 1) << 51) /* Dirty Bit Management */ #define PTE_PXN (_AT(pteval_t, 1) << 53) /* Privileged XN */ #define PTE_UXN (_AT(pteval_t, 1) << 54) /* User XN */ @@ -168,5 +169,7 @@ #define TCR_TG1_64K (UL(3) << 30) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) +#define TCR_HA (UL(1) << 39) +#define TCR_HD (UL(1) << 40) #endif diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 56283f8a675c5f..6900b2d953717c 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -16,6 +16,7 @@ #ifndef __ASM_PGTABLE_H #define __ASM_PGTABLE_H +#include #include #include @@ -27,7 +28,11 @@ #define PTE_VALID (_AT(pteval_t, 1) << 0) #define PTE_DIRTY (_AT(pteval_t, 1) << 55) #define PTE_SPECIAL (_AT(pteval_t, 1) << 56) +#ifdef CONFIG_ARM64_HW_AFDBM +#define PTE_WRITE (PTE_DBM) /* same as DBM */ +#else #define PTE_WRITE (_AT(pteval_t, 1) << 57) +#endif #define PTE_PROT_NONE (_AT(pteval_t, 1) << 58) /* only when !PTE_VALID */ /* @@ -48,18 +53,16 @@ #define FIRST_USER_ADDRESS 0UL #ifndef __ASSEMBLY__ + +#include + extern void __pte_error(const char *file, int line, unsigned long val); extern void __pmd_error(const char *file, int line, unsigned long val); extern void __pud_error(const char *file, int line, unsigned long val); extern void __pgd_error(const char *file, int line, unsigned long val); -#ifdef CONFIG_SMP #define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) #define PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) -#else -#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF) -#define PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF) -#endif #define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_ATTRINDX(MT_DEVICE_nGnRE)) #define PROT_NORMAL_NC (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_ATTRINDX(MT_NORMAL_NC)) @@ -137,12 +140,20 @@ extern struct page *empty_zero_page; * The following only work if pte_present(). Undefined behaviour otherwise. */ #define pte_present(pte) (!!(pte_val(pte) & (PTE_VALID | PTE_PROT_NONE))) -#define pte_dirty(pte) (!!(pte_val(pte) & PTE_DIRTY)) #define pte_young(pte) (!!(pte_val(pte) & PTE_AF)) #define pte_special(pte) (!!(pte_val(pte) & PTE_SPECIAL)) #define pte_write(pte) (!!(pte_val(pte) & PTE_WRITE)) #define pte_exec(pte) (!(pte_val(pte) & PTE_UXN)) +#ifdef CONFIG_ARM64_HW_AFDBM +#define pte_hw_dirty(pte) (!(pte_val(pte) & PTE_RDONLY)) +#else +#define pte_hw_dirty(pte) (0) +#endif +#define pte_sw_dirty(pte) (!!(pte_val(pte) & PTE_DIRTY)) +#define pte_dirty(pte) (pte_sw_dirty(pte) || pte_hw_dirty(pte)) + +#define pte_valid(pte) (!!(pte_val(pte) & PTE_VALID)) #define pte_valid_user(pte) \ ((pte_val(pte) & (PTE_VALID | PTE_USER)) == (PTE_VALID | PTE_USER)) #define pte_valid_not_user(pte) \ @@ -209,20 +220,49 @@ static inline void set_pte(pte_t *ptep, pte_t pte) } } +struct mm_struct; +struct vm_area_struct; + extern void __sync_icache_dcache(pte_t pteval, unsigned long addr); +/* + * PTE bits configuration in the presence of hardware Dirty Bit Management + * (PTE_WRITE == PTE_DBM): + * + * Dirty Writable | PTE_RDONLY PTE_WRITE PTE_DIRTY (sw) + * 0 0 | 1 0 0 + * 0 1 | 1 1 0 + * 1 0 | 1 0 1 + * 1 1 | 0 1 x + * + * When hardware DBM is not present, the sofware PTE_DIRTY bit is updated via + * the page fault mechanism. Checking the dirty status of a pte becomes: + * + * PTE_DIRTY || !PTE_RDONLY + */ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte) { if (pte_valid_user(pte)) { if (!pte_special(pte) && pte_exec(pte)) __sync_icache_dcache(pte, addr); - if (pte_dirty(pte) && pte_write(pte)) + if (pte_sw_dirty(pte) && pte_write(pte)) pte_val(pte) &= ~PTE_RDONLY; else pte_val(pte) |= PTE_RDONLY; } + /* + * If the existing pte is valid, check for potential race with + * hardware updates of the pte (ptep_set_access_flags safely changes + * valid ptes without going through an invalid entry). + */ + if (IS_ENABLED(CONFIG_DEBUG_VM) && IS_ENABLED(CONFIG_ARM64_HW_AFDBM) && + pte_valid(*ptep)) { + BUG_ON(!pte_young(pte)); + BUG_ON(pte_write(*ptep) && !pte_dirty(pte)); + } + set_pte(ptep, pte); } @@ -461,6 +501,9 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) { const pteval_t mask = PTE_USER | PTE_PXN | PTE_UXN | PTE_RDONLY | PTE_PROT_NONE | PTE_WRITE | PTE_TYPE_MASK; + /* preserve the hardware dirty information */ + if (pte_hw_dirty(pte)) + newprot |= PTE_DIRTY; pte_val(pte) = (pte_val(pte) & ~mask) | (pgprot_val(newprot) & mask); return pte; } @@ -470,6 +513,101 @@ static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot) return pte_pmd(pte_modify(pmd_pte(pmd), newprot)); } +#ifdef CONFIG_ARM64_HW_AFDBM +/* + * Atomic pte/pmd modifications. + */ +#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG +static inline int ptep_test_and_clear_young(struct vm_area_struct *vma, + unsigned long address, + pte_t *ptep) +{ + pteval_t pteval; + unsigned int tmp, res; + + asm volatile("// ptep_test_and_clear_young\n" + " prfm pstl1strm, %2\n" + "1: ldxr %0, %2\n" + " ubfx %w3, %w0, %5, #1 // extract PTE_AF (young)\n" + " and %0, %0, %4 // clear PTE_AF\n" + " stxr %w1, %0, %2\n" + " cbnz %w1, 1b\n" + : "=&r" (pteval), "=&r" (tmp), "+Q" (pte_val(*ptep)), "=&r" (res) + : "L" (~PTE_AF), "I" (ilog2(PTE_AF))); + + return res; +} + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#define __HAVE_ARCH_PMDP_TEST_AND_CLEAR_YOUNG +static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma, + unsigned long address, + pmd_t *pmdp) +{ + return ptep_test_and_clear_young(vma, address, (pte_t *)pmdp); +} +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + +#define __HAVE_ARCH_PTEP_GET_AND_CLEAR +static inline pte_t ptep_get_and_clear(struct mm_struct *mm, + unsigned long address, pte_t *ptep) +{ + pteval_t old_pteval; + unsigned int tmp; + + asm volatile("// ptep_get_and_clear\n" + " prfm pstl1strm, %2\n" + "1: ldxr %0, %2\n" + " stxr %w1, xzr, %2\n" + " cbnz %w1, 1b\n" + : "=&r" (old_pteval), "=&r" (tmp), "+Q" (pte_val(*ptep))); + + return __pte(old_pteval); +} + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#define __HAVE_ARCH_PMDP_GET_AND_CLEAR +static inline pmd_t pmdp_get_and_clear(struct mm_struct *mm, + unsigned long address, pmd_t *pmdp) +{ + return pte_pmd(ptep_get_and_clear(mm, address, (pte_t *)pmdp)); +} +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + +/* + * ptep_set_wrprotect - mark read-only while trasferring potential hardware + * dirty status (PTE_DBM && !PTE_RDONLY) to the software PTE_DIRTY bit. + */ +#define __HAVE_ARCH_PTEP_SET_WRPROTECT +static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address, pte_t *ptep) +{ + pteval_t pteval; + unsigned long tmp; + + asm volatile("// ptep_set_wrprotect\n" + " prfm pstl1strm, %2\n" + "1: ldxr %0, %2\n" + " tst %0, %4 // check for hw dirty (!PTE_RDONLY)\n" + " csel %1, %3, xzr, eq // set PTE_DIRTY|PTE_RDONLY if dirty\n" + " orr %0, %0, %1 // if !dirty, PTE_RDONLY is already set\n" + " and %0, %0, %5 // clear PTE_WRITE/PTE_DBM\n" + " stxr %w1, %0, %2\n" + " cbnz %w1, 1b\n" + : "=&r" (pteval), "=&r" (tmp), "+Q" (pte_val(*ptep)) + : "r" (PTE_DIRTY|PTE_RDONLY), "L" (PTE_RDONLY), "L" (~PTE_WRITE) + : "cc"); +} + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#define __HAVE_ARCH_PMDP_SET_WRPROTECT +static inline void pmdp_set_wrprotect(struct mm_struct *mm, + unsigned long address, pmd_t *pmdp) +{ + ptep_set_wrprotect(mm, address, (pte_t *)pmdp); +} +#endif +#endif /* CONFIG_ARM64_HW_AFDBM */ + extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; extern pgd_t idmap_pg_dir[PTRS_PER_PGD]; @@ -505,6 +643,21 @@ extern int kern_addr_valid(unsigned long addr); #define pgtable_cache_init() do { } while (0) +/* + * On AArch64, the cache coherency is handled via the set_pte_at() function. + */ +static inline void update_mmu_cache(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep) +{ + /* + * set_pte() does not have a DSB for user mappings, so make sure that + * the page table write is visible. + */ + dsb(ishst); +} + +#define update_mmu_cache_pmd(vma, address, pmd) do { } while (0) + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index e4c893e54f01a9..98f32355dc9728 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -186,4 +186,6 @@ static inline void spin_lock_prefetch(const void *x) #endif +void cpu_enable_pan(void); + #endif /* __ASM_PROCESSOR_H */ diff --git a/arch/arm64/include/asm/psci.h b/arch/arm64/include/asm/psci.h deleted file mode 100644 index 49d7e1aaebdce6..00000000000000 --- a/arch/arm64/include/asm/psci.h +++ /dev/null @@ -1,28 +0,0 @@ -/* - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * Copyright (C) 2013 ARM Limited - */ - -#ifndef __ASM_PSCI_H -#define __ASM_PSCI_H - -int __init psci_dt_init(void); - -#ifdef CONFIG_ACPI -int __init psci_acpi_init(void); -bool __init acpi_psci_present(void); -bool __init acpi_psci_use_hvc(void); -#else -static inline int psci_acpi_init(void) { return 0; } -static inline bool acpi_psci_present(void) { return false; } -#endif - -#endif /* __ASM_PSCI_H */ diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index d6dd9fdbc3bee6..536274ed292ea6 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -183,11 +183,7 @@ static inline int valid_user_regs(struct user_pt_regs *regs) #define instruction_pointer(regs) ((unsigned long)(regs)->pc) -#ifdef CONFIG_SMP extern unsigned long profile_pc(struct pt_regs *regs); -#else -#define profile_pc(regs) instruction_pointer(regs) -#endif #endif /* __ASSEMBLY__ */ #endif diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index db02be81b90a1c..d9c3d6a6100ac5 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -20,10 +20,6 @@ #include #include -#ifndef CONFIG_SMP -# error " included in non-SMP build" -#endif - #define raw_smp_processor_id() (current_thread_info()->cpu) struct seq_file; diff --git a/arch/arm64/include/asm/smp_plat.h b/arch/arm64/include/asm/smp_plat.h index 7abf7570c00f8e..af58dcdefb21a0 100644 --- a/arch/arm64/include/asm/smp_plat.h +++ b/arch/arm64/include/asm/smp_plat.h @@ -56,6 +56,4 @@ static inline int get_logical_index(u64 mpidr) return -EINVAL; } -void __init do_post_cpus_up_work(void); - #endif /* __ASM_SMP_PLAT_H */ diff --git a/arch/arm64/include/asm/spinlock.h b/arch/arm64/include/asm/spinlock.h index cee128732435c7..c85e96d174a5fb 100644 --- a/arch/arm64/include/asm/spinlock.h +++ b/arch/arm64/include/asm/spinlock.h @@ -16,6 +16,7 @@ #ifndef __ASM_SPINLOCK_H #define __ASM_SPINLOCK_H +#include #include #include @@ -38,11 +39,21 @@ static inline void arch_spin_lock(arch_spinlock_t *lock) asm volatile( /* Atomically increment the next ticket. */ + ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ " prfm pstl1strm, %3\n" "1: ldaxr %w0, %3\n" " add %w1, %w0, %w5\n" " stxr %w2, %w1, %3\n" -" cbnz %w2, 1b\n" +" cbnz %w2, 1b\n", + /* LSE atomics */ +" mov %w2, %w5\n" +" ldadda %w2, %w0, %3\n" +" nop\n" +" nop\n" +" nop\n" + ) + /* Did we get the lock? */ " eor %w1, %w0, %w0, ror #16\n" " cbz %w1, 3f\n" @@ -67,15 +78,25 @@ static inline int arch_spin_trylock(arch_spinlock_t *lock) unsigned int tmp; arch_spinlock_t lockval; - asm volatile( -" prfm pstl1strm, %2\n" -"1: ldaxr %w0, %2\n" -" eor %w1, %w0, %w0, ror #16\n" -" cbnz %w1, 2f\n" -" add %w0, %w0, %3\n" -" stxr %w1, %w0, %2\n" -" cbnz %w1, 1b\n" -"2:" + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " prfm pstl1strm, %2\n" + "1: ldaxr %w0, %2\n" + " eor %w1, %w0, %w0, ror #16\n" + " cbnz %w1, 2f\n" + " add %w0, %w0, %3\n" + " stxr %w1, %w0, %2\n" + " cbnz %w1, 1b\n" + "2:", + /* LSE atomics */ + " ldr %w0, %2\n" + " eor %w1, %w0, %w0, ror #16\n" + " cbnz %w1, 1f\n" + " add %w1, %w0, %3\n" + " casa %w0, %w1, %2\n" + " and %w1, %w1, #0xffff\n" + " eor %w1, %w1, %w0, lsr #16\n" + "1:") : "=&r" (lockval), "=&r" (tmp), "+Q" (*lock) : "I" (1 << TICKET_SHIFT) : "memory"); @@ -85,10 +106,19 @@ static inline int arch_spin_trylock(arch_spinlock_t *lock) static inline void arch_spin_unlock(arch_spinlock_t *lock) { - asm volatile( -" stlrh %w1, %0\n" - : "=Q" (lock->owner) - : "r" (lock->owner + 1) + unsigned long tmp; + + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " ldrh %w1, %0\n" + " add %w1, %w1, #1\n" + " stlrh %w1, %0", + /* LSE atomics */ + " mov %w1, #1\n" + " nop\n" + " staddlh %w1, %0") + : "=Q" (lock->owner), "=&r" (tmp) + : : "memory"); } @@ -123,13 +153,24 @@ static inline void arch_write_lock(arch_rwlock_t *rw) { unsigned int tmp; - asm volatile( + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ " sevl\n" "1: wfe\n" "2: ldaxr %w0, %1\n" " cbnz %w0, 1b\n" " stxr %w0, %w2, %1\n" " cbnz %w0, 2b\n" + " nop", + /* LSE atomics */ + "1: mov %w0, wzr\n" + "2: casa %w0, %w2, %1\n" + " cbz %w0, 3f\n" + " ldxr %w0, %1\n" + " cbz %w0, 2b\n" + " wfe\n" + " b 1b\n" + "3:") : "=&r" (tmp), "+Q" (rw->lock) : "r" (0x80000000) : "memory"); @@ -139,11 +180,18 @@ static inline int arch_write_trylock(arch_rwlock_t *rw) { unsigned int tmp; - asm volatile( - " ldaxr %w0, %1\n" - " cbnz %w0, 1f\n" + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + "1: ldaxr %w0, %1\n" + " cbnz %w0, 2f\n" " stxr %w0, %w2, %1\n" - "1:\n" + " cbnz %w0, 1b\n" + "2:", + /* LSE atomics */ + " mov %w0, wzr\n" + " casa %w0, %w2, %1\n" + " nop\n" + " nop") : "=&r" (tmp), "+Q" (rw->lock) : "r" (0x80000000) : "memory"); @@ -153,9 +201,10 @@ static inline int arch_write_trylock(arch_rwlock_t *rw) static inline void arch_write_unlock(arch_rwlock_t *rw) { - asm volatile( - " stlr %w1, %0\n" - : "=Q" (rw->lock) : "r" (0) : "memory"); + asm volatile(ARM64_LSE_ATOMIC_INSN( + " stlr wzr, %0", + " swpl wzr, wzr, %0") + : "=Q" (rw->lock) :: "memory"); } /* write_can_lock - would write_trylock() succeed? */ @@ -172,6 +221,10 @@ static inline void arch_write_unlock(arch_rwlock_t *rw) * * The memory barriers are implicit with the load-acquire and store-release * instructions. + * + * Note that in UNDEFINED cases, such as unlocking a lock twice, the LL/SC + * and LSE implementations may exhibit different behaviour (although this + * will have no effect on lockdep). */ static inline void arch_read_lock(arch_rwlock_t *rw) { @@ -179,26 +232,43 @@ static inline void arch_read_lock(arch_rwlock_t *rw) asm volatile( " sevl\n" + ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ "1: wfe\n" "2: ldaxr %w0, %2\n" " add %w0, %w0, #1\n" " tbnz %w0, #31, 1b\n" " stxr %w1, %w0, %2\n" - " cbnz %w1, 2b\n" + " nop\n" + " cbnz %w1, 2b", + /* LSE atomics */ + "1: wfe\n" + "2: ldxr %w0, %2\n" + " adds %w1, %w0, #1\n" + " tbnz %w1, #31, 1b\n" + " casa %w0, %w1, %2\n" + " sbc %w0, %w1, %w0\n" + " cbnz %w0, 2b") : "=&r" (tmp), "=&r" (tmp2), "+Q" (rw->lock) : - : "memory"); + : "cc", "memory"); } static inline void arch_read_unlock(arch_rwlock_t *rw) { unsigned int tmp, tmp2; - asm volatile( + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ "1: ldxr %w0, %2\n" " sub %w0, %w0, #1\n" " stlxr %w1, %w0, %2\n" - " cbnz %w1, 1b\n" + " cbnz %w1, 1b", + /* LSE atomics */ + " movn %w0, #0\n" + " nop\n" + " nop\n" + " staddl %w0, %2") : "=&r" (tmp), "=&r" (tmp2), "+Q" (rw->lock) : : "memory"); @@ -206,17 +276,28 @@ static inline void arch_read_unlock(arch_rwlock_t *rw) static inline int arch_read_trylock(arch_rwlock_t *rw) { - unsigned int tmp, tmp2 = 1; + unsigned int tmp, tmp2; - asm volatile( - " ldaxr %w0, %2\n" + asm volatile(ARM64_LSE_ATOMIC_INSN( + /* LL/SC */ + " mov %w1, #1\n" + "1: ldaxr %w0, %2\n" " add %w0, %w0, #1\n" - " tbnz %w0, #31, 1f\n" + " tbnz %w0, #31, 2f\n" " stxr %w1, %w0, %2\n" - "1:\n" - : "=&r" (tmp), "+r" (tmp2), "+Q" (rw->lock) + " cbnz %w1, 1b\n" + "2:", + /* LSE atomics */ + " ldr %w0, %2\n" + " adds %w1, %w0, #1\n" + " tbnz %w1, #31, 1f\n" + " casa %w0, %w1, %2\n" + " sbc %w1, %w1, %w0\n" + " nop\n" + "1:") + : "=&r" (tmp), "=&r" (tmp2), "+Q" (rw->lock) : - : "memory"); + : "cc", "memory"); return !tmp2; } diff --git a/arch/arm64/include/asm/spinlock_types.h b/arch/arm64/include/asm/spinlock_types.h index b8d383665f56b0..55be59a35e3fe9 100644 --- a/arch/arm64/include/asm/spinlock_types.h +++ b/arch/arm64/include/asm/spinlock_types.h @@ -20,6 +20,8 @@ # error "please don't include this file directly" #endif +#include + #define TICKET_SHIFT 16 typedef struct { diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 5c89df0acbcb26..a7f3d4b2514d61 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -20,8 +20,29 @@ #ifndef __ASM_SYSREG_H #define __ASM_SYSREG_H +#include + +#define SCTLR_EL1_CP15BEN (0x1 << 5) +#define SCTLR_EL1_SED (0x1 << 8) + +/* + * ARMv8 ARM reserves the following encoding for system registers: + * (Ref: ARMv8 ARM, Section: "System instruction class encoding overview", + * C5.2, version:ARM DDI 0487A.f) + * [20-19] : Op0 + * [18-16] : Op1 + * [15-12] : CRn + * [11-8] : CRm + * [7-5] : Op2 + */ #define sys_reg(op0, op1, crn, crm, op2) \ - ((((op0)-2)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5)) + ((((op0)&3)<<19)|((op1)<<16)|((crn)<<12)|((crm)<<8)|((op2)<<5)) + +#define REG_PSTATE_PAN_IMM sys_reg(0, 0, 4, 0, 4) +#define SCTLR_EL1_SPAN (1 << 23) + +#define SET_PSTATE_PAN(x) __inst_arm(0xd5000000 | REG_PSTATE_PAN_IMM |\ + (!!x)<<8 | 0x1f) #ifdef __ASSEMBLY__ @@ -31,11 +52,11 @@ .equ __reg_num_xzr, 31 .macro mrs_s, rt, sreg - .inst 0xd5300000|(\sreg)|(__reg_num_\rt) + .inst 0xd5200000|(\sreg)|(__reg_num_\rt) .endm .macro msr_s, sreg, rt - .inst 0xd5100000|(\sreg)|(__reg_num_\rt) + .inst 0xd5000000|(\sreg)|(__reg_num_\rt) .endm #else @@ -47,14 +68,23 @@ asm( " .equ __reg_num_xzr, 31\n" "\n" " .macro mrs_s, rt, sreg\n" -" .inst 0xd5300000|(\\sreg)|(__reg_num_\\rt)\n" +" .inst 0xd5200000|(\\sreg)|(__reg_num_\\rt)\n" " .endm\n" "\n" " .macro msr_s, sreg, rt\n" -" .inst 0xd5100000|(\\sreg)|(__reg_num_\\rt)\n" +" .inst 0xd5000000|(\\sreg)|(__reg_num_\\rt)\n" " .endm\n" ); +static inline void config_sctlr_el1(u32 clear, u32 set) +{ + u32 val; + + asm volatile("mrs %0, sctlr_el1" : "=r" (val)); + val &= ~clear; + val |= set; + asm volatile("msr sctlr_el1, %0" : : "r" (val)); +} #endif #endif /* __ASM_SYSREG_H */ diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h index 3a0242c7eb8dab..d6e6b666038032 100644 --- a/arch/arm64/include/asm/tlb.h +++ b/arch/arm64/include/asm/tlb.h @@ -41,7 +41,12 @@ static inline void tlb_flush(struct mmu_gather *tlb) flush_tlb_mm(tlb->mm); } else { struct vm_area_struct vma = { .vm_mm = tlb->mm, }; - flush_tlb_range(&vma, tlb->start, tlb->end); + /* + * The intermediate page table levels are already handled by + * the __(pte|pmd|pud)_free_tlb() functions, so last level + * TLBI is sufficient here. + */ + __flush_tlb_range(&vma, tlb->start, tlb->end, true); } } diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index 934815d45eda0f..7bd2da021658ad 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -87,27 +87,56 @@ static inline void flush_tlb_page(struct vm_area_struct *vma, ((unsigned long)ASID(vma->vm_mm) << 48); dsb(ishst); - asm("tlbi vae1is, %0" : : "r" (addr)); + asm("tlbi vale1is, %0" : : "r" (addr)); dsb(ish); } +/* + * This is meant to avoid soft lock-ups on large TLB flushing ranges and not + * necessarily a performance improvement. + */ +#define MAX_TLB_RANGE (1024UL << PAGE_SHIFT) + static inline void __flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) + unsigned long start, unsigned long end, + bool last_level) { unsigned long asid = (unsigned long)ASID(vma->vm_mm) << 48; unsigned long addr; + + if ((end - start) > MAX_TLB_RANGE) { + flush_tlb_mm(vma->vm_mm); + return; + } + start = asid | (start >> 12); end = asid | (end >> 12); dsb(ishst); - for (addr = start; addr < end; addr += 1 << (PAGE_SHIFT - 12)) - asm("tlbi vae1is, %0" : : "r"(addr)); + for (addr = start; addr < end; addr += 1 << (PAGE_SHIFT - 12)) { + if (last_level) + asm("tlbi vale1is, %0" : : "r"(addr)); + else + asm("tlbi vae1is, %0" : : "r"(addr)); + } dsb(ish); } -static inline void __flush_tlb_kernel_range(unsigned long start, unsigned long end) +static inline void flush_tlb_range(struct vm_area_struct *vma, + unsigned long start, unsigned long end) +{ + __flush_tlb_range(vma, start, end, false); +} + +static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end) { unsigned long addr; + + if ((end - start) > MAX_TLB_RANGE) { + flush_tlb_all(); + return; + } + start >>= 12; end >>= 12; @@ -118,29 +147,6 @@ static inline void __flush_tlb_kernel_range(unsigned long start, unsigned long e isb(); } -/* - * This is meant to avoid soft lock-ups on large TLB flushing ranges and not - * necessarily a performance improvement. - */ -#define MAX_TLB_RANGE (1024UL << PAGE_SHIFT) - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - if ((end - start) <= MAX_TLB_RANGE) - __flush_tlb_range(vma, start, end); - else - flush_tlb_mm(vma->vm_mm); -} - -static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end) -{ - if ((end - start) <= MAX_TLB_RANGE) - __flush_tlb_kernel_range(start, end); - else - flush_tlb_all(); -} - /* * Used to invalidate the TLB (walk caches) corresponding to intermediate page * table levels (pgd/pud/pmd). @@ -154,20 +160,6 @@ static inline void __flush_tlb_pgtable(struct mm_struct *mm, asm("tlbi vae1is, %0" : : "r" (addr)); dsb(ish); } -/* - * On AArch64, the cache coherency is handled via the set_pte_at() function. - */ -static inline void update_mmu_cache(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) -{ - /* - * set_pte() does not have a DSB for user mappings, so make sure that - * the page table write is visible. - */ - dsb(ishst); -} - -#define update_mmu_cache_pmd(vma, address, pmd) do { } while (0) #endif diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h index 225ec3524fbfc6..a3e9d6fdbf2136 100644 --- a/arch/arm64/include/asm/topology.h +++ b/arch/arm64/include/asm/topology.h @@ -1,8 +1,6 @@ #ifndef __ASM_TOPOLOGY_H #define __ASM_TOPOLOGY_H -#ifdef CONFIG_SMP - #include struct cpu_topology { @@ -24,13 +22,6 @@ void init_cpu_topology(void); void store_cpu_topology(unsigned int cpuid); const struct cpumask *cpu_coregroup_mask(int cpu); -#else - -static inline void init_cpu_topology(void) { } -static inline void store_cpu_topology(unsigned int cpuid) { } - -#endif - #include #endif /* _ASM_ARM_TOPOLOGY_H */ diff --git a/arch/arm64/include/asm/traps.h b/arch/arm64/include/asm/traps.h index 232e4ba5d314ca..0cc2f29bf9da34 100644 --- a/arch/arm64/include/asm/traps.h +++ b/arch/arm64/include/asm/traps.h @@ -34,13 +34,32 @@ struct undef_hook { void register_undef_hook(struct undef_hook *hook); void unregister_undef_hook(struct undef_hook *hook); +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +static inline int __in_irqentry_text(unsigned long ptr) +{ + extern char __irqentry_text_start[]; + extern char __irqentry_text_end[]; + + return ptr >= (unsigned long)&__irqentry_text_start && + ptr < (unsigned long)&__irqentry_text_end; +} +#else +static inline int __in_irqentry_text(unsigned long ptr) +{ + return 0; +} +#endif + static inline int in_exception_text(unsigned long ptr) { extern char __exception_text_start[]; extern char __exception_text_end[]; + int in; + + in = ptr >= (unsigned long)&__exception_text_start && + ptr < (unsigned long)&__exception_text_end; - return ptr >= (unsigned long)&__exception_text_start && - ptr < (unsigned long)&__exception_text_end; + return in ? : __in_irqentry_text(ptr); } #endif diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 07e1ba449bf1e0..b2ede967fe7d49 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -24,7 +24,10 @@ #include #include +#include +#include #include +#include #include #include #include @@ -131,6 +134,8 @@ static inline void set_fs(mm_segment_t fs) do { \ unsigned long __gu_val; \ __chk_user_ptr(ptr); \ + asm(ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN)); \ switch (sizeof(*(ptr))) { \ case 1: \ __get_user_asm("ldrb", "%w", __gu_val, (ptr), (err)); \ @@ -148,6 +153,8 @@ do { \ BUILD_BUG(); \ } \ (x) = (__force __typeof__(*(ptr)))__gu_val; \ + asm(ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN)); \ } while (0) #define __get_user(x, ptr) \ @@ -194,6 +201,8 @@ do { \ do { \ __typeof__(*(ptr)) __pu_val = (x); \ __chk_user_ptr(ptr); \ + asm(ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN)); \ switch (sizeof(*(ptr))) { \ case 1: \ __put_user_asm("strb", "%w", __pu_val, (ptr), (err)); \ @@ -210,6 +219,8 @@ do { \ default: \ BUILD_BUG(); \ } \ + asm(ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN)); \ } while (0) #define __put_user(x, ptr) \ diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h index 73cf0f54d57cc2..361c8a8ef55f37 100644 --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -27,5 +27,6 @@ #define HWCAP_SHA1 (1 << 5) #define HWCAP_SHA2 (1 << 6) #define HWCAP_CRC32 (1 << 7) +#define HWCAP_ATOMICS (1 << 8) #endif /* _UAPI__ASM_HWCAP_H */ diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h index 6913643bbe54eb..208db3df135a48 100644 --- a/arch/arm64/include/uapi/asm/ptrace.h +++ b/arch/arm64/include/uapi/asm/ptrace.h @@ -44,6 +44,7 @@ #define PSR_I_BIT 0x00000080 #define PSR_A_BIT 0x00000100 #define PSR_D_BIT 0x00000200 +#define PSR_PAN_BIT 0x00400000 #define PSR_Q_BIT 0x08000000 #define PSR_V_BIT 0x10000000 #define PSR_C_BIT 0x20000000 diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 426d0763c81bf3..22dc9bc781be60 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -17,15 +17,15 @@ arm64-obj-y := debug-monitors.o entry.o irq.o fpsimd.o \ sys.o stacktrace.o time.o traps.o io.o vdso.o \ hyp-stub.o psci.o psci-call.o cpu_ops.o insn.o \ return_address.o cpuinfo.o cpu_errata.o \ - cpufeature.o alternative.o cacheinfo.o + cpufeature.o alternative.o cacheinfo.o \ + smp.o smp_spin_table.o topology.o arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \ sys_compat.o entry32.o \ ../../arm/kernel/opcodes.o arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o -arm64-obj-$(CONFIG_SMP) += smp.o smp_spin_table.o topology.o -arm64-obj-$(CONFIG_PERF_EVENTS) += perf_regs.o +arm64-obj-$(CONFIG_PERF_EVENTS) += perf_regs.o perf_callchain.o arm64-obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o arm64-obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o arm64-obj-$(CONFIG_CPU_PM) += sleep.o suspend.o diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index 221b98312f0c6a..ab9db0e9818c0c 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -85,7 +85,7 @@ static u32 get_alt_insn(struct alt_instr *alt, u32 *insnptr, u32 *altinsnptr) return insn; } -static int __apply_alternatives(void *alt_region) +static void __apply_alternatives(void *alt_region) { struct alt_instr *alt; struct alt_region *region = alt_region; @@ -114,19 +114,39 @@ static int __apply_alternatives(void *alt_region) flush_icache_range((uintptr_t)origptr, (uintptr_t)(origptr + nr_inst)); } - - return 0; } -void apply_alternatives_all(void) +/* + * We might be patching the stop_machine state machine, so implement a + * really simple polling protocol here. + */ +static int __apply_alternatives_multi_stop(void *unused) { + static int patched = 0; struct alt_region region = { .begin = __alt_instructions, .end = __alt_instructions_end, }; + /* We always have a CPU 0 at this point (__init) */ + if (smp_processor_id()) { + while (!READ_ONCE(patched)) + cpu_relax(); + isb(); + } else { + BUG_ON(patched); + __apply_alternatives(®ion); + /* Barriers provided by the cache flushing */ + WRITE_ONCE(patched, 1); + } + + return 0; +} + +void __init apply_alternatives_all(void) +{ /* better not try code patching on a live SMP system */ - stop_machine(__apply_alternatives, ®ion, NULL); + stop_machine(__apply_alternatives_multi_stop, NULL, cpu_online_mask); } void apply_alternatives(void *start, size_t length) diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c index 7922c2e710cadf..bcee7abac68ebb 100644 --- a/arch/arm64/kernel/armv8_deprecated.c +++ b/arch/arm64/kernel/armv8_deprecated.c @@ -14,8 +14,11 @@ #include #include +#include +#include #include #include +#include #include #include #include @@ -279,6 +282,8 @@ static void register_insn_emulation_sysctl(struct ctl_table *table) */ #define __user_swpX_asm(data, addr, res, temp, B) \ __asm__ __volatile__( \ + ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) \ " mov %w2, %w1\n" \ "0: ldxr"B" %w1, [%3]\n" \ "1: stxr"B" %w0, %w2, [%3]\n" \ @@ -294,7 +299,9 @@ static void register_insn_emulation_sysctl(struct ctl_table *table) " .align 3\n" \ " .quad 0b, 3b\n" \ " .quad 1b, 3b\n" \ - " .popsection" \ + " .popsection\n" \ + ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) \ : "=&r" (res), "+r" (data), "=&r" (temp) \ : "r" (addr), "i" (-EAGAIN), "i" (-EFAULT) \ : "memory") @@ -504,16 +511,6 @@ static int cp15barrier_handler(struct pt_regs *regs, u32 instr) return 0; } -static inline void config_sctlr_el1(u32 clear, u32 set) -{ - u32 val; - - asm volatile("mrs %0, sctlr_el1" : "=r" (val)); - val &= ~clear; - val |= set; - asm volatile("msr sctlr_el1, %0" : : "r" (val)); -} - static int cp15_barrier_set_hw_mode(bool enable) { if (enable) diff --git a/arch/arm64/kernel/cpu_ops.c b/arch/arm64/kernel/cpu_ops.c index 5ea337dd2f15d2..b6bd7d44776833 100644 --- a/arch/arm64/kernel/cpu_ops.c +++ b/arch/arm64/kernel/cpu_ops.c @@ -30,9 +30,7 @@ extern const struct cpu_operations cpu_psci_ops; const struct cpu_operations *cpu_ops[NR_CPUS]; static const struct cpu_operations *supported_cpu_ops[] __initconst = { -#ifdef CONFIG_SMP &smp_spin_table_ops, -#endif &cpu_psci_ops, NULL, }; diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 5ad86ceac01016..3c9aed32f70b21 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -21,24 +21,57 @@ #include #include #include +#include static bool -has_id_aa64pfr0_feature(const struct arm64_cpu_capabilities *entry) +feature_matches(u64 reg, const struct arm64_cpu_capabilities *entry) { - u64 val; + int val = cpuid_feature_extract_field(reg, entry->field_pos); - val = read_cpuid(id_aa64pfr0_el1); - return (val & entry->register_mask) == entry->register_value; + return val >= entry->min_field_value; } +#define __ID_FEAT_CHK(reg) \ +static bool __maybe_unused \ +has_##reg##_feature(const struct arm64_cpu_capabilities *entry) \ +{ \ + u64 val; \ + \ + val = read_cpuid(reg##_el1); \ + return feature_matches(val, entry); \ +} + +__ID_FEAT_CHK(id_aa64pfr0); +__ID_FEAT_CHK(id_aa64mmfr1); +__ID_FEAT_CHK(id_aa64isar0); + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", .capability = ARM64_HAS_SYSREG_GIC_CPUIF, .matches = has_id_aa64pfr0_feature, - .register_mask = (0xf << 24), - .register_value = (1 << 24), + .field_pos = 24, + .min_field_value = 1, + }, +#ifdef CONFIG_ARM64_PAN + { + .desc = "Privileged Access Never", + .capability = ARM64_HAS_PAN, + .matches = has_id_aa64mmfr1_feature, + .field_pos = 20, + .min_field_value = 1, + .enable = cpu_enable_pan, }, +#endif /* CONFIG_ARM64_PAN */ +#if defined(CONFIG_AS_LSE) && defined(CONFIG_ARM64_LSE_ATOMICS) + { + .desc = "LSE atomic instructions", + .capability = ARM64_HAS_LSE_ATOMICS, + .matches = has_id_aa64isar0_feature, + .field_pos = 20, + .min_field_value = 2, + }, +#endif /* CONFIG_AS_LSE && CONFIG_ARM64_LSE_ATOMICS */ {}, }; @@ -55,9 +88,15 @@ void check_cpu_capabilities(const struct arm64_cpu_capabilities *caps, pr_info("%s %s\n", info, caps[i].desc); cpus_set_cap(caps[i].capability); } + + /* second pass allows enable() to consider interacting capabilities */ + for (i = 0; caps[i].desc; i++) { + if (cpus_have_cap(caps[i].capability) && caps[i].enable) + caps[i].enable(); + } } void check_local_cpu_features(void) { - check_cpu_capabilities(arm64_features, "detected feature"); + check_cpu_capabilities(arm64_features, "detected feature:"); } diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c index b056369fd47da0..9b3b62ac9c244b 100644 --- a/arch/arm64/kernel/debug-monitors.c +++ b/arch/arm64/kernel/debug-monitors.c @@ -82,7 +82,7 @@ early_param("nodebugmon", early_debug_disable); static DEFINE_PER_CPU(int, mde_ref_count); static DEFINE_PER_CPU(int, kde_ref_count); -void enable_debug_monitors(enum debug_el el) +void enable_debug_monitors(enum dbg_active_el el) { u32 mdscr, enable = 0; @@ -102,7 +102,7 @@ void enable_debug_monitors(enum debug_el el) } } -void disable_debug_monitors(enum debug_el el) +void disable_debug_monitors(enum dbg_active_el el) { u32 mdscr, disable = 0; diff --git a/arch/arm64/kernel/efi-stub.c b/arch/arm64/kernel/efi-stub.c index f5374065ad535f..816120ece6bcec 100644 --- a/arch/arm64/kernel/efi-stub.c +++ b/arch/arm64/kernel/efi-stub.c @@ -13,7 +13,7 @@ #include #include -efi_status_t __init handle_kernel_image(efi_system_table_t *sys_table, +efi_status_t __init handle_kernel_image(efi_system_table_t *sys_table_arg, unsigned long *image_addr, unsigned long *image_size, unsigned long *reserve_addr, @@ -23,21 +23,44 @@ efi_status_t __init handle_kernel_image(efi_system_table_t *sys_table, { efi_status_t status; unsigned long kernel_size, kernel_memsize = 0; + unsigned long nr_pages; + void *old_image_addr = (void *)*image_addr; /* Relocate the image, if required. */ kernel_size = _edata - _text; if (*image_addr != (dram_base + TEXT_OFFSET)) { kernel_memsize = kernel_size + (_end - _edata); - status = efi_low_alloc(sys_table, kernel_memsize + TEXT_OFFSET, - SZ_2M, reserve_addr); + + /* + * First, try a straight allocation at the preferred offset. + * This will work around the issue where, if dram_base == 0x0, + * efi_low_alloc() refuses to allocate at 0x0 (to prevent the + * address of the allocation to be mistaken for a FAIL return + * value or a NULL pointer). It will also ensure that, on + * platforms where the [dram_base, dram_base + TEXT_OFFSET) + * interval is partially occupied by the firmware (like on APM + * Mustang), we can still place the kernel at the address + * 'dram_base + TEXT_OFFSET'. + */ + *image_addr = *reserve_addr = dram_base + TEXT_OFFSET; + nr_pages = round_up(kernel_memsize, EFI_ALLOC_ALIGN) / + EFI_PAGE_SIZE; + status = efi_call_early(allocate_pages, EFI_ALLOCATE_ADDRESS, + EFI_LOADER_DATA, nr_pages, + (efi_physical_addr_t *)reserve_addr); if (status != EFI_SUCCESS) { - pr_efi_err(sys_table, "Failed to relocate kernel\n"); - return status; + kernel_memsize += TEXT_OFFSET; + status = efi_low_alloc(sys_table_arg, kernel_memsize, + SZ_2M, reserve_addr); + + if (status != EFI_SUCCESS) { + pr_efi_err(sys_table_arg, "Failed to relocate kernel\n"); + return status; + } + *image_addr = *reserve_addr + TEXT_OFFSET; } - memcpy((void *)*reserve_addr + TEXT_OFFSET, (void *)*image_addr, - kernel_size); - *image_addr = *reserve_addr + TEXT_OFFSET; - *reserve_size = kernel_memsize + TEXT_OFFSET; + memcpy((void *)*image_addr, old_image_addr, kernel_size); + *reserve_size = kernel_memsize; } diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index e16351819fed9a..4306c937b1ffc4 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -116,41 +116,34 @@ */ .endm - .macro kernel_exit, el, ret = 0 + .macro kernel_exit, el ldp x21, x22, [sp, #S_PC] // load ELR, SPSR .if \el == 0 ct_user_enter ldr x23, [sp, #S_SP] // load return stack pointer msr sp_el0, x23 - #ifdef CONFIG_ARM64_ERRATUM_845719 - -#undef SEQUENCE_ORG -#undef SEQUENCE_ALT - +alternative_if_not ARM64_WORKAROUND_845719 + nop + nop #ifdef CONFIG_PID_IN_CONTEXTIDR - -#define SEQUENCE_ORG "nop ; nop ; nop" -#define SEQUENCE_ALT "tbz x22, #4, 1f ; mrs x29, contextidr_el1; msr contextidr_el1, x29; 1:" - + nop +#endif +alternative_else + tbz x22, #4, 1f +#ifdef CONFIG_PID_IN_CONTEXTIDR + mrs x29, contextidr_el1 + msr contextidr_el1, x29 #else - -#define SEQUENCE_ORG "nop ; nop" -#define SEQUENCE_ALT "tbz x22, #4, 1f ; msr contextidr_el1, xzr; 1:" - + msr contextidr_el1, xzr #endif - - alternative_insn SEQUENCE_ORG, SEQUENCE_ALT, ARM64_WORKAROUND_845719 - +1: +alternative_endif #endif .endif msr elr_el1, x21 // set up the return data msr spsr_el1, x22 - .if \ret - ldr x1, [sp, #S_X1] // preserve x0 (syscall return) - .else ldp x0, x1, [sp, #16 * 0] - .endif ldp x2, x3, [sp, #16 * 1] ldp x4, x5, [sp, #16 * 2] ldp x6, x7, [sp, #16 * 3] @@ -613,22 +606,21 @@ ENDPROC(cpu_switch_to) */ ret_fast_syscall: disable_irq // disable interrupts + str x0, [sp, #S_X0] // returned x0 ldr x1, [tsk, #TI_FLAGS] // re-check for syscall tracing and x2, x1, #_TIF_SYSCALL_WORK cbnz x2, ret_fast_syscall_trace and x2, x1, #_TIF_WORK_MASK - cbnz x2, fast_work_pending + cbnz x2, work_pending enable_step_tsk x1, x2 - kernel_exit 0, ret = 1 + kernel_exit 0 ret_fast_syscall_trace: enable_irq // enable interrupts - b __sys_trace_return + b __sys_trace_return_skipped // we already saved x0 /* * Ok, we need to do extra processing, enter the slow path. */ -fast_work_pending: - str x0, [sp, #S_X0] // returned x0 work_pending: tbnz x1, #TIF_NEED_RESCHED, work_resched /* TIF_SIGPENDING, TIF_NOTIFY_RESUME or TIF_FOREIGN_FPSTATE case */ @@ -652,7 +644,7 @@ ret_to_user: cbnz x2, work_pending enable_step_tsk x1, x2 no_work_pending: - kernel_exit 0, ret = 0 + kernel_exit 0 ENDPROC(ret_to_user) /* diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 44d6f7545505f9..c56956a16d3f0c 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -158,6 +158,7 @@ void fpsimd_thread_switch(struct task_struct *next) void fpsimd_flush_thread(void) { memset(¤t->thread.fpsimd_state, 0, sizeof(struct fpsimd_state)); + fpsimd_flush_task_state(current); set_thread_flag(TIF_FOREIGN_FPSTATE); } diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index c0ff3ce4299e97..a055be6125cf59 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -62,13 +62,8 @@ /* * Initial memory map attributes. */ -#ifndef CONFIG_SMP -#define PTE_FLAGS PTE_TYPE_PAGE | PTE_AF -#define PMD_FLAGS PMD_TYPE_SECT | PMD_SECT_AF -#else #define PTE_FLAGS PTE_TYPE_PAGE | PTE_AF | PTE_SHARED #define PMD_FLAGS PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S -#endif #ifdef CONFIG_ARM64_64K_PAGES #define MM_MMUFLAGS PTE_ATTRINDX(MT_NORMAL) | PTE_FLAGS @@ -574,7 +569,6 @@ ENTRY(__boot_cpu_mode) .long BOOT_CPU_MODE_EL1 .popsection -#ifdef CONFIG_SMP /* * This provides a "holding pen" for platforms to hold all secondary * cores are held until we're ready for them to initialise. @@ -622,7 +616,6 @@ ENTRY(__secondary_switched) mov x29, #0 b secondary_start_kernel ENDPROC(__secondary_switched) -#endif /* CONFIG_SMP */ /* * Enable the MMU. @@ -641,5 +634,13 @@ __enable_mmu: isb msr sctlr_el1, x0 isb + /* + * Invalidate the local I-cache so that any instructions fetched + * speculatively from the PoC are discarded, since they may have + * been dynamically patched at the PoU. + */ + ic iallu + dsb nsh + isb br x27 ENDPROC(__enable_mmu) diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c index 7a1a5da6c8c1b9..003bc3d50636f5 100644 --- a/arch/arm64/kernel/hw_breakpoint.c +++ b/arch/arm64/kernel/hw_breakpoint.c @@ -156,7 +156,7 @@ static void write_wb_reg(int reg, int n, u64 val) * Convert a breakpoint privilege level to the corresponding exception * level. */ -static enum debug_el debug_exception_level(int privilege) +static enum dbg_active_el debug_exception_level(int privilege) { switch (privilege) { case AARCH64_BREAKPOINT_EL0: @@ -230,7 +230,7 @@ static int hw_breakpoint_control(struct perf_event *bp, struct perf_event **slots; struct debug_info *debug_info = ¤t->thread.debug; int i, max_slots, ctrl_reg, val_reg, reg_enable; - enum debug_el dbg_el = debug_exception_level(info->ctrl.privilege); + enum dbg_active_el dbg_el = debug_exception_level(info->ctrl.privilege); u32 ctrl; if (info->ctrl.type == ARM_BREAKPOINT_EXECUTE) { @@ -537,7 +537,7 @@ int arch_validate_hwbkpt_settings(struct perf_event *bp) * exception level at the register level. * This is used when single-stepping after a breakpoint exception. */ -static void toggle_bp_registers(int reg, enum debug_el el, int enable) +static void toggle_bp_registers(int reg, enum dbg_active_el el, int enable) { int i, max_slots, privilege; u32 ctrl; diff --git a/arch/arm64/kernel/insn.c b/arch/arm64/kernel/insn.c index dd9671cd0bb255..f341866aa81034 100644 --- a/arch/arm64/kernel/insn.c +++ b/arch/arm64/kernel/insn.c @@ -101,9 +101,8 @@ static void __kprobes *patch_map(void *addr, int fixmap) return addr; BUG_ON(!page); - set_fixmap(fixmap, page_to_phys(page)); - - return (void *) (__fix_to_virt(fixmap) + (uintaddr & ~PAGE_MASK)); + return (void *)set_fixmap_offset(fixmap, page_to_phys(page) + + (uintaddr & ~PAGE_MASK)); } static void __kprobes patch_unmap(int fixmap) diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c index 463fa2e7e34c10..11dc3fd4785372 100644 --- a/arch/arm64/kernel/irq.c +++ b/arch/arm64/kernel/irq.c @@ -33,9 +33,7 @@ unsigned long irq_err_count; int arch_show_interrupts(struct seq_file *p, int prec) { -#ifdef CONFIG_SMP show_ipi_list(p, prec); -#endif seq_printf(p, "%*s: %10lu\n", prec, "Err", irq_err_count); return 0; } diff --git a/arch/arm64/kernel/jump_label.c b/arch/arm64/kernel/jump_label.c index 4f1fec7a46dbb1..c2dd1ad3e648eb 100644 --- a/arch/arm64/kernel/jump_label.c +++ b/arch/arm64/kernel/jump_label.c @@ -28,7 +28,7 @@ void arch_jump_label_transform(struct jump_entry *entry, void *addr = (void *)entry->code; u32 insn; - if (type == JUMP_LABEL_ENABLE) { + if (type == JUMP_LABEL_JMP) { insn = aarch64_insn_gen_branch_imm(entry->code, entry->target, AARCH64_INSN_BRANCH_NOLINK); diff --git a/arch/arm64/kernel/kgdb.c b/arch/arm64/kernel/kgdb.c index a0d10c55f307a5..bcac81e600b9af 100644 --- a/arch/arm64/kernel/kgdb.c +++ b/arch/arm64/kernel/kgdb.c @@ -235,13 +235,13 @@ static int kgdb_step_brk_fn(struct pt_regs *regs, unsigned int esr) static struct break_hook kgdb_brkpt_hook = { .esr_mask = 0xffffffff, - .esr_val = DBG_ESR_VAL_BRK(KGDB_DYN_DBG_BRK_IMM), + .esr_val = (u32)ESR_ELx_VAL_BRK64(KGDB_DYN_DBG_BRK_IMM), .fn = kgdb_brk_fn }; static struct break_hook kgdb_compiled_brkpt_hook = { .esr_mask = 0xffffffff, - .esr_val = DBG_ESR_VAL_BRK(KGDB_COMPILED_DBG_BRK_IMM), + .esr_val = (u32)ESR_ELx_VAL_BRK64(KGDB_COMPILED_DBG_BRK_IMM), .fn = kgdb_compiled_brk_fn }; @@ -328,9 +328,9 @@ void kgdb_arch_exit(void) */ struct kgdb_arch arch_kgdb_ops = { .gdb_bpt_instr = { - KGDB_DYN_BRK_INS_BYTE0, - KGDB_DYN_BRK_INS_BYTE1, - KGDB_DYN_BRK_INS_BYTE2, - KGDB_DYN_BRK_INS_BYTE3, + KGDB_DYN_BRK_INS_BYTE(0), + KGDB_DYN_BRK_INS_BYTE(1), + KGDB_DYN_BRK_INS_BYTE(2), + KGDB_DYN_BRK_INS_BYTE(3), } }; diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c index 4095379dc06983..b3d098bd34aa3d 100644 --- a/arch/arm64/kernel/pci.c +++ b/arch/arm64/kernel/pci.c @@ -38,6 +38,19 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res, return res->start; } +/** + * pcibios_enable_device - Enable I/O and memory. + * @dev: PCI device to be enabled + * @mask: bitmask of BARs to enable + */ +int pcibios_enable_device(struct pci_dev *dev, int mask) +{ + if (pci_has_flag(PCI_PROBE_ONLY)) + return 0; + + return pci_enable_resources(dev, mask); +} + /* * Try to assign the IRQ number from DT when adding a new device */ diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c new file mode 100644 index 00000000000000..3aa74830cc69af --- /dev/null +++ b/arch/arm64/kernel/perf_callchain.c @@ -0,0 +1,196 @@ +/* + * arm64 callchain support + * + * Copyright (C) 2015 ARM Limited + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ +#include +#include + +#include + +struct frame_tail { + struct frame_tail __user *fp; + unsigned long lr; +} __attribute__((packed)); + +/* + * Get the return address for a single stackframe and return a pointer to the + * next frame tail. + */ +static struct frame_tail __user * +user_backtrace(struct frame_tail __user *tail, + struct perf_callchain_entry *entry) +{ + struct frame_tail buftail; + unsigned long err; + + /* Also check accessibility of one struct frame_tail beyond */ + if (!access_ok(VERIFY_READ, tail, sizeof(buftail))) + return NULL; + + pagefault_disable(); + err = __copy_from_user_inatomic(&buftail, tail, sizeof(buftail)); + pagefault_enable(); + + if (err) + return NULL; + + perf_callchain_store(entry, buftail.lr); + + /* + * Frame pointers should strictly progress back up the stack + * (towards higher addresses). + */ + if (tail >= buftail.fp) + return NULL; + + return buftail.fp; +} + +#ifdef CONFIG_COMPAT +/* + * The registers we're interested in are at the end of the variable + * length saved register structure. The fp points at the end of this + * structure so the address of this struct is: + * (struct compat_frame_tail *)(xxx->fp)-1 + * + * This code has been adapted from the ARM OProfile support. + */ +struct compat_frame_tail { + compat_uptr_t fp; /* a (struct compat_frame_tail *) in compat mode */ + u32 sp; + u32 lr; +} __attribute__((packed)); + +static struct compat_frame_tail __user * +compat_user_backtrace(struct compat_frame_tail __user *tail, + struct perf_callchain_entry *entry) +{ + struct compat_frame_tail buftail; + unsigned long err; + + /* Also check accessibility of one struct frame_tail beyond */ + if (!access_ok(VERIFY_READ, tail, sizeof(buftail))) + return NULL; + + pagefault_disable(); + err = __copy_from_user_inatomic(&buftail, tail, sizeof(buftail)); + pagefault_enable(); + + if (err) + return NULL; + + perf_callchain_store(entry, buftail.lr); + + /* + * Frame pointers should strictly progress back up the stack + * (towards higher addresses). + */ + if (tail + 1 >= (struct compat_frame_tail __user *) + compat_ptr(buftail.fp)) + return NULL; + + return (struct compat_frame_tail __user *)compat_ptr(buftail.fp) - 1; +} +#endif /* CONFIG_COMPAT */ + +void perf_callchain_user(struct perf_callchain_entry *entry, + struct pt_regs *regs) +{ + if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + /* We don't support guest os callchain now */ + return; + } + + perf_callchain_store(entry, regs->pc); + + if (!compat_user_mode(regs)) { + /* AARCH64 mode */ + struct frame_tail __user *tail; + + tail = (struct frame_tail __user *)regs->regs[29]; + + while (entry->nr < PERF_MAX_STACK_DEPTH && + tail && !((unsigned long)tail & 0xf)) + tail = user_backtrace(tail, entry); + } else { +#ifdef CONFIG_COMPAT + /* AARCH32 compat mode */ + struct compat_frame_tail __user *tail; + + tail = (struct compat_frame_tail __user *)regs->compat_fp - 1; + + while ((entry->nr < PERF_MAX_STACK_DEPTH) && + tail && !((unsigned long)tail & 0x3)) + tail = compat_user_backtrace(tail, entry); +#endif + } +} + +/* + * Gets called by walk_stackframe() for every stackframe. This will be called + * whist unwinding the stackframe and is like a subroutine return so we use + * the PC. + */ +static int callchain_trace(struct stackframe *frame, void *data) +{ + struct perf_callchain_entry *entry = data; + perf_callchain_store(entry, frame->pc); + return 0; +} + +void perf_callchain_kernel(struct perf_callchain_entry *entry, + struct pt_regs *regs) +{ + struct stackframe frame; + + if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + /* We don't support guest os callchain now */ + return; + } + + frame.fp = regs->regs[29]; + frame.sp = regs->sp; + frame.pc = regs->pc; + + walk_stackframe(&frame, callchain_trace, entry); +} + +unsigned long perf_instruction_pointer(struct pt_regs *regs) +{ + if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) + return perf_guest_cbs->get_guest_ip(); + + return instruction_pointer(regs); +} + +unsigned long perf_misc_flags(struct pt_regs *regs) +{ + int misc = 0; + + if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { + if (perf_guest_cbs->is_user_mode()) + misc |= PERF_RECORD_MISC_GUEST_USER; + else + misc |= PERF_RECORD_MISC_GUEST_KERNEL; + } else { + if (user_mode(regs)) + misc |= PERF_RECORD_MISC_USER; + else + misc |= PERF_RECORD_MISC_KERNEL; + } + + return misc; +} diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c index b31e9a4b62754b..f9a74d4fff3b5e 100644 --- a/arch/arm64/kernel/perf_event.c +++ b/arch/arm64/kernel/perf_event.c @@ -25,7 +25,7 @@ #include #include #include -#include +#include #include #include #include @@ -36,7 +36,6 @@ #include #include #include -#include /* * ARMv8 supports a maximum of 32 events. @@ -78,6 +77,16 @@ EXPORT_SYMBOL_GPL(perf_num_counters); #define CACHE_OP_UNSUPPORTED 0xFFFF +#define PERF_MAP_ALL_UNSUPPORTED \ + [0 ... PERF_COUNT_HW_MAX - 1] = HW_OP_UNSUPPORTED + +#define PERF_CACHE_MAP_ALL_UNSUPPORTED \ +[0 ... C(MAX) - 1] = { \ + [0 ... C(OP_MAX) - 1] = { \ + [0 ... C(RESULT_MAX) - 1] = CACHE_OP_UNSUPPORTED, \ + }, \ +} + static int armpmu_map_cache_event(const unsigned (*cache_map) [PERF_COUNT_HW_CACHE_MAX] @@ -435,10 +444,8 @@ armpmu_reserve_hardware(struct arm_pmu *armpmu) unsigned int i, irqs; struct platform_device *pmu_device = armpmu->plat_device; - if (!pmu_device) { - pr_err("no PMU device registered\n"); + if (!pmu_device) return -ENODEV; - } irqs = min(pmu_device->num_resources, num_possible_cpus()); if (!irqs) { @@ -703,118 +710,28 @@ enum armv8_pmuv3_perf_types { /* PMUv3 HW events mapping. */ static const unsigned armv8_pmuv3_perf_map[PERF_COUNT_HW_MAX] = { + PERF_MAP_ALL_UNSUPPORTED, [PERF_COUNT_HW_CPU_CYCLES] = ARMV8_PMUV3_PERFCTR_CLOCK_CYCLES, [PERF_COUNT_HW_INSTRUCTIONS] = ARMV8_PMUV3_PERFCTR_INSTR_EXECUTED, [PERF_COUNT_HW_CACHE_REFERENCES] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_ACCESS, [PERF_COUNT_HW_CACHE_MISSES] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_REFILL, - [PERF_COUNT_HW_BRANCH_INSTRUCTIONS] = HW_OP_UNSUPPORTED, [PERF_COUNT_HW_BRANCH_MISSES] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_MIS_PRED, - [PERF_COUNT_HW_BUS_CYCLES] = HW_OP_UNSUPPORTED, - [PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] = HW_OP_UNSUPPORTED, - [PERF_COUNT_HW_STALLED_CYCLES_BACKEND] = HW_OP_UNSUPPORTED, }; static const unsigned armv8_pmuv3_perf_cache_map[PERF_COUNT_HW_CACHE_MAX] [PERF_COUNT_HW_CACHE_OP_MAX] [PERF_COUNT_HW_CACHE_RESULT_MAX] = { - [C(L1D)] = { - [C(OP_READ)] = { - [C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_ACCESS, - [C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_REFILL, - }, - [C(OP_WRITE)] = { - [C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_ACCESS, - [C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_REFILL, - }, - [C(OP_PREFETCH)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - }, - [C(L1I)] = { - [C(OP_READ)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_WRITE)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_PREFETCH)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - }, - [C(LL)] = { - [C(OP_READ)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_WRITE)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_PREFETCH)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - }, - [C(DTLB)] = { - [C(OP_READ)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_WRITE)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_PREFETCH)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - }, - [C(ITLB)] = { - [C(OP_READ)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_WRITE)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_PREFETCH)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - }, - [C(BPU)] = { - [C(OP_READ)] = { - [C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_PRED, - [C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_MIS_PRED, - }, - [C(OP_WRITE)] = { - [C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_PRED, - [C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_MIS_PRED, - }, - [C(OP_PREFETCH)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - }, - [C(NODE)] = { - [C(OP_READ)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_WRITE)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - [C(OP_PREFETCH)] = { - [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED, - [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED, - }, - }, + PERF_CACHE_MAP_ALL_UNSUPPORTED, + + [C(L1D)][C(OP_READ)][C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_ACCESS, + [C(L1D)][C(OP_READ)][C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_REFILL, + [C(L1D)][C(OP_WRITE)][C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_ACCESS, + [C(L1D)][C(OP_WRITE)][C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_REFILL, + + [C(BPU)][C(OP_READ)][C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_PRED, + [C(BPU)][C(OP_READ)][C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_MIS_PRED, + [C(BPU)][C(OP_WRITE)][C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_PRED, + [C(BPU)][C(OP_WRITE)][C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_MIS_PRED, }; /* @@ -1337,7 +1254,7 @@ static int armpmu_device_probe(struct platform_device *pdev) } for_each_possible_cpu(cpu) - if (arch_find_n_match_cpu_physical_id(dn, cpu, NULL)) + if (dn == of_cpu_device_node_get(cpu)) break; if (cpu >= nr_cpu_ids) { @@ -1415,180 +1332,3 @@ static int __init init_hw_perf_events(void) } early_initcall(init_hw_perf_events); -/* - * Callchain handling code. - */ -struct frame_tail { - struct frame_tail __user *fp; - unsigned long lr; -} __attribute__((packed)); - -/* - * Get the return address for a single stackframe and return a pointer to the - * next frame tail. - */ -static struct frame_tail __user * -user_backtrace(struct frame_tail __user *tail, - struct perf_callchain_entry *entry) -{ - struct frame_tail buftail; - unsigned long err; - - /* Also check accessibility of one struct frame_tail beyond */ - if (!access_ok(VERIFY_READ, tail, sizeof(buftail))) - return NULL; - - pagefault_disable(); - err = __copy_from_user_inatomic(&buftail, tail, sizeof(buftail)); - pagefault_enable(); - - if (err) - return NULL; - - perf_callchain_store(entry, buftail.lr); - - /* - * Frame pointers should strictly progress back up the stack - * (towards higher addresses). - */ - if (tail >= buftail.fp) - return NULL; - - return buftail.fp; -} - -#ifdef CONFIG_COMPAT -/* - * The registers we're interested in are at the end of the variable - * length saved register structure. The fp points at the end of this - * structure so the address of this struct is: - * (struct compat_frame_tail *)(xxx->fp)-1 - * - * This code has been adapted from the ARM OProfile support. - */ -struct compat_frame_tail { - compat_uptr_t fp; /* a (struct compat_frame_tail *) in compat mode */ - u32 sp; - u32 lr; -} __attribute__((packed)); - -static struct compat_frame_tail __user * -compat_user_backtrace(struct compat_frame_tail __user *tail, - struct perf_callchain_entry *entry) -{ - struct compat_frame_tail buftail; - unsigned long err; - - /* Also check accessibility of one struct frame_tail beyond */ - if (!access_ok(VERIFY_READ, tail, sizeof(buftail))) - return NULL; - - pagefault_disable(); - err = __copy_from_user_inatomic(&buftail, tail, sizeof(buftail)); - pagefault_enable(); - - if (err) - return NULL; - - perf_callchain_store(entry, buftail.lr); - - /* - * Frame pointers should strictly progress back up the stack - * (towards higher addresses). - */ - if (tail + 1 >= (struct compat_frame_tail __user *) - compat_ptr(buftail.fp)) - return NULL; - - return (struct compat_frame_tail __user *)compat_ptr(buftail.fp) - 1; -} -#endif /* CONFIG_COMPAT */ - -void perf_callchain_user(struct perf_callchain_entry *entry, - struct pt_regs *regs) -{ - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { - /* We don't support guest os callchain now */ - return; - } - - perf_callchain_store(entry, regs->pc); - - if (!compat_user_mode(regs)) { - /* AARCH64 mode */ - struct frame_tail __user *tail; - - tail = (struct frame_tail __user *)regs->regs[29]; - - while (entry->nr < PERF_MAX_STACK_DEPTH && - tail && !((unsigned long)tail & 0xf)) - tail = user_backtrace(tail, entry); - } else { -#ifdef CONFIG_COMPAT - /* AARCH32 compat mode */ - struct compat_frame_tail __user *tail; - - tail = (struct compat_frame_tail __user *)regs->compat_fp - 1; - - while ((entry->nr < PERF_MAX_STACK_DEPTH) && - tail && !((unsigned long)tail & 0x3)) - tail = compat_user_backtrace(tail, entry); -#endif - } -} - -/* - * Gets called by walk_stackframe() for every stackframe. This will be called - * whist unwinding the stackframe and is like a subroutine return so we use - * the PC. - */ -static int callchain_trace(struct stackframe *frame, void *data) -{ - struct perf_callchain_entry *entry = data; - perf_callchain_store(entry, frame->pc); - return 0; -} - -void perf_callchain_kernel(struct perf_callchain_entry *entry, - struct pt_regs *regs) -{ - struct stackframe frame; - - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { - /* We don't support guest os callchain now */ - return; - } - - frame.fp = regs->regs[29]; - frame.sp = regs->sp; - frame.pc = regs->pc; - - walk_stackframe(&frame, callchain_trace, entry); -} - -unsigned long perf_instruction_pointer(struct pt_regs *regs) -{ - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) - return perf_guest_cbs->get_guest_ip(); - - return instruction_pointer(regs); -} - -unsigned long perf_misc_flags(struct pt_regs *regs) -{ - int misc = 0; - - if (perf_guest_cbs && perf_guest_cbs->is_in_guest()) { - if (perf_guest_cbs->is_user_mode()) - misc |= PERF_RECORD_MISC_GUEST_USER; - else - misc |= PERF_RECORD_MISC_GUEST_KERNEL; - } else { - if (user_mode(regs)) - misc |= PERF_RECORD_MISC_USER; - else - misc |= PERF_RECORD_MISC_KERNEL; - } - - return misc; -} diff --git a/arch/arm64/kernel/psci.c b/arch/arm64/kernel/psci.c index 869f202748e8ea..aa94a88f627996 100644 --- a/arch/arm64/kernel/psci.c +++ b/arch/arm64/kernel/psci.c @@ -18,23 +18,17 @@ #include #include #include -#include -#include #include +#include #include + #include #include -#include #include #include -#include #include #include -#include - -#define PSCI_POWER_STATE_TYPE_STANDBY 0 -#define PSCI_POWER_STATE_TYPE_POWER_DOWN 1 static bool psci_power_state_loses_context(u32 state) { @@ -50,122 +44,8 @@ static bool psci_power_state_is_valid(u32 state) return !(state & ~valid_mask); } -/* - * The CPU any Trusted OS is resident on. The trusted OS may reject CPU_OFF - * calls to its resident CPU, so we must avoid issuing those. We never migrate - * a Trusted OS even if it claims to be capable of migration -- doing so will - * require cooperation with a Trusted OS driver. - */ -static int resident_cpu = -1; - -struct psci_operations { - int (*cpu_suspend)(u32 state, unsigned long entry_point); - int (*cpu_off)(u32 state); - int (*cpu_on)(unsigned long cpuid, unsigned long entry_point); - int (*migrate)(unsigned long cpuid); - int (*affinity_info)(unsigned long target_affinity, - unsigned long lowest_affinity_level); - int (*migrate_info_type)(void); -}; - -static struct psci_operations psci_ops; - -typedef unsigned long (psci_fn)(unsigned long, unsigned long, - unsigned long, unsigned long); -asmlinkage psci_fn __invoke_psci_fn_hvc; -asmlinkage psci_fn __invoke_psci_fn_smc; -static psci_fn *invoke_psci_fn; - -enum psci_function { - PSCI_FN_CPU_SUSPEND, - PSCI_FN_CPU_ON, - PSCI_FN_CPU_OFF, - PSCI_FN_MIGRATE, - PSCI_FN_MAX, -}; - static DEFINE_PER_CPU_READ_MOSTLY(u32 *, psci_power_state); -static u32 psci_function_id[PSCI_FN_MAX]; - -static int psci_to_linux_errno(int errno) -{ - switch (errno) { - case PSCI_RET_SUCCESS: - return 0; - case PSCI_RET_NOT_SUPPORTED: - return -EOPNOTSUPP; - case PSCI_RET_INVALID_PARAMS: - return -EINVAL; - case PSCI_RET_DENIED: - return -EPERM; - }; - - return -EINVAL; -} - -static u32 psci_get_version(void) -{ - return invoke_psci_fn(PSCI_0_2_FN_PSCI_VERSION, 0, 0, 0); -} - -static int psci_cpu_suspend(u32 state, unsigned long entry_point) -{ - int err; - u32 fn; - - fn = psci_function_id[PSCI_FN_CPU_SUSPEND]; - err = invoke_psci_fn(fn, state, entry_point, 0); - return psci_to_linux_errno(err); -} - -static int psci_cpu_off(u32 state) -{ - int err; - u32 fn; - - fn = psci_function_id[PSCI_FN_CPU_OFF]; - err = invoke_psci_fn(fn, state, 0, 0); - return psci_to_linux_errno(err); -} - -static int psci_cpu_on(unsigned long cpuid, unsigned long entry_point) -{ - int err; - u32 fn; - - fn = psci_function_id[PSCI_FN_CPU_ON]; - err = invoke_psci_fn(fn, cpuid, entry_point, 0); - return psci_to_linux_errno(err); -} - -static int psci_migrate(unsigned long cpuid) -{ - int err; - u32 fn; - - fn = psci_function_id[PSCI_FN_MIGRATE]; - err = invoke_psci_fn(fn, cpuid, 0, 0); - return psci_to_linux_errno(err); -} - -static int psci_affinity_info(unsigned long target_affinity, - unsigned long lowest_affinity_level) -{ - return invoke_psci_fn(PSCI_0_2_FN64_AFFINITY_INFO, target_affinity, - lowest_affinity_level, 0); -} - -static int psci_migrate_info_type(void) -{ - return invoke_psci_fn(PSCI_0_2_FN_MIGRATE_INFO_TYPE, 0, 0, 0); -} - -static unsigned long psci_migrate_info_up_cpu(void) -{ - return invoke_psci_fn(PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU, 0, 0, 0); -} - static int __maybe_unused cpu_psci_cpu_init_idle(unsigned int cpu) { int i, ret, count = 0; @@ -230,240 +110,6 @@ static int __maybe_unused cpu_psci_cpu_init_idle(unsigned int cpu) return ret; } -static int get_set_conduit_method(struct device_node *np) -{ - const char *method; - - pr_info("probing for conduit method from DT.\n"); - - if (of_property_read_string(np, "method", &method)) { - pr_warn("missing \"method\" property\n"); - return -ENXIO; - } - - if (!strcmp("hvc", method)) { - invoke_psci_fn = __invoke_psci_fn_hvc; - } else if (!strcmp("smc", method)) { - invoke_psci_fn = __invoke_psci_fn_smc; - } else { - pr_warn("invalid \"method\" property: %s\n", method); - return -EINVAL; - } - return 0; -} - -static void psci_sys_reset(enum reboot_mode reboot_mode, const char *cmd) -{ - invoke_psci_fn(PSCI_0_2_FN_SYSTEM_RESET, 0, 0, 0); -} - -static void psci_sys_poweroff(void) -{ - invoke_psci_fn(PSCI_0_2_FN_SYSTEM_OFF, 0, 0, 0); -} - -/* - * Detect the presence of a resident Trusted OS which may cause CPU_OFF to - * return DENIED (which would be fatal). - */ -static void __init psci_init_migrate(void) -{ - unsigned long cpuid; - int type, cpu; - - type = psci_ops.migrate_info_type(); - - if (type == PSCI_0_2_TOS_MP) { - pr_info("Trusted OS migration not required\n"); - return; - } - - if (type == PSCI_RET_NOT_SUPPORTED) { - pr_info("MIGRATE_INFO_TYPE not supported.\n"); - return; - } - - if (type != PSCI_0_2_TOS_UP_MIGRATE && - type != PSCI_0_2_TOS_UP_NO_MIGRATE) { - pr_err("MIGRATE_INFO_TYPE returned unknown type (%d)\n", type); - return; - } - - cpuid = psci_migrate_info_up_cpu(); - if (cpuid & ~MPIDR_HWID_BITMASK) { - pr_warn("MIGRATE_INFO_UP_CPU reported invalid physical ID (0x%lx)\n", - cpuid); - return; - } - - cpu = get_logical_index(cpuid); - resident_cpu = cpu >= 0 ? cpu : -1; - - pr_info("Trusted OS resident on physical CPU 0x%lx\n", cpuid); -} - -static void __init psci_0_2_set_functions(void) -{ - pr_info("Using standard PSCI v0.2 function IDs\n"); - psci_function_id[PSCI_FN_CPU_SUSPEND] = PSCI_0_2_FN64_CPU_SUSPEND; - psci_ops.cpu_suspend = psci_cpu_suspend; - - psci_function_id[PSCI_FN_CPU_OFF] = PSCI_0_2_FN_CPU_OFF; - psci_ops.cpu_off = psci_cpu_off; - - psci_function_id[PSCI_FN_CPU_ON] = PSCI_0_2_FN64_CPU_ON; - psci_ops.cpu_on = psci_cpu_on; - - psci_function_id[PSCI_FN_MIGRATE] = PSCI_0_2_FN64_MIGRATE; - psci_ops.migrate = psci_migrate; - - psci_ops.affinity_info = psci_affinity_info; - - psci_ops.migrate_info_type = psci_migrate_info_type; - - arm_pm_restart = psci_sys_reset; - - pm_power_off = psci_sys_poweroff; -} - -/* - * Probe function for PSCI firmware versions >= 0.2 - */ -static int __init psci_probe(void) -{ - u32 ver = psci_get_version(); - - pr_info("PSCIv%d.%d detected in firmware.\n", - PSCI_VERSION_MAJOR(ver), - PSCI_VERSION_MINOR(ver)); - - if (PSCI_VERSION_MAJOR(ver) == 0 && PSCI_VERSION_MINOR(ver) < 2) { - pr_err("Conflicting PSCI version detected.\n"); - return -EINVAL; - } - - psci_0_2_set_functions(); - - psci_init_migrate(); - - return 0; -} - -typedef int (*psci_initcall_t)(const struct device_node *); - -/* - * PSCI init function for PSCI versions >=0.2 - * - * Probe based on PSCI PSCI_VERSION function - */ -static int __init psci_0_2_init(struct device_node *np) -{ - int err; - - err = get_set_conduit_method(np); - - if (err) - goto out_put_node; - /* - * Starting with v0.2, the PSCI specification introduced a call - * (PSCI_VERSION) that allows probing the firmware version, so - * that PSCI function IDs and version specific initialization - * can be carried out according to the specific version reported - * by firmware - */ - err = psci_probe(); - -out_put_node: - of_node_put(np); - return err; -} - -/* - * PSCI < v0.2 get PSCI Function IDs via DT. - */ -static int __init psci_0_1_init(struct device_node *np) -{ - u32 id; - int err; - - err = get_set_conduit_method(np); - - if (err) - goto out_put_node; - - pr_info("Using PSCI v0.1 Function IDs from DT\n"); - - if (!of_property_read_u32(np, "cpu_suspend", &id)) { - psci_function_id[PSCI_FN_CPU_SUSPEND] = id; - psci_ops.cpu_suspend = psci_cpu_suspend; - } - - if (!of_property_read_u32(np, "cpu_off", &id)) { - psci_function_id[PSCI_FN_CPU_OFF] = id; - psci_ops.cpu_off = psci_cpu_off; - } - - if (!of_property_read_u32(np, "cpu_on", &id)) { - psci_function_id[PSCI_FN_CPU_ON] = id; - psci_ops.cpu_on = psci_cpu_on; - } - - if (!of_property_read_u32(np, "migrate", &id)) { - psci_function_id[PSCI_FN_MIGRATE] = id; - psci_ops.migrate = psci_migrate; - } - -out_put_node: - of_node_put(np); - return err; -} - -static const struct of_device_id psci_of_match[] __initconst = { - { .compatible = "arm,psci", .data = psci_0_1_init}, - { .compatible = "arm,psci-0.2", .data = psci_0_2_init}, - {}, -}; - -int __init psci_dt_init(void) -{ - struct device_node *np; - const struct of_device_id *matched_np; - psci_initcall_t init_fn; - - np = of_find_matching_node_and_match(NULL, psci_of_match, &matched_np); - - if (!np) - return -ENODEV; - - init_fn = (psci_initcall_t)matched_np->data; - return init_fn(np); -} - -#ifdef CONFIG_ACPI -/* - * We use PSCI 0.2+ when ACPI is deployed on ARM64 and it's - * explicitly clarified in SBBR - */ -int __init psci_acpi_init(void) -{ - if (!acpi_psci_present()) { - pr_info("is not implemented in ACPI.\n"); - return -EOPNOTSUPP; - } - - pr_info("probing for conduit method from ACPI.\n"); - - if (acpi_psci_use_hvc()) - invoke_psci_fn = __invoke_psci_fn_hvc; - else - invoke_psci_fn = __invoke_psci_fn_smc; - - return psci_probe(); -} -#endif - -#ifdef CONFIG_SMP - static int __init cpu_psci_cpu_init(unsigned int cpu) { return 0; @@ -489,11 +135,6 @@ static int cpu_psci_cpu_boot(unsigned int cpu) } #ifdef CONFIG_HOTPLUG_CPU -static bool psci_tos_resident_on(int cpu) -{ - return cpu == resident_cpu; -} - static int cpu_psci_cpu_disable(unsigned int cpu) { /* Fail early if we don't have CPU_OFF support */ @@ -550,7 +191,6 @@ static int cpu_psci_cpu_kill(unsigned int cpu) return -ETIMEDOUT; } #endif -#endif static int psci_suspend_finisher(unsigned long index) { @@ -585,7 +225,6 @@ const struct cpu_operations cpu_psci_ops = { .cpu_init_idle = cpu_psci_cpu_init_idle, .cpu_suspend = cpu_psci_cpu_suspend, #endif -#ifdef CONFIG_SMP .cpu_init = cpu_psci_cpu_init, .cpu_prepare = cpu_psci_cpu_prepare, .cpu_boot = cpu_psci_cpu_boot, @@ -594,6 +233,5 @@ const struct cpu_operations cpu_psci_ops = { .cpu_die = cpu_psci_cpu_die, .cpu_kill = cpu_psci_cpu_kill, #endif -#endif }; diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index d882b833dbdb59..1971f491bb90c2 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -826,6 +826,30 @@ static int compat_vfp_set(struct task_struct *target, return ret; } +static int compat_tls_get(struct task_struct *target, + const struct user_regset *regset, unsigned int pos, + unsigned int count, void *kbuf, void __user *ubuf) +{ + compat_ulong_t tls = (compat_ulong_t)target->thread.tp_value; + return user_regset_copyout(&pos, &count, &kbuf, &ubuf, &tls, 0, -1); +} + +static int compat_tls_set(struct task_struct *target, + const struct user_regset *regset, unsigned int pos, + unsigned int count, const void *kbuf, + const void __user *ubuf) +{ + int ret; + compat_ulong_t tls; + + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &tls, 0, -1); + if (ret) + return ret; + + target->thread.tp_value = tls; + return ret; +} + static const struct user_regset aarch32_regsets[] = { [REGSET_COMPAT_GPR] = { .core_note_type = NT_PRSTATUS, @@ -850,6 +874,64 @@ static const struct user_regset_view user_aarch32_view = { .regsets = aarch32_regsets, .n = ARRAY_SIZE(aarch32_regsets) }; +static const struct user_regset aarch32_ptrace_regsets[] = { + [REGSET_GPR] = { + .core_note_type = NT_PRSTATUS, + .n = COMPAT_ELF_NGREG, + .size = sizeof(compat_elf_greg_t), + .align = sizeof(compat_elf_greg_t), + .get = compat_gpr_get, + .set = compat_gpr_set + }, + [REGSET_FPR] = { + .core_note_type = NT_ARM_VFP, + .n = VFP_STATE_SIZE / sizeof(compat_ulong_t), + .size = sizeof(compat_ulong_t), + .align = sizeof(compat_ulong_t), + .get = compat_vfp_get, + .set = compat_vfp_set + }, + [REGSET_TLS] = { + .core_note_type = NT_ARM_TLS, + .n = 1, + .size = sizeof(compat_ulong_t), + .align = sizeof(compat_ulong_t), + .get = compat_tls_get, + .set = compat_tls_set, + }, +#ifdef CONFIG_HAVE_HW_BREAKPOINT + [REGSET_HW_BREAK] = { + .core_note_type = NT_ARM_HW_BREAK, + .n = sizeof(struct user_hwdebug_state) / sizeof(u32), + .size = sizeof(u32), + .align = sizeof(u32), + .get = hw_break_get, + .set = hw_break_set, + }, + [REGSET_HW_WATCH] = { + .core_note_type = NT_ARM_HW_WATCH, + .n = sizeof(struct user_hwdebug_state) / sizeof(u32), + .size = sizeof(u32), + .align = sizeof(u32), + .get = hw_break_get, + .set = hw_break_set, + }, +#endif + [REGSET_SYSTEM_CALL] = { + .core_note_type = NT_ARM_SYSTEM_CALL, + .n = 1, + .size = sizeof(int), + .align = sizeof(int), + .get = system_call_get, + .set = system_call_set, + }, +}; + +static const struct user_regset_view user_aarch32_ptrace_view = { + .name = "aarch32", .e_machine = EM_ARM, + .regsets = aarch32_ptrace_regsets, .n = ARRAY_SIZE(aarch32_ptrace_regsets) +}; + static int compat_ptrace_read_user(struct task_struct *tsk, compat_ulong_t off, compat_ulong_t __user *ret) { @@ -1109,8 +1191,16 @@ long compat_arch_ptrace(struct task_struct *child, compat_long_t request, const struct user_regset_view *task_user_regset_view(struct task_struct *task) { #ifdef CONFIG_COMPAT - if (is_compat_thread(task_thread_info(task))) + /* + * Core dumping of 32-bit tasks or compat ptrace requests must use the + * user_aarch32_view compatible with arm32. Native ptrace requests on + * 32-bit children use an extended user_aarch32_ptrace_view to allow + * access to the TLS register. + */ + if (is_compat_task()) return &user_aarch32_view; + else if (is_compat_thread(task_thread_info(task))) + return &user_aarch32_ptrace_view; #endif return &user_aarch64_view; } diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index 926ae8d9abc5f7..8884788812433b 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -45,6 +45,7 @@ #include #include #include +#include #include #include @@ -60,9 +61,7 @@ #include #include #include -#include #include -#include #include unsigned long elf_hwcap __read_mostly; @@ -130,7 +129,6 @@ bool arch_match_cpu_phys_id(int cpu, u64 phys_id) } struct mpidr_hash mpidr_hash; -#ifdef CONFIG_SMP /** * smp_build_mpidr_hash - Pre-compute shifts required at each affinity * level in order to build a linear index from an @@ -196,35 +194,11 @@ static void __init smp_build_mpidr_hash(void) pr_warn("Large number of MPIDR hash buckets detected\n"); __flush_dcache_area(&mpidr_hash, sizeof(struct mpidr_hash)); } -#endif - -static void __init hyp_mode_check(void) -{ - if (is_hyp_mode_available()) - pr_info("CPU: All CPU(s) started at EL2\n"); - else if (is_hyp_mode_mismatched()) - WARN_TAINT(1, TAINT_CPU_OUT_OF_SPEC, - "CPU: CPUs started in inconsistent modes"); - else - pr_info("CPU: All CPU(s) started at EL1\n"); -} - -void __init do_post_cpus_up_work(void) -{ - hyp_mode_check(); - apply_alternatives_all(); -} - -#ifdef CONFIG_UP_LATE_INIT -void __init up_late_init(void) -{ - do_post_cpus_up_work(); -} -#endif /* CONFIG_UP_LATE_INIT */ static void __init setup_processor(void) { - u64 features, block; + u64 features; + s64 block; u32 cwg; int cls; @@ -254,8 +228,8 @@ static void __init setup_processor(void) * for non-negative values. Negative values are reserved. */ features = read_cpuid(ID_AA64ISAR0_EL1); - block = (features >> 4) & 0xf; - if (!(block & 0x8)) { + block = cpuid_feature_extract_field(features, 4); + if (block > 0) { switch (block) { default: case 2: @@ -267,26 +241,36 @@ static void __init setup_processor(void) } } - block = (features >> 8) & 0xf; - if (block && !(block & 0x8)) + if (cpuid_feature_extract_field(features, 8) > 0) elf_hwcap |= HWCAP_SHA1; - block = (features >> 12) & 0xf; - if (block && !(block & 0x8)) + if (cpuid_feature_extract_field(features, 12) > 0) elf_hwcap |= HWCAP_SHA2; - block = (features >> 16) & 0xf; - if (block && !(block & 0x8)) + if (cpuid_feature_extract_field(features, 16) > 0) elf_hwcap |= HWCAP_CRC32; + block = cpuid_feature_extract_field(features, 20); + if (block > 0) { + switch (block) { + default: + case 2: + elf_hwcap |= HWCAP_ATOMICS; + case 1: + /* RESERVED */ + case 0: + break; + } + } + #ifdef CONFIG_COMPAT /* * ID_ISAR5_EL1 carries similar information as above, but pertaining to - * the Aarch32 32-bit execution state. + * the AArch32 32-bit execution state. */ features = read_cpuid(ID_ISAR5_EL1); - block = (features >> 4) & 0xf; - if (!(block & 0x8)) { + block = cpuid_feature_extract_field(features, 4); + if (block > 0) { switch (block) { default: case 2: @@ -298,16 +282,13 @@ static void __init setup_processor(void) } } - block = (features >> 8) & 0xf; - if (block && !(block & 0x8)) + if (cpuid_feature_extract_field(features, 8) > 0) compat_elf_hwcap2 |= COMPAT_HWCAP2_SHA1; - block = (features >> 12) & 0xf; - if (block && !(block & 0x8)) + if (cpuid_feature_extract_field(features, 12) > 0) compat_elf_hwcap2 |= COMPAT_HWCAP2_SHA2; - block = (features >> 16) & 0xf; - if (block && !(block & 0x8)) + if (cpuid_feature_extract_field(features, 16) > 0) compat_elf_hwcap2 |= COMPAT_HWCAP2_CRC32; #endif } @@ -404,10 +385,8 @@ void __init setup_arch(char **cmdline_p) xen_early_init(); cpu_read_bootcpu_ops(); -#ifdef CONFIG_SMP smp_init_cpus(); smp_build_mpidr_hash(); -#endif #ifdef CONFIG_VT #if defined(CONFIG_VGA_CONSOLE) @@ -426,8 +405,13 @@ void __init setup_arch(char **cmdline_p) static int __init arm64_device_init(void) { - of_iommu_init(); - of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL); + if (of_have_populated_dt()) { + of_iommu_init(); + of_platform_populate(NULL, of_default_bus_match_table, + NULL, NULL); + } else if (acpi_disabled) { + pr_crit("Device tree not populated\n"); + } return 0; } arch_initcall_sync(arm64_device_init); @@ -455,6 +439,7 @@ static const char *hwcap_str[] = { "sha1", "sha2", "crc32", + "atomics", NULL }; @@ -507,9 +492,7 @@ static int c_show(struct seq_file *m, void *v) * online processors, looking for lines beginning with * "processor". Give glibc what it expects. */ -#ifdef CONFIG_SMP seq_printf(m, "processor\t: %d\n", i); -#endif /* * Dump out the common processor features in a single line. diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S index 803cfea41962f4..f586f7c875e292 100644 --- a/arch/arm64/kernel/sleep.S +++ b/arch/arm64/kernel/sleep.S @@ -82,7 +82,6 @@ ENTRY(__cpu_suspend_enter) str x2, [x0, #CPU_CTX_SP] ldr x1, =sleep_save_sp ldr x1, [x1, #SLEEP_SAVE_SP_VIRT] -#ifdef CONFIG_SMP mrs x7, mpidr_el1 ldr x9, =mpidr_hash ldr x10, [x9, #MPIDR_HASH_MASK] @@ -94,7 +93,6 @@ ENTRY(__cpu_suspend_enter) ldp w5, w6, [x9, #(MPIDR_HASH_SHIFTS + 8)] compute_mpidr_hash x8, x3, x4, x5, x6, x7, x10 add x1, x1, x8, lsl #3 -#endif bl __cpu_suspend_save /* * Grab suspend finisher in x20 and its argument in x19 @@ -135,6 +133,14 @@ ENTRY(cpu_resume_mmu) ldr x3, =cpu_resume_after_mmu msr sctlr_el1, x0 // restore sctlr_el1 isb + /* + * Invalidate the local I-cache so that any instructions fetched + * speculatively from the PoC are discarded, since they may have + * been dynamically patched at the PoU. + */ + ic iallu + dsb nsh + isb br x3 // global jump to virtual address ENDPROC(cpu_resume_mmu) .popsection @@ -151,7 +157,6 @@ ENDPROC(cpu_resume_after_mmu) ENTRY(cpu_resume) bl el2_setup // if in EL2 drop to EL1 cleanly -#ifdef CONFIG_SMP mrs x1, mpidr_el1 adrp x8, mpidr_hash add x8, x8, #:lo12:mpidr_hash // x8 = struct mpidr_hash phys address @@ -161,9 +166,6 @@ ENTRY(cpu_resume) ldp w5, w6, [x8, #(MPIDR_HASH_SHIFTS + 8)] compute_mpidr_hash x7, x3, x4, x5, x6, x1, x2 /* x7 contains hash index, let's use it to grab context pointer */ -#else - mov x7, xzr -#endif ldr_l x0, sleep_save_sp + SLEEP_SAVE_SP_PHYS ldr x0, [x0, x7, lsl #3] /* load sp from context */ diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 50fb4696654ea5..dbdaacddd9a562 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -52,6 +52,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -310,10 +311,22 @@ void cpu_die(void) } #endif +static void __init hyp_mode_check(void) +{ + if (is_hyp_mode_available()) + pr_info("CPU: All CPU(s) started at EL2\n"); + else if (is_hyp_mode_mismatched()) + WARN_TAINT(1, TAINT_CPU_OUT_OF_SPEC, + "CPU: CPUs started in inconsistent modes"); + else + pr_info("CPU: All CPU(s) started at EL1\n"); +} + void __init smp_cpus_done(unsigned int max_cpus) { pr_info("SMP: Total of %d processors activated.\n", num_online_cpus()); - do_post_cpus_up_work(); + hyp_mode_check(); + apply_alternatives_all(); } void __init smp_prepare_boot_cpu(void) diff --git a/arch/arm64/kernel/time.c b/arch/arm64/kernel/time.c index 42f9195cf2f8ab..149151fb42bb15 100644 --- a/arch/arm64/kernel/time.c +++ b/arch/arm64/kernel/time.c @@ -42,7 +42,6 @@ #include #include -#ifdef CONFIG_SMP unsigned long profile_pc(struct pt_regs *regs) { struct stackframe frame; @@ -62,7 +61,6 @@ unsigned long profile_pc(struct pt_regs *regs) return frame.pc; } EXPORT_SYMBOL(profile_pc); -#endif void __init time_init(void) { diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index fcb8f7b42271c8..694f6deedbab89 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -300,6 +300,6 @@ void __init init_cpu_topology(void) * Discard anything that was parsed if we hit an error so we * don't use partial information. */ - if (parse_dt_topology()) + if (of_have_populated_dt() && parse_dt_topology()) reset_cpu_topology(); } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 566bc4c3504054..f93aae5e43075c 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -17,6 +17,7 @@ * along with this program. If not, see . */ +#include #include #include #include @@ -32,8 +33,10 @@ #include #include +#include #include #include +#include #include #include #include @@ -52,11 +55,12 @@ int show_unhandled_signals = 1; * Dump out the contents of some memory nicely... */ static void dump_mem(const char *lvl, const char *str, unsigned long bottom, - unsigned long top) + unsigned long top, bool compat) { unsigned long first; mm_segment_t fs; int i; + unsigned int width = compat ? 4 : 8; /* * We need to switch to kernel mode so that we can use __get_user @@ -75,13 +79,22 @@ static void dump_mem(const char *lvl, const char *str, unsigned long bottom, memset(str, ' ', sizeof(str)); str[sizeof(str) - 1] = '\0'; - for (p = first, i = 0; i < 8 && p < top; i++, p += 4) { + for (p = first, i = 0; i < (32 / width) + && p < top; i++, p += width) { if (p >= bottom && p < top) { - unsigned int val; - if (__get_user(val, (unsigned int *)p) == 0) - sprintf(str + i * 9, " %08x", val); - else - sprintf(str + i * 9, " ????????"); + unsigned long val; + + if (width == 8) { + if (__get_user(val, (unsigned long *)p) == 0) + sprintf(str + i * 17, " %016lx", val); + else + sprintf(str + i * 17, " ????????????????"); + } else { + if (__get_user(val, (unsigned int *)p) == 0) + sprintf(str + i * 9, " %08lx", val); + else + sprintf(str + i * 9, " ????????"); + } } } printk("%s%04lx:%s\n", lvl, first & 0xffff, str); @@ -95,7 +108,7 @@ static void dump_backtrace_entry(unsigned long where, unsigned long stack) print_ip_sym(where); if (in_exception_text(where)) dump_mem("", "Exception stack", stack, - stack + sizeof(struct pt_regs)); + stack + sizeof(struct pt_regs), false); } static void dump_instr(const char *lvl, struct pt_regs *regs) @@ -179,11 +192,7 @@ void show_stack(struct task_struct *tsk, unsigned long *sp) #else #define S_PREEMPT "" #endif -#ifdef CONFIG_SMP #define S_SMP " SMP" -#else -#define S_SMP "" -#endif static int __die(const char *str, int err, struct thread_info *thread, struct pt_regs *regs) @@ -207,7 +216,8 @@ static int __die(const char *str, int err, struct thread_info *thread, if (!user_mode(regs) || in_interrupt()) { dump_mem(KERN_EMERG, "Stack: ", regs->sp, - THREAD_SIZE + (unsigned long)task_stack_page(tsk)); + THREAD_SIZE + (unsigned long)task_stack_page(tsk), + compat_user_mode(regs)); dump_backtrace(regs, tsk); dump_instr(KERN_EMERG, regs); } @@ -459,7 +469,63 @@ void __pgd_error(const char *file, int line, unsigned long val) pr_crit("%s:%d: bad pgd %016lx.\n", file, line, val); } +/* GENERIC_BUG traps */ + +int is_valid_bugaddr(unsigned long addr) +{ + /* + * bug_handler() only called for BRK #BUG_BRK_IMM. + * So the answer is trivial -- any spurious instances with no + * bug table entry will be rejected by report_bug() and passed + * back to the debug-monitors code and handled as a fatal + * unexpected debug exception. + */ + return 1; +} + +static int bug_handler(struct pt_regs *regs, unsigned int esr) +{ + if (user_mode(regs)) + return DBG_HOOK_ERROR; + + switch (report_bug(regs->pc, regs)) { + case BUG_TRAP_TYPE_BUG: + die("Oops - BUG", regs, 0); + break; + + case BUG_TRAP_TYPE_WARN: + /* Ideally, report_bug() should backtrace for us... but no. */ + dump_backtrace(regs, NULL); + break; + + default: + /* unknown/unrecognised bug trap type */ + return DBG_HOOK_ERROR; + } + + /* If thread survives, skip over the BUG instruction and continue: */ + regs->pc += AARCH64_INSN_SIZE; /* skip BRK and resume */ + return DBG_HOOK_HANDLED; +} + +static struct break_hook bug_break_hook = { + .esr_val = 0xf2000000 | BUG_BRK_IMM, + .esr_mask = 0xffffffff, + .fn = bug_handler, +}; + +/* + * Initial handler for AArch64 BRK exceptions + * This handler only used until debug_traps_init(). + */ +int __init early_brk64(unsigned long addr, unsigned int esr, + struct pt_regs *regs) +{ + return bug_handler(regs, esr) != DBG_HOOK_HANDLED; +} + +/* This registration must happen early, before debug_traps_init(). */ void __init trap_init(void) { - return; + register_break_hook(&bug_break_hook); } diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S index 17a8fb14f428d4..10915aaf0b015b 100644 --- a/arch/arm64/kvm/hyp.S +++ b/arch/arm64/kvm/hyp.S @@ -810,7 +810,11 @@ * Call into the vgic backend for state saving */ .macro save_vgic_state - alternative_insn "bl __save_vgic_v2_state", "bl __save_vgic_v3_state", ARM64_HAS_SYSREG_GIC_CPUIF +alternative_if_not ARM64_HAS_SYSREG_GIC_CPUIF + bl __save_vgic_v2_state +alternative_else + bl __save_vgic_v3_state +alternative_endif mrs x24, hcr_el2 mov x25, #HCR_INT_OVERRIDE neg x25, x25 @@ -827,7 +831,11 @@ orr x24, x24, #HCR_INT_OVERRIDE orr x24, x24, x25 msr hcr_el2, x24 - alternative_insn "bl __restore_vgic_v2_state", "bl __restore_vgic_v3_state", ARM64_HAS_SYSREG_GIC_CPUIF +alternative_if_not ARM64_HAS_SYSREG_GIC_CPUIF + bl __restore_vgic_v2_state +alternative_else + bl __restore_vgic_v3_state +alternative_endif .endm .macro save_timer_state diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile index d98d3e39879eb9..1a811ecf71da8a 100644 --- a/arch/arm64/lib/Makefile +++ b/arch/arm64/lib/Makefile @@ -3,3 +3,16 @@ lib-y := bitops.o clear_user.o delay.o copy_from_user.o \ clear_page.o memchr.o memcpy.o memmove.o memset.o \ memcmp.o strcmp.o strncmp.o strlen.o strnlen.o \ strchr.o strrchr.o + +# Tell the compiler to treat all general purpose registers as +# callee-saved, which allows for efficient runtime patching of the bl +# instruction in the caller with an atomic instruction when supported by +# the CPU. Result and argument registers are handled correctly, based on +# the function prototype. +lib-$(CONFIG_ARM64_LSE_ATOMICS) += atomic_ll_sc.o +CFLAGS_atomic_ll_sc.o := -fcall-used-x0 -ffixed-x1 -ffixed-x2 \ + -ffixed-x3 -ffixed-x4 -ffixed-x5 -ffixed-x6 \ + -ffixed-x7 -fcall-saved-x8 -fcall-saved-x9 \ + -fcall-saved-x10 -fcall-saved-x11 -fcall-saved-x12 \ + -fcall-saved-x13 -fcall-saved-x14 -fcall-saved-x15 \ + -fcall-saved-x16 -fcall-saved-x17 -fcall-saved-x18 diff --git a/arch/arm64/lib/atomic_ll_sc.c b/arch/arm64/lib/atomic_ll_sc.c new file mode 100644 index 00000000000000..b0c538b0da28c6 --- /dev/null +++ b/arch/arm64/lib/atomic_ll_sc.c @@ -0,0 +1,3 @@ +#include +#define __ARM64_IN_ATOMIC_IMPL +#include diff --git a/arch/arm64/lib/bitops.S b/arch/arm64/lib/bitops.S index 7dac371cc9a2f8..43ac736baa5bf1 100644 --- a/arch/arm64/lib/bitops.S +++ b/arch/arm64/lib/bitops.S @@ -18,52 +18,59 @@ #include #include +#include /* * x0: bits 5:0 bit offset * bits 31:6 word offset * x1: address */ - .macro bitop, name, instr + .macro bitop, name, llsc, lse ENTRY( \name ) and w3, w0, #63 // Get bit offset eor w0, w0, w3 // Clear low bits mov x2, #1 add x1, x1, x0, lsr #3 // Get word offset +alt_lse " prfm pstl1strm, [x1]", "nop" lsl x3, x2, x3 // Create mask -1: ldxr x2, [x1] - \instr x2, x2, x3 - stxr w0, x2, [x1] - cbnz w0, 1b + +alt_lse "1: ldxr x2, [x1]", "\lse x3, [x1]" +alt_lse " \llsc x2, x2, x3", "nop" +alt_lse " stxr w0, x2, [x1]", "nop" +alt_lse " cbnz w0, 1b", "nop" + ret ENDPROC(\name ) .endm - .macro testop, name, instr + .macro testop, name, llsc, lse ENTRY( \name ) and w3, w0, #63 // Get bit offset eor w0, w0, w3 // Clear low bits mov x2, #1 add x1, x1, x0, lsr #3 // Get word offset +alt_lse " prfm pstl1strm, [x1]", "nop" lsl x4, x2, x3 // Create mask -1: ldxr x2, [x1] - lsr x0, x2, x3 // Save old value of bit - \instr x2, x2, x4 // toggle bit - stlxr w5, x2, [x1] - cbnz w5, 1b - dmb ish + +alt_lse "1: ldxr x2, [x1]", "\lse x4, x2, [x1]" + lsr x0, x2, x3 +alt_lse " \llsc x2, x2, x4", "nop" +alt_lse " stlxr w5, x2, [x1]", "nop" +alt_lse " cbnz w5, 1b", "nop" +alt_lse " dmb ish", "nop" + and x0, x0, #1 -3: ret + ret ENDPROC(\name ) .endm /* * Atomic bit operations. */ - bitop change_bit, eor - bitop clear_bit, bic - bitop set_bit, orr + bitop change_bit, eor, steor + bitop clear_bit, bic, stclr + bitop set_bit, orr, stset - testop test_and_change_bit, eor - testop test_and_clear_bit, bic - testop test_and_set_bit, orr + testop test_and_change_bit, eor, ldeoral + testop test_and_clear_bit, bic, ldclral + testop test_and_set_bit, orr, ldsetal diff --git a/arch/arm64/lib/clear_user.S b/arch/arm64/lib/clear_user.S index c17967fdf5f600..a9723c71c52b20 100644 --- a/arch/arm64/lib/clear_user.S +++ b/arch/arm64/lib/clear_user.S @@ -16,7 +16,11 @@ * along with this program. If not, see . */ #include + +#include #include +#include +#include .text @@ -29,6 +33,8 @@ * Alignment fixed up by hardware. */ ENTRY(__clear_user) +ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) mov x2, x1 // save the size for fixup return subs x1, x1, #8 b.mi 2f @@ -48,6 +54,8 @@ USER(9f, strh wzr, [x0], #2 ) b.mi 5f USER(9f, strb wzr, [x0] ) 5: mov x0, #0 +ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) ret ENDPROC(__clear_user) diff --git a/arch/arm64/lib/copy_from_user.S b/arch/arm64/lib/copy_from_user.S index 5e27add9d3624e..1be9ef27be9704 100644 --- a/arch/arm64/lib/copy_from_user.S +++ b/arch/arm64/lib/copy_from_user.S @@ -15,7 +15,11 @@ */ #include + +#include #include +#include +#include /* * Copy from user space to a kernel buffer (alignment handled by the hardware) @@ -28,14 +32,21 @@ * x0 - bytes not copied */ ENTRY(__copy_from_user) - add x4, x1, x2 // upper user buffer boundary - subs x2, x2, #8 +ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) + add x5, x1, x2 // upper user buffer boundary + subs x2, x2, #16 + b.mi 1f +0: +USER(9f, ldp x3, x4, [x1], #16) + subs x2, x2, #16 + stp x3, x4, [x0], #16 + b.pl 0b +1: adds x2, x2, #8 b.mi 2f -1: USER(9f, ldr x3, [x1], #8 ) - subs x2, x2, #8 + sub x2, x2, #8 str x3, [x0], #8 - b.pl 1b 2: adds x2, x2, #4 b.mi 3f USER(9f, ldr w3, [x1], #4 ) @@ -51,12 +62,14 @@ USER(9f, ldrh w3, [x1], #2 ) USER(9f, ldrb w3, [x1] ) strb w3, [x0] 5: mov x0, #0 +ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) ret ENDPROC(__copy_from_user) .section .fixup,"ax" .align 2 -9: sub x2, x4, x1 +9: sub x2, x5, x1 mov x3, x2 10: strb wzr, [x0], #1 // zero remaining buffer space subs x3, x3, #1 diff --git a/arch/arm64/lib/copy_in_user.S b/arch/arm64/lib/copy_in_user.S index 84b6c9bb9b936b..1b94661e22b3f4 100644 --- a/arch/arm64/lib/copy_in_user.S +++ b/arch/arm64/lib/copy_in_user.S @@ -17,7 +17,11 @@ */ #include + +#include #include +#include +#include /* * Copy from user space to user space (alignment handled by the hardware) @@ -30,14 +34,21 @@ * x0 - bytes not copied */ ENTRY(__copy_in_user) - add x4, x0, x2 // upper user buffer boundary - subs x2, x2, #8 +ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) + add x5, x0, x2 // upper user buffer boundary + subs x2, x2, #16 + b.mi 1f +0: +USER(9f, ldp x3, x4, [x1], #16) + subs x2, x2, #16 +USER(9f, stp x3, x4, [x0], #16) + b.pl 0b +1: adds x2, x2, #8 b.mi 2f -1: USER(9f, ldr x3, [x1], #8 ) - subs x2, x2, #8 + sub x2, x2, #8 USER(9f, str x3, [x0], #8 ) - b.pl 1b 2: adds x2, x2, #4 b.mi 3f USER(9f, ldr w3, [x1], #4 ) @@ -53,11 +64,13 @@ USER(9f, strh w3, [x0], #2 ) USER(9f, ldrb w3, [x1] ) USER(9f, strb w3, [x0] ) 5: mov x0, #0 +ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) ret ENDPROC(__copy_in_user) .section .fixup,"ax" .align 2 -9: sub x0, x4, x0 // bytes not copied +9: sub x0, x5, x0 // bytes not copied ret .previous diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S index a0aeeb9b7a284a..a257b47e2dc493 100644 --- a/arch/arm64/lib/copy_to_user.S +++ b/arch/arm64/lib/copy_to_user.S @@ -15,7 +15,11 @@ */ #include + +#include #include +#include +#include /* * Copy to user space from a kernel buffer (alignment handled by the hardware) @@ -28,14 +32,21 @@ * x0 - bytes not copied */ ENTRY(__copy_to_user) - add x4, x0, x2 // upper user buffer boundary - subs x2, x2, #8 +ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) + add x5, x0, x2 // upper user buffer boundary + subs x2, x2, #16 + b.mi 1f +0: + ldp x3, x4, [x1], #16 + subs x2, x2, #16 +USER(9f, stp x3, x4, [x0], #16) + b.pl 0b +1: adds x2, x2, #8 b.mi 2f -1: ldr x3, [x1], #8 - subs x2, x2, #8 + sub x2, x2, #8 USER(9f, str x3, [x0], #8 ) - b.pl 1b 2: adds x2, x2, #4 b.mi 3f ldr w3, [x1], #4 @@ -51,11 +62,13 @@ USER(9f, strh w3, [x0], #2 ) ldrb w3, [x1] USER(9f, strb w3, [x0] ) 5: mov x0, #0 +ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_HAS_PAN, \ + CONFIG_ARM64_PAN) ret ENDPROC(__copy_to_user) .section .fixup,"ax" .align 2 -9: sub x0, x4, x0 // bytes not copied +9: sub x0, x5, x0 // bytes not copied ret .previous diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index bdeb5d38c2dd72..eb48d5df4a0f72 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -143,7 +143,12 @@ __dma_clean_range: dcache_line_size x2, x3 sub x3, x2, #1 bic x0, x0, x3 -1: alternative_insn "dc cvac, x0", "dc civac, x0", ARM64_WORKAROUND_CLEAN_CACHE +1: +alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE + dc cvac, x0 +alternative_else + dc civac, x0 +alternative_endif add x0, x0, x2 cmp x0, x1 b.lo 1b diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index 76c1e6cd36fc46..d70ff14dbdbdd3 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -53,8 +53,6 @@ static void flush_context(void) __flush_icache_all(); } -#ifdef CONFIG_SMP - static void set_mm_context(struct mm_struct *mm, unsigned int asid) { unsigned long flags; @@ -110,23 +108,12 @@ static void reset_context(void *info) cpu_switch_mm(mm->pgd, mm); } -#else - -static inline void set_mm_context(struct mm_struct *mm, unsigned int asid) -{ - mm->context.id = asid; - cpumask_copy(mm_cpumask(mm), cpumask_of(smp_processor_id())); -} - -#endif - void __new_context(struct mm_struct *mm) { unsigned int asid; unsigned int bits = asid_bits(); raw_spin_lock(&cpu_asid_lock); -#ifdef CONFIG_SMP /* * Check the ASID again, in case the change was broadcast from another * CPU before we acquired the lock. @@ -136,7 +123,6 @@ void __new_context(struct mm_struct *mm) raw_spin_unlock(&cpu_asid_lock); return; } -#endif /* * At this point, it is guaranteed that the current mm (with an old * ASID) isn't active on any other CPU since the ASIDs are changed @@ -155,10 +141,8 @@ void __new_context(struct mm_struct *mm) cpu_last_asid = ASID_FIRST_VERSION; asid = cpu_last_asid + smp_processor_id(); flush_context(); -#ifdef CONFIG_SMP smp_wmb(); smp_call_function(reset_context, NULL, 1); -#endif cpu_last_asid += NR_CPUS - 1; } diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c index d16a1cead23f1c..0bcc4bc94b4ad3 100644 --- a/arch/arm64/mm/dma-mapping.c +++ b/arch/arm64/mm/dma-mapping.c @@ -144,6 +144,7 @@ static void *__dma_alloc(struct device *dev, size_t size, struct page *page; void *ptr, *coherent_ptr; bool coherent = is_device_dma_coherent(dev); + pgprot_t prot = __get_dma_pgprot(attrs, PAGE_KERNEL, false); size = PAGE_ALIGN(size); @@ -171,9 +172,7 @@ static void *__dma_alloc(struct device *dev, size_t size, /* create a coherent mapping */ page = virt_to_page(ptr); coherent_ptr = dma_common_contiguous_remap(page, size, VM_USERMAP, - __get_dma_pgprot(attrs, - __pgprot(PROT_NORMAL_NC), false), - NULL); + prot, NULL); if (!coherent_ptr) goto no_map; @@ -303,9 +302,10 @@ static void __swiotlb_sync_sg_for_device(struct device *dev, sg->length, dir); } -/* vma->vm_page_prot must be set appropriately before calling this function */ -static int __dma_common_mmap(struct device *dev, struct vm_area_struct *vma, - void *cpu_addr, dma_addr_t dma_addr, size_t size) +static int __swiotlb_mmap(struct device *dev, + struct vm_area_struct *vma, + void *cpu_addr, dma_addr_t dma_addr, size_t size, + struct dma_attrs *attrs) { int ret = -ENXIO; unsigned long nr_vma_pages = (vma->vm_end - vma->vm_start) >> @@ -314,6 +314,9 @@ static int __dma_common_mmap(struct device *dev, struct vm_area_struct *vma, unsigned long pfn = dma_to_phys(dev, dma_addr) >> PAGE_SHIFT; unsigned long off = vma->vm_pgoff; + vma->vm_page_prot = __get_dma_pgprot(attrs, vma->vm_page_prot, + is_device_dma_coherent(dev)); + if (dma_mmap_from_coherent(dev, vma, cpu_addr, size, &ret)) return ret; @@ -327,20 +330,24 @@ static int __dma_common_mmap(struct device *dev, struct vm_area_struct *vma, return ret; } -static int __swiotlb_mmap(struct device *dev, - struct vm_area_struct *vma, - void *cpu_addr, dma_addr_t dma_addr, size_t size, - struct dma_attrs *attrs) +static int __swiotlb_get_sgtable(struct device *dev, struct sg_table *sgt, + void *cpu_addr, dma_addr_t handle, size_t size, + struct dma_attrs *attrs) { - vma->vm_page_prot = __get_dma_pgprot(attrs, vma->vm_page_prot, - is_device_dma_coherent(dev)); - return __dma_common_mmap(dev, vma, cpu_addr, dma_addr, size); + int ret = sg_alloc_table(sgt, 1, GFP_KERNEL); + + if (!ret) + sg_set_page(sgt->sgl, phys_to_page(dma_to_phys(dev, handle)), + PAGE_ALIGN(size), 0); + + return ret; } static struct dma_map_ops swiotlb_dma_ops = { .alloc = __dma_alloc, .free = __dma_free, .mmap = __swiotlb_mmap, + .get_sgtable = __swiotlb_get_sgtable, .map_page = __swiotlb_map_page, .unmap_page = __swiotlb_unmap_page, .map_sg = __swiotlb_map_sg_attrs, diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 94d98cd1aad8a1..aba9ead1384c03 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -30,9 +30,11 @@ #include #include +#include #include #include #include +#include #include #include #include @@ -223,6 +225,13 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } + /* + * PAN bit set implies the fault happened in kernel space, but not + * in the arch's user access functions. + */ + if (IS_ENABLED(CONFIG_ARM64_PAN) && (regs->pstate & PSR_PAN_BIT)) + goto no_context; + /* * As per x86, we may deadlock here. However, since the kernel only * validly references user space from well defined areas of the code, @@ -492,14 +501,22 @@ asmlinkage void __exception do_sp_pc_abort(unsigned long addr, arm64_notify_die("Oops - SP/PC alignment exception", regs, &info, esr); } -static struct fault_info debug_fault_info[] = { +int __init early_brk64(unsigned long addr, unsigned int esr, + struct pt_regs *regs); + +/* + * __refdata because early_brk64 is __init, but the reference to it is + * clobbered at arch_initcall time. + * See traps.c and debug-monitors.c:debug_traps_init(). + */ +static struct fault_info __refdata debug_fault_info[] = { { do_bad, SIGTRAP, TRAP_HWBKPT, "hardware breakpoint" }, { do_bad, SIGTRAP, TRAP_HWBKPT, "hardware single-step" }, { do_bad, SIGTRAP, TRAP_HWBKPT, "hardware watchpoint" }, { do_bad, SIGBUS, 0, "unknown 3" }, { do_bad, SIGTRAP, TRAP_BRKPT, "aarch32 BKPT" }, { do_bad, SIGTRAP, 0, "aarch32 vector catch" }, - { do_bad, SIGTRAP, TRAP_BRKPT, "aarch64 BRK" }, + { early_brk64, SIGTRAP, TRAP_BRKPT, "aarch64 BRK" }, { do_bad, SIGBUS, 0, "unknown 7" }, }; @@ -536,3 +553,10 @@ asmlinkage int __exception do_debug_exception(unsigned long addr, return 0; } + +#ifdef CONFIG_ARM64_PAN +void cpu_enable_pan(void) +{ + config_sctlr_el1(SCTLR_EL1_SPAN, 0); +} +#endif /* CONFIG_ARM64_PAN */ diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 4dfa3975ce5bef..c26b804015e80c 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -60,14 +60,10 @@ void copy_to_user_page(struct vm_area_struct *vma, struct page *page, unsigned long uaddr, void *dst, const void *src, unsigned long len) { -#ifdef CONFIG_SMP preempt_disable(); -#endif memcpy(dst, src, len); flush_ptrace_access(vma, page, uaddr, dst, len); -#ifdef CONFIG_SMP preempt_enable(); -#endif } void __sync_icache_dcache(pte_t pte, unsigned long addr) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 831ec534d449b0..383b03ff38f850 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -13,10 +13,6 @@ * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write to the Free Software - * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ #include diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index ad87ce826cce45..f5c0680d17d9ef 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -358,9 +358,9 @@ void free_initmem(void) #ifdef CONFIG_BLK_DEV_INITRD -static int keep_initrd; +static int keep_initrd __initdata; -void free_initrd_mem(unsigned long start, unsigned long end) +void __init free_initrd_mem(unsigned long start, unsigned long end) { if (!keep_initrd) free_reserved_area((void *)start, (void *)end, 0, "initrd"); diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index a4ede4e2ddd1e7..9211b8527f2580 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -267,7 +267,7 @@ static void *late_alloc(unsigned long size) return ptr; } -static void __ref create_mapping(phys_addr_t phys, unsigned long virt, +static void __init create_mapping(phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot) { if (virt < VMALLOC_START) { @@ -460,17 +460,6 @@ void __init paging_init(void) cpu_set_default_tcr_t0sz(); } -/* - * Enable the identity mapping to allow the MMU disabling. - */ -void setup_mm_for_reboot(void) -{ - cpu_set_reserved_ttbr0(); - flush_tlb_all(); - cpu_set_idmap_tcr_t0sz(); - cpu_switch_mm(idmap_pg_dir, &init_mm); -} - /* * Check whether a kernel address is valid (derived from arch/x86/). */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 39139a3aa16d81..e4ee7bd8830aed 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -34,11 +34,7 @@ #define TCR_TG_FLAGS TCR_TG0_4K | TCR_TG1_4K #endif -#ifdef CONFIG_SMP #define TCR_SMP_FLAGS TCR_SHARED -#else -#define TCR_SMP_FLAGS 0 -#endif /* PTWs cacheable, inner/outer WBWA */ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA @@ -150,13 +146,13 @@ ENDPROC(cpu_do_switch_mm) * value of the SCTLR_EL1 register. */ ENTRY(__cpu_setup) - ic iallu // I+BTB cache invalidate tlbi vmalle1is // invalidate I + D TLBs dsb ish mov x0, #3 << 20 msr cpacr_el1, x0 // Enable FP/ASIMD - msr mdscr_el1, xzr // Reset mdscr_el1 + mov x0, #1 << 12 // Reset mdscr_el1 and disable + msr mdscr_el1, x0 // access to the DCC from EL0 /* * Memory region attributes for LPAE: * @@ -196,6 +192,19 @@ ENTRY(__cpu_setup) */ mrs x9, ID_AA64MMFR0_EL1 bfi x10, x9, #32, #3 +#ifdef CONFIG_ARM64_HW_AFDBM + /* + * Hardware update of the Access and Dirty bits. + */ + mrs x9, ID_AA64MMFR1_EL1 + and x9, x9, #0xf + cbz x9, 2f + cmp x9, #2 + b.lt 1f + orr x10, x10, #TCR_HD // hardware Dirty flag update +1: orr x10, x10, #TCR_HA // hardware Access flag update +2: +#endif /* CONFIG_ARM64_HW_AFDBM */ msr tcr_el1, x10 ret // return to head.S ENDPROC(__cpu_setup) diff --git a/arch/avr32/include/asm/atomic.h b/arch/avr32/include/asm/atomic.h index 2d07ce1c5327ca..97c9bdf8340962 100644 --- a/arch/avr32/include/asm/atomic.h +++ b/arch/avr32/include/asm/atomic.h @@ -44,6 +44,18 @@ static inline int __atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OP_RETURN(sub, sub, rKs21) ATOMIC_OP_RETURN(add, add, r) +#define ATOMIC_OP(op, asm_op) \ +ATOMIC_OP_RETURN(op, asm_op, r) \ +static inline void atomic_##op(int i, atomic_t *v) \ +{ \ + (void)__atomic_##op##_return(i, v); \ +} + +ATOMIC_OP(and, and) +ATOMIC_OP(or, or) +ATOMIC_OP(xor, eor) + +#undef ATOMIC_OP #undef ATOMIC_OP_RETURN /* diff --git a/arch/blackfin/include/asm/atomic.h b/arch/blackfin/include/asm/atomic.h index a107a98e99783e..1c1c42330c99c9 100644 --- a/arch/blackfin/include/asm/atomic.h +++ b/arch/blackfin/include/asm/atomic.h @@ -16,19 +16,21 @@ #include asmlinkage int __raw_uncached_fetch_asm(const volatile int *ptr); -asmlinkage int __raw_atomic_update_asm(volatile int *ptr, int value); -asmlinkage int __raw_atomic_clear_asm(volatile int *ptr, int value); -asmlinkage int __raw_atomic_set_asm(volatile int *ptr, int value); +asmlinkage int __raw_atomic_add_asm(volatile int *ptr, int value); + +asmlinkage int __raw_atomic_and_asm(volatile int *ptr, int value); +asmlinkage int __raw_atomic_or_asm(volatile int *ptr, int value); asmlinkage int __raw_atomic_xor_asm(volatile int *ptr, int value); asmlinkage int __raw_atomic_test_asm(const volatile int *ptr, int value); #define atomic_read(v) __raw_uncached_fetch_asm(&(v)->counter) -#define atomic_add_return(i, v) __raw_atomic_update_asm(&(v)->counter, i) -#define atomic_sub_return(i, v) __raw_atomic_update_asm(&(v)->counter, -(i)) +#define atomic_add_return(i, v) __raw_atomic_add_asm(&(v)->counter, i) +#define atomic_sub_return(i, v) __raw_atomic_add_asm(&(v)->counter, -(i)) -#define atomic_clear_mask(m, v) __raw_atomic_clear_asm(&(v)->counter, m) -#define atomic_set_mask(m, v) __raw_atomic_set_asm(&(v)->counter, m) +#define atomic_or(i, v) (void)__raw_atomic_or_asm(&(v)->counter, i) +#define atomic_and(i, v) (void)__raw_atomic_and_asm(&(v)->counter, i) +#define atomic_xor(i, v) (void)__raw_atomic_xor_asm(&(v)->counter, i) #endif diff --git a/arch/blackfin/kernel/bfin_ksyms.c b/arch/blackfin/kernel/bfin_ksyms.c index c446591b961ddc..a401c27b69b4a3 100644 --- a/arch/blackfin/kernel/bfin_ksyms.c +++ b/arch/blackfin/kernel/bfin_ksyms.c @@ -83,11 +83,12 @@ EXPORT_SYMBOL(insl); EXPORT_SYMBOL(insl_16); #ifdef CONFIG_SMP -EXPORT_SYMBOL(__raw_atomic_update_asm); -EXPORT_SYMBOL(__raw_atomic_clear_asm); -EXPORT_SYMBOL(__raw_atomic_set_asm); +EXPORT_SYMBOL(__raw_atomic_add_asm); +EXPORT_SYMBOL(__raw_atomic_and_asm); +EXPORT_SYMBOL(__raw_atomic_or_asm); EXPORT_SYMBOL(__raw_atomic_xor_asm); EXPORT_SYMBOL(__raw_atomic_test_asm); + EXPORT_SYMBOL(__raw_xchg_1_asm); EXPORT_SYMBOL(__raw_xchg_2_asm); EXPORT_SYMBOL(__raw_xchg_4_asm); diff --git a/arch/blackfin/mach-bf561/atomic.S b/arch/blackfin/mach-bf561/atomic.S index 2a08df8e8c4c77..26fccb5568b97e 100644 --- a/arch/blackfin/mach-bf561/atomic.S +++ b/arch/blackfin/mach-bf561/atomic.S @@ -587,10 +587,10 @@ ENDPROC(___raw_write_unlock_asm) * r0 = ptr * r1 = value * - * Add a signed value to a 32bit word and return the new value atomically. + * ADD a signed value to a 32bit word and return the new value atomically. * Clobbers: r3:0, p1:0 */ -ENTRY(___raw_atomic_update_asm) +ENTRY(___raw_atomic_add_asm) p1 = r0; r3 = r1; [--sp] = rets; @@ -603,19 +603,19 @@ ENTRY(___raw_atomic_update_asm) r0 = r3; rets = [sp++]; rts; -ENDPROC(___raw_atomic_update_asm) +ENDPROC(___raw_atomic_add_asm) /* * r0 = ptr * r1 = mask * - * Clear the mask bits from a 32bit word and return the old 32bit value + * AND the mask bits from a 32bit word and return the old 32bit value * atomically. * Clobbers: r3:0, p1:0 */ -ENTRY(___raw_atomic_clear_asm) +ENTRY(___raw_atomic_and_asm) p1 = r0; - r3 = ~r1; + r3 = r1; [--sp] = rets; call _get_core_lock; r2 = [p1]; @@ -627,17 +627,17 @@ ENTRY(___raw_atomic_clear_asm) r0 = r3; rets = [sp++]; rts; -ENDPROC(___raw_atomic_clear_asm) +ENDPROC(___raw_atomic_and_asm) /* * r0 = ptr * r1 = mask * - * Set the mask bits into a 32bit word and return the old 32bit value + * OR the mask bits into a 32bit word and return the old 32bit value * atomically. * Clobbers: r3:0, p1:0 */ -ENTRY(___raw_atomic_set_asm) +ENTRY(___raw_atomic_or_asm) p1 = r0; r3 = r1; [--sp] = rets; @@ -651,7 +651,7 @@ ENTRY(___raw_atomic_set_asm) r0 = r3; rets = [sp++]; rts; -ENDPROC(___raw_atomic_set_asm) +ENDPROC(___raw_atomic_or_asm) /* * r0 = ptr @@ -787,7 +787,7 @@ ENTRY(___raw_bit_set_asm) r2 = r1; r1 = 1; r1 <<= r2; - jump ___raw_atomic_set_asm + jump ___raw_atomic_or_asm ENDPROC(___raw_bit_set_asm) /* @@ -798,10 +798,10 @@ ENDPROC(___raw_bit_set_asm) * Clobbers: r3:0, p1:0 */ ENTRY(___raw_bit_clear_asm) - r2 = r1; - r1 = 1; - r1 <<= r2; - jump ___raw_atomic_clear_asm + r2 = 1; + r2 <<= r1; + r1 = ~r2; + jump ___raw_atomic_and_asm ENDPROC(___raw_bit_clear_asm) /* diff --git a/arch/blackfin/mach-common/smp.c b/arch/blackfin/mach-common/smp.c index 1c7259597395e5..0030e21cfceb3b 100644 --- a/arch/blackfin/mach-common/smp.c +++ b/arch/blackfin/mach-common/smp.c @@ -195,7 +195,7 @@ void send_ipi(const struct cpumask *cpumask, enum ipi_message_type msg) local_irq_save(flags); for_each_cpu(cpu, cpumask) { bfin_ipi_data = &per_cpu(bfin_ipi, cpu); - atomic_set_mask((1 << msg), &bfin_ipi_data->bits); + atomic_or((1 << msg), &bfin_ipi_data->bits); atomic_inc(&bfin_ipi_data->count); } local_irq_restore(flags); diff --git a/arch/frv/include/asm/atomic.h b/arch/frv/include/asm/atomic.h index 102190a61d65a1..0da689def4cc66 100644 --- a/arch/frv/include/asm/atomic.h +++ b/arch/frv/include/asm/atomic.h @@ -15,7 +15,6 @@ #define _ASM_ATOMIC_H #include -#include #include #include @@ -23,6 +22,8 @@ #error not SMP safe #endif +#include + /* * Atomic operations that C can't guarantee us. Useful for * resource counting etc.. @@ -34,56 +35,26 @@ #define atomic_read(v) ACCESS_ONCE((v)->counter) #define atomic_set(v, i) (((v)->counter) = (i)) -#ifndef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS -static inline int atomic_add_return(int i, atomic_t *v) +static inline int atomic_inc_return(atomic_t *v) { - unsigned long val; + return __atomic_add_return(1, &v->counter); +} - asm("0: \n" - " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */ - " ckeq icc3,cc7 \n" - " ld.p %M0,%1 \n" /* LD.P/ORCR must be atomic */ - " orcr cc7,cc7,cc3 \n" /* set CC3 to true */ - " add%I2 %1,%2,%1 \n" - " cst.p %1,%M0 ,cc3,#1 \n" - " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* clear ICC3.Z if store happens */ - " beq icc3,#0,0b \n" - : "+U"(v->counter), "=&r"(val) - : "NPr"(i) - : "memory", "cc7", "cc3", "icc3" - ); +static inline int atomic_dec_return(atomic_t *v) +{ + return __atomic_sub_return(1, &v->counter); +} - return val; +static inline int atomic_add_return(int i, atomic_t *v) +{ + return __atomic_add_return(i, &v->counter); } static inline int atomic_sub_return(int i, atomic_t *v) { - unsigned long val; - - asm("0: \n" - " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */ - " ckeq icc3,cc7 \n" - " ld.p %M0,%1 \n" /* LD.P/ORCR must be atomic */ - " orcr cc7,cc7,cc3 \n" /* set CC3 to true */ - " sub%I2 %1,%2,%1 \n" - " cst.p %1,%M0 ,cc3,#1 \n" - " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* clear ICC3.Z if store happens */ - " beq icc3,#0,0b \n" - : "+U"(v->counter), "=&r"(val) - : "NPr"(i) - : "memory", "cc7", "cc3", "icc3" - ); - - return val; + return __atomic_sub_return(i, &v->counter); } -#else - -extern int atomic_add_return(int i, atomic_t *v); -extern int atomic_sub_return(int i, atomic_t *v); - -#endif - static inline int atomic_add_negative(int i, atomic_t *v) { return atomic_add_return(i, v) < 0; @@ -101,17 +72,14 @@ static inline void atomic_sub(int i, atomic_t *v) static inline void atomic_inc(atomic_t *v) { - atomic_add_return(1, v); + atomic_inc_return(v); } static inline void atomic_dec(atomic_t *v) { - atomic_sub_return(1, v); + atomic_dec_return(v); } -#define atomic_dec_return(v) atomic_sub_return(1, (v)) -#define atomic_inc_return(v) atomic_add_return(1, (v)) - #define atomic_sub_and_test(i,v) (atomic_sub_return((i), (v)) == 0) #define atomic_dec_and_test(v) (atomic_sub_return(1, (v)) == 0) #define atomic_inc_and_test(v) (atomic_add_return(1, (v)) == 0) @@ -120,18 +88,19 @@ static inline void atomic_dec(atomic_t *v) * 64-bit atomic ops */ typedef struct { - volatile long long counter; + long long counter; } atomic64_t; #define ATOMIC64_INIT(i) { (i) } -static inline long long atomic64_read(atomic64_t *v) +static inline long long atomic64_read(const atomic64_t *v) { long long counter; asm("ldd%I1 %M1,%0" : "=e"(counter) : "m"(v->counter)); + return counter; } @@ -142,10 +111,25 @@ static inline void atomic64_set(atomic64_t *v, long long i) : "e"(i)); } -extern long long atomic64_inc_return(atomic64_t *v); -extern long long atomic64_dec_return(atomic64_t *v); -extern long long atomic64_add_return(long long i, atomic64_t *v); -extern long long atomic64_sub_return(long long i, atomic64_t *v); +static inline long long atomic64_inc_return(atomic64_t *v) +{ + return __atomic64_add_return(1, &v->counter); +} + +static inline long long atomic64_dec_return(atomic64_t *v) +{ + return __atomic64_sub_return(1, &v->counter); +} + +static inline long long atomic64_add_return(long long i, atomic64_t *v) +{ + return __atomic64_add_return(i, &v->counter); +} + +static inline long long atomic64_sub_return(long long i, atomic64_t *v) +{ + return __atomic64_sub_return(i, &v->counter); +} static inline long long atomic64_add_negative(long long i, atomic64_t *v) { @@ -176,6 +160,7 @@ static inline void atomic64_dec(atomic64_t *v) #define atomic64_dec_and_test(v) (atomic64_dec_return((v)) == 0) #define atomic64_inc_and_test(v) (atomic64_inc_return((v)) == 0) + #define atomic_cmpxchg(v, old, new) (cmpxchg(&(v)->counter, old, new)) #define atomic_xchg(v, new) (xchg(&(v)->counter, new)) #define atomic64_cmpxchg(v, old, new) (__cmpxchg_64(old, new, &(v)->counter)) @@ -196,5 +181,21 @@ static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u) return c; } +#define ATOMIC_OP(op) \ +static inline void atomic_##op(int i, atomic_t *v) \ +{ \ + (void)__atomic32_fetch_##op(i, &v->counter); \ +} \ + \ +static inline void atomic64_##op(long long i, atomic64_t *v) \ +{ \ + (void)__atomic64_fetch_##op(i, &v->counter); \ +} + +ATOMIC_OP(or) +ATOMIC_OP(and) +ATOMIC_OP(xor) + +#undef ATOMIC_OP #endif /* _ASM_ATOMIC_H */ diff --git a/arch/frv/include/asm/atomic_defs.h b/arch/frv/include/asm/atomic_defs.h new file mode 100644 index 00000000000000..36e126d2f801ad --- /dev/null +++ b/arch/frv/include/asm/atomic_defs.h @@ -0,0 +1,172 @@ + +#include + +#ifdef __ATOMIC_LIB__ + +#ifdef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS + +#define ATOMIC_QUALS +#define ATOMIC_EXPORT(x) EXPORT_SYMBOL(x) + +#else /* !OUTOFLINE && LIB */ + +#define ATOMIC_OP_RETURN(op) +#define ATOMIC_FETCH_OP(op) + +#endif /* OUTOFLINE */ + +#else /* !__ATOMIC_LIB__ */ + +#define ATOMIC_EXPORT(x) + +#ifdef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS + +#define ATOMIC_OP_RETURN(op) \ +extern int __atomic_##op##_return(int i, int *v); \ +extern long long __atomic64_##op##_return(long long i, long long *v); + +#define ATOMIC_FETCH_OP(op) \ +extern int __atomic32_fetch_##op(int i, int *v); \ +extern long long __atomic64_fetch_##op(long long i, long long *v); + +#else /* !OUTOFLINE && !LIB */ + +#define ATOMIC_QUALS static inline + +#endif /* OUTOFLINE */ +#endif /* __ATOMIC_LIB__ */ + + +/* + * Note on the 64 bit inline asm variants... + * + * CSTD is a conditional instruction and needs a constrained memory reference. + * Normally 'U' provides the correct constraints for conditional instructions + * and this is used for the 32 bit version, however 'U' does not appear to work + * for 64 bit values (gcc-4.9) + * + * The exact constraint is that conditional instructions cannot deal with an + * immediate displacement in the memory reference, so what we do is we read the + * address through a volatile cast into a local variable in order to insure we + * _have_ to compute the correct address without displacement. This allows us + * to use the regular 'm' for the memory address. + * + * Furthermore, the %Ln operand, which prints the low word register (r+1), + * really only works for registers, this means we cannot allow immediate values + * for the 64 bit versions -- like we do for the 32 bit ones. + * + */ + +#ifndef ATOMIC_OP_RETURN +#define ATOMIC_OP_RETURN(op) \ +ATOMIC_QUALS int __atomic_##op##_return(int i, int *v) \ +{ \ + int val; \ + \ + asm volatile( \ + "0: \n" \ + " orcc gr0,gr0,gr0,icc3 \n" \ + " ckeq icc3,cc7 \n" \ + " ld.p %M0,%1 \n" \ + " orcr cc7,cc7,cc3 \n" \ + " "#op"%I2 %1,%2,%1 \n" \ + " cst.p %1,%M0 ,cc3,#1 \n" \ + " corcc gr29,gr29,gr0 ,cc3,#1 \n" \ + " beq icc3,#0,0b \n" \ + : "+U"(*v), "=&r"(val) \ + : "NPr"(i) \ + : "memory", "cc7", "cc3", "icc3" \ + ); \ + \ + return val; \ +} \ +ATOMIC_EXPORT(__atomic_##op##_return); \ + \ +ATOMIC_QUALS long long __atomic64_##op##_return(long long i, long long *v) \ +{ \ + long long *__v = READ_ONCE(v); \ + long long val; \ + \ + asm volatile( \ + "0: \n" \ + " orcc gr0,gr0,gr0,icc3 \n" \ + " ckeq icc3,cc7 \n" \ + " ldd.p %M0,%1 \n" \ + " orcr cc7,cc7,cc3 \n" \ + " "#op"cc %L1,%L2,%L1,icc0 \n" \ + " "#op"x %1,%2,%1,icc0 \n" \ + " cstd.p %1,%M0 ,cc3,#1 \n" \ + " corcc gr29,gr29,gr0 ,cc3,#1 \n" \ + " beq icc3,#0,0b \n" \ + : "+m"(*__v), "=&e"(val) \ + : "e"(i) \ + : "memory", "cc7", "cc3", "icc0", "icc3" \ + ); \ + \ + return val; \ +} \ +ATOMIC_EXPORT(__atomic64_##op##_return); +#endif + +#ifndef ATOMIC_FETCH_OP +#define ATOMIC_FETCH_OP(op) \ +ATOMIC_QUALS int __atomic32_fetch_##op(int i, int *v) \ +{ \ + int old, tmp; \ + \ + asm volatile( \ + "0: \n" \ + " orcc gr0,gr0,gr0,icc3 \n" \ + " ckeq icc3,cc7 \n" \ + " ld.p %M0,%1 \n" \ + " orcr cc7,cc7,cc3 \n" \ + " "#op"%I3 %1,%3,%2 \n" \ + " cst.p %2,%M0 ,cc3,#1 \n" \ + " corcc gr29,gr29,gr0 ,cc3,#1 \n" \ + " beq icc3,#0,0b \n" \ + : "+U"(*v), "=&r"(old), "=r"(tmp) \ + : "NPr"(i) \ + : "memory", "cc7", "cc3", "icc3" \ + ); \ + \ + return old; \ +} \ +ATOMIC_EXPORT(__atomic32_fetch_##op); \ + \ +ATOMIC_QUALS long long __atomic64_fetch_##op(long long i, long long *v) \ +{ \ + long long *__v = READ_ONCE(v); \ + long long old, tmp; \ + \ + asm volatile( \ + "0: \n" \ + " orcc gr0,gr0,gr0,icc3 \n" \ + " ckeq icc3,cc7 \n" \ + " ldd.p %M0,%1 \n" \ + " orcr cc7,cc7,cc3 \n" \ + " "#op" %L1,%L3,%L2 \n" \ + " "#op" %1,%3,%2 \n" \ + " cstd.p %2,%M0 ,cc3,#1 \n" \ + " corcc gr29,gr29,gr0 ,cc3,#1 \n" \ + " beq icc3,#0,0b \n" \ + : "+m"(*__v), "=&e"(old), "=e"(tmp) \ + : "e"(i) \ + : "memory", "cc7", "cc3", "icc3" \ + ); \ + \ + return old; \ +} \ +ATOMIC_EXPORT(__atomic64_fetch_##op); +#endif + +ATOMIC_FETCH_OP(or) +ATOMIC_FETCH_OP(and) +ATOMIC_FETCH_OP(xor) + +ATOMIC_OP_RETURN(add) +ATOMIC_OP_RETURN(sub) + +#undef ATOMIC_FETCH_OP +#undef ATOMIC_OP_RETURN +#undef ATOMIC_QUALS +#undef ATOMIC_EXPORT diff --git a/arch/frv/include/asm/bitops.h b/arch/frv/include/asm/bitops.h index 96de220ef131ed..0df8e95e37151d 100644 --- a/arch/frv/include/asm/bitops.h +++ b/arch/frv/include/asm/bitops.h @@ -25,109 +25,30 @@ #include -#ifndef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS -static inline -unsigned long atomic_test_and_ANDNOT_mask(unsigned long mask, volatile unsigned long *v) -{ - unsigned long old, tmp; - - asm volatile( - "0: \n" - " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */ - " ckeq icc3,cc7 \n" - " ld.p %M0,%1 \n" /* LD.P/ORCR are atomic */ - " orcr cc7,cc7,cc3 \n" /* set CC3 to true */ - " and%I3 %1,%3,%2 \n" - " cst.p %2,%M0 ,cc3,#1 \n" /* if store happens... */ - " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* ... clear ICC3.Z */ - " beq icc3,#0,0b \n" - : "+U"(*v), "=&r"(old), "=r"(tmp) - : "NPr"(~mask) - : "memory", "cc7", "cc3", "icc3" - ); - - return old; -} - -static inline -unsigned long atomic_test_and_OR_mask(unsigned long mask, volatile unsigned long *v) -{ - unsigned long old, tmp; - - asm volatile( - "0: \n" - " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */ - " ckeq icc3,cc7 \n" - " ld.p %M0,%1 \n" /* LD.P/ORCR are atomic */ - " orcr cc7,cc7,cc3 \n" /* set CC3 to true */ - " or%I3 %1,%3,%2 \n" - " cst.p %2,%M0 ,cc3,#1 \n" /* if store happens... */ - " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* ... clear ICC3.Z */ - " beq icc3,#0,0b \n" - : "+U"(*v), "=&r"(old), "=r"(tmp) - : "NPr"(mask) - : "memory", "cc7", "cc3", "icc3" - ); - - return old; -} - -static inline -unsigned long atomic_test_and_XOR_mask(unsigned long mask, volatile unsigned long *v) -{ - unsigned long old, tmp; - - asm volatile( - "0: \n" - " orcc gr0,gr0,gr0,icc3 \n" /* set ICC3.Z */ - " ckeq icc3,cc7 \n" - " ld.p %M0,%1 \n" /* LD.P/ORCR are atomic */ - " orcr cc7,cc7,cc3 \n" /* set CC3 to true */ - " xor%I3 %1,%3,%2 \n" - " cst.p %2,%M0 ,cc3,#1 \n" /* if store happens... */ - " corcc gr29,gr29,gr0 ,cc3,#1 \n" /* ... clear ICC3.Z */ - " beq icc3,#0,0b \n" - : "+U"(*v), "=&r"(old), "=r"(tmp) - : "NPr"(mask) - : "memory", "cc7", "cc3", "icc3" - ); - - return old; -} - -#else - -extern unsigned long atomic_test_and_ANDNOT_mask(unsigned long mask, volatile unsigned long *v); -extern unsigned long atomic_test_and_OR_mask(unsigned long mask, volatile unsigned long *v); -extern unsigned long atomic_test_and_XOR_mask(unsigned long mask, volatile unsigned long *v); - -#endif - -#define atomic_clear_mask(mask, v) atomic_test_and_ANDNOT_mask((mask), (v)) -#define atomic_set_mask(mask, v) atomic_test_and_OR_mask((mask), (v)) +#include static inline int test_and_clear_bit(unsigned long nr, volatile void *addr) { - volatile unsigned long *ptr = addr; - unsigned long mask = 1UL << (nr & 31); + unsigned int *ptr = (void *)addr; + unsigned int mask = 1UL << (nr & 31); ptr += nr >> 5; - return (atomic_test_and_ANDNOT_mask(mask, ptr) & mask) != 0; + return (__atomic32_fetch_and(~mask, ptr) & mask) != 0; } static inline int test_and_set_bit(unsigned long nr, volatile void *addr) { - volatile unsigned long *ptr = addr; - unsigned long mask = 1UL << (nr & 31); + unsigned int *ptr = (void *)addr; + unsigned int mask = 1UL << (nr & 31); ptr += nr >> 5; - return (atomic_test_and_OR_mask(mask, ptr) & mask) != 0; + return (__atomic32_fetch_or(mask, ptr) & mask) != 0; } static inline int test_and_change_bit(unsigned long nr, volatile void *addr) { - volatile unsigned long *ptr = addr; - unsigned long mask = 1UL << (nr & 31); + unsigned int *ptr = (void *)addr; + unsigned int mask = 1UL << (nr & 31); ptr += nr >> 5; - return (atomic_test_and_XOR_mask(mask, ptr) & mask) != 0; + return (__atomic32_fetch_xor(mask, ptr) & mask) != 0; } static inline void clear_bit(unsigned long nr, volatile void *addr) diff --git a/arch/frv/kernel/dma.c b/arch/frv/kernel/dma.c index 156184e17e57d0..370dc9fa0b1191 100644 --- a/arch/frv/kernel/dma.c +++ b/arch/frv/kernel/dma.c @@ -109,13 +109,13 @@ static struct frv_dma_channel frv_dma_channels[FRV_DMA_NCHANS] = { static DEFINE_RWLOCK(frv_dma_channels_lock); -unsigned long frv_dma_inprogress; +unsigned int frv_dma_inprogress; #define frv_clear_dma_inprogress(channel) \ - atomic_clear_mask(1 << (channel), &frv_dma_inprogress); + (void)__atomic32_fetch_and(~(1 << (channel)), &frv_dma_inprogress); #define frv_set_dma_inprogress(channel) \ - atomic_set_mask(1 << (channel), &frv_dma_inprogress); + (void)__atomic32_fetch_or(1 << (channel), &frv_dma_inprogress); /*****************************************************************************/ /* diff --git a/arch/frv/kernel/frv_ksyms.c b/arch/frv/kernel/frv_ksyms.c index 86c516d96dcda6..cdb4ce9960ebe2 100644 --- a/arch/frv/kernel/frv_ksyms.c +++ b/arch/frv/kernel/frv_ksyms.c @@ -58,11 +58,6 @@ EXPORT_SYMBOL(__outsl_ns); EXPORT_SYMBOL(__insl_ns); #ifdef CONFIG_FRV_OUTOFLINE_ATOMIC_OPS -EXPORT_SYMBOL(atomic_test_and_ANDNOT_mask); -EXPORT_SYMBOL(atomic_test_and_OR_mask); -EXPORT_SYMBOL(atomic_test_and_XOR_mask); -EXPORT_SYMBOL(atomic_add_return); -EXPORT_SYMBOL(atomic_sub_return); EXPORT_SYMBOL(__xchg_32); EXPORT_SYMBOL(__cmpxchg_32); #endif diff --git a/arch/frv/lib/Makefile b/arch/frv/lib/Makefile index 4ff2fb1e6b1694..970e8b4f1a0262 100644 --- a/arch/frv/lib/Makefile +++ b/arch/frv/lib/Makefile @@ -5,4 +5,4 @@ lib-y := \ __ashldi3.o __lshrdi3.o __muldi3.o __ashrdi3.o __negdi2.o __ucmpdi2.o \ checksum.o memcpy.o memset.o atomic-ops.o atomic64-ops.o \ - outsl_ns.o outsl_sw.o insl_ns.o insl_sw.o cache.o + outsl_ns.o outsl_sw.o insl_ns.o insl_sw.o cache.o atomic-lib.o diff --git a/arch/frv/lib/atomic-lib.c b/arch/frv/lib/atomic-lib.c new file mode 100644 index 00000000000000..4d1b887c248bda --- /dev/null +++ b/arch/frv/lib/atomic-lib.c @@ -0,0 +1,7 @@ + +#include +#include + +#define __ATOMIC_LIB__ + +#include diff --git a/arch/frv/lib/atomic-ops.S b/arch/frv/lib/atomic-ops.S index 5e9e6ab5dd0e89..b7439a960b5baa 100644 --- a/arch/frv/lib/atomic-ops.S +++ b/arch/frv/lib/atomic-ops.S @@ -17,116 +17,6 @@ .text .balign 4 -############################################################################### -# -# unsigned long atomic_test_and_ANDNOT_mask(unsigned long mask, volatile unsigned long *v); -# -############################################################################### - .globl atomic_test_and_ANDNOT_mask - .type atomic_test_and_ANDNOT_mask,@function -atomic_test_and_ANDNOT_mask: - not.p gr8,gr10 -0: - orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */ - ckeq icc3,cc7 - ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */ - orcr cc7,cc7,cc3 /* set CC3 to true */ - and gr8,gr10,gr11 - cst.p gr11,@(gr9,gr0) ,cc3,#1 - corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */ - beq icc3,#0,0b - bralr - - .size atomic_test_and_ANDNOT_mask, .-atomic_test_and_ANDNOT_mask - -############################################################################### -# -# unsigned long atomic_test_and_OR_mask(unsigned long mask, volatile unsigned long *v); -# -############################################################################### - .globl atomic_test_and_OR_mask - .type atomic_test_and_OR_mask,@function -atomic_test_and_OR_mask: - or.p gr8,gr8,gr10 -0: - orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */ - ckeq icc3,cc7 - ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */ - orcr cc7,cc7,cc3 /* set CC3 to true */ - or gr8,gr10,gr11 - cst.p gr11,@(gr9,gr0) ,cc3,#1 - corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */ - beq icc3,#0,0b - bralr - - .size atomic_test_and_OR_mask, .-atomic_test_and_OR_mask - -############################################################################### -# -# unsigned long atomic_test_and_XOR_mask(unsigned long mask, volatile unsigned long *v); -# -############################################################################### - .globl atomic_test_and_XOR_mask - .type atomic_test_and_XOR_mask,@function -atomic_test_and_XOR_mask: - or.p gr8,gr8,gr10 -0: - orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */ - ckeq icc3,cc7 - ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */ - orcr cc7,cc7,cc3 /* set CC3 to true */ - xor gr8,gr10,gr11 - cst.p gr11,@(gr9,gr0) ,cc3,#1 - corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */ - beq icc3,#0,0b - bralr - - .size atomic_test_and_XOR_mask, .-atomic_test_and_XOR_mask - -############################################################################### -# -# int atomic_add_return(int i, atomic_t *v) -# -############################################################################### - .globl atomic_add_return - .type atomic_add_return,@function -atomic_add_return: - or.p gr8,gr8,gr10 -0: - orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */ - ckeq icc3,cc7 - ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */ - orcr cc7,cc7,cc3 /* set CC3 to true */ - add gr8,gr10,gr8 - cst.p gr8,@(gr9,gr0) ,cc3,#1 - corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */ - beq icc3,#0,0b - bralr - - .size atomic_add_return, .-atomic_add_return - -############################################################################### -# -# int atomic_sub_return(int i, atomic_t *v) -# -############################################################################### - .globl atomic_sub_return - .type atomic_sub_return,@function -atomic_sub_return: - or.p gr8,gr8,gr10 -0: - orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */ - ckeq icc3,cc7 - ld.p @(gr9,gr0),gr8 /* LD.P/ORCR must be atomic */ - orcr cc7,cc7,cc3 /* set CC3 to true */ - sub gr8,gr10,gr8 - cst.p gr8,@(gr9,gr0) ,cc3,#1 - corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */ - beq icc3,#0,0b - bralr - - .size atomic_sub_return, .-atomic_sub_return - ############################################################################### # # uint32_t __xchg_32(uint32_t i, uint32_t *v) diff --git a/arch/frv/lib/atomic64-ops.S b/arch/frv/lib/atomic64-ops.S index b6194eeac127e4..c4c472308a33a3 100644 --- a/arch/frv/lib/atomic64-ops.S +++ b/arch/frv/lib/atomic64-ops.S @@ -18,100 +18,6 @@ .balign 4 -############################################################################### -# -# long long atomic64_inc_return(atomic64_t *v) -# -############################################################################### - .globl atomic64_inc_return - .type atomic64_inc_return,@function -atomic64_inc_return: - or.p gr8,gr8,gr10 -0: - orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */ - ckeq icc3,cc7 - ldd.p @(gr10,gr0),gr8 /* LDD.P/ORCR must be atomic */ - orcr cc7,cc7,cc3 /* set CC3 to true */ - addicc gr9,#1,gr9,icc0 - addxi gr8,#0,gr8,icc0 - cstd.p gr8,@(gr10,gr0) ,cc3,#1 - corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */ - beq icc3,#0,0b - bralr - - .size atomic64_inc_return, .-atomic64_inc_return - -############################################################################### -# -# long long atomic64_dec_return(atomic64_t *v) -# -############################################################################### - .globl atomic64_dec_return - .type atomic64_dec_return,@function -atomic64_dec_return: - or.p gr8,gr8,gr10 -0: - orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */ - ckeq icc3,cc7 - ldd.p @(gr10,gr0),gr8 /* LDD.P/ORCR must be atomic */ - orcr cc7,cc7,cc3 /* set CC3 to true */ - subicc gr9,#1,gr9,icc0 - subxi gr8,#0,gr8,icc0 - cstd.p gr8,@(gr10,gr0) ,cc3,#1 - corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */ - beq icc3,#0,0b - bralr - - .size atomic64_dec_return, .-atomic64_dec_return - -############################################################################### -# -# long long atomic64_add_return(long long i, atomic64_t *v) -# -############################################################################### - .globl atomic64_add_return - .type atomic64_add_return,@function -atomic64_add_return: - or.p gr8,gr8,gr4 - or gr9,gr9,gr5 -0: - orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */ - ckeq icc3,cc7 - ldd.p @(gr10,gr0),gr8 /* LDD.P/ORCR must be atomic */ - orcr cc7,cc7,cc3 /* set CC3 to true */ - addcc gr9,gr5,gr9,icc0 - addx gr8,gr4,gr8,icc0 - cstd.p gr8,@(gr10,gr0) ,cc3,#1 - corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */ - beq icc3,#0,0b - bralr - - .size atomic64_add_return, .-atomic64_add_return - -############################################################################### -# -# long long atomic64_sub_return(long long i, atomic64_t *v) -# -############################################################################### - .globl atomic64_sub_return - .type atomic64_sub_return,@function -atomic64_sub_return: - or.p gr8,gr8,gr4 - or gr9,gr9,gr5 -0: - orcc gr0,gr0,gr0,icc3 /* set ICC3.Z */ - ckeq icc3,cc7 - ldd.p @(gr10,gr0),gr8 /* LDD.P/ORCR must be atomic */ - orcr cc7,cc7,cc3 /* set CC3 to true */ - subcc gr9,gr5,gr9,icc0 - subx gr8,gr4,gr8,icc0 - cstd.p gr8,@(gr10,gr0) ,cc3,#1 - corcc gr29,gr29,gr0 ,cc3,#1 /* clear ICC3.Z if store happens */ - beq icc3,#0,0b - bralr - - .size atomic64_sub_return, .-atomic64_sub_return - ############################################################################### # # uint64_t __xchg_64(uint64_t i, uint64_t *v) diff --git a/arch/h8300/include/asm/atomic.h b/arch/h8300/include/asm/atomic.h index 7ca73f8546cc89..702ee539f87da5 100644 --- a/arch/h8300/include/asm/atomic.h +++ b/arch/h8300/include/asm/atomic.h @@ -16,83 +16,52 @@ #include -static inline int atomic_add_return(int i, atomic_t *v) -{ - h8300flags flags; - int ret; - - flags = arch_local_irq_save(); - ret = v->counter += i; - arch_local_irq_restore(flags); - return ret; +#define ATOMIC_OP_RETURN(op, c_op) \ +static inline int atomic_##op##_return(int i, atomic_t *v) \ +{ \ + h8300flags flags; \ + int ret; \ + \ + flags = arch_local_irq_save(); \ + ret = v->counter c_op i; \ + arch_local_irq_restore(flags); \ + return ret; \ } -#define atomic_add(i, v) atomic_add_return(i, v) -#define atomic_add_negative(a, v) (atomic_add_return((a), (v)) < 0) - -static inline int atomic_sub_return(int i, atomic_t *v) -{ - h8300flags flags; - int ret; - - flags = arch_local_irq_save(); - ret = v->counter -= i; - arch_local_irq_restore(flags); - return ret; +#define ATOMIC_OP(op, c_op) \ +static inline void atomic_##op(int i, atomic_t *v) \ +{ \ + h8300flags flags; \ + \ + flags = arch_local_irq_save(); \ + v->counter c_op i; \ + arch_local_irq_restore(flags); \ } -#define atomic_sub(i, v) atomic_sub_return(i, v) -#define atomic_sub_and_test(i, v) (atomic_sub_return(i, v) == 0) +ATOMIC_OP_RETURN(add, +=) +ATOMIC_OP_RETURN(sub, -=) -static inline int atomic_inc_return(atomic_t *v) -{ - h8300flags flags; - int ret; +ATOMIC_OP(and, &=) +ATOMIC_OP(or, |=) +ATOMIC_OP(xor, ^=) - flags = arch_local_irq_save(); - v->counter++; - ret = v->counter; - arch_local_irq_restore(flags); - return ret; -} - -#define atomic_inc(v) atomic_inc_return(v) +#undef ATOMIC_OP_RETURN +#undef ATOMIC_OP -/* - * atomic_inc_and_test - increment and test - * @v: pointer of type atomic_t - * - * Atomically increments @v by 1 - * and returns true if the result is zero, or false for all - * other cases. - */ -#define atomic_inc_and_test(v) (atomic_inc_return(v) == 0) +#define atomic_add(i, v) (void)atomic_add_return(i, v) +#define atomic_add_negative(a, v) (atomic_add_return((a), (v)) < 0) -static inline int atomic_dec_return(atomic_t *v) -{ - h8300flags flags; - int ret; +#define atomic_sub(i, v) (void)atomic_sub_return(i, v) +#define atomic_sub_and_test(i, v) (atomic_sub_return(i, v) == 0) - flags = arch_local_irq_save(); - --v->counter; - ret = v->counter; - arch_local_irq_restore(flags); - return ret; -} +#define atomic_inc_return(v) atomic_add_return(1, v) +#define atomic_dec_return(v) atomic_sub_return(1, v) -#define atomic_dec(v) atomic_dec_return(v) +#define atomic_inc(v) (void)atomic_inc_return(v) +#define atomic_inc_and_test(v) (atomic_inc_return(v) == 0) -static inline int atomic_dec_and_test(atomic_t *v) -{ - h8300flags flags; - int ret; - - flags = arch_local_irq_save(); - --v->counter; - ret = v->counter; - arch_local_irq_restore(flags); - return ret == 0; -} +#define atomic_dec(v) (void)atomic_dec_return(v) +#define atomic_dec_and_test(v) (atomic_dec_return(v) == 0) static inline int atomic_cmpxchg(atomic_t *v, int old, int new) { @@ -120,40 +89,4 @@ static inline int __atomic_add_unless(atomic_t *v, int a, int u) return ret; } -static inline void atomic_clear_mask(unsigned long mask, unsigned long *v) -{ - unsigned char ccr; - unsigned long tmp; - - __asm__ __volatile__("stc ccr,%w3\n\t" - "orc #0x80,ccr\n\t" - "mov.l %0,%1\n\t" - "and.l %2,%1\n\t" - "mov.l %1,%0\n\t" - "ldc %w3,ccr" - : "=m"(*v), "=r"(tmp) - : "g"(~(mask)), "r"(ccr)); -} - -static inline void atomic_set_mask(unsigned long mask, unsigned long *v) -{ - unsigned char ccr; - unsigned long tmp; - - __asm__ __volatile__("stc ccr,%w3\n\t" - "orc #0x80,ccr\n\t" - "mov.l %0,%1\n\t" - "or.l %2,%1\n\t" - "mov.l %1,%0\n\t" - "ldc %w3,ccr" - : "=m"(*v), "=r"(tmp) - : "g"(~(mask)), "r"(ccr)); -} - -/* Atomic operations are already serializing */ -#define smp_mb__before_atomic_dec() barrier() -#define smp_mb__after_atomic_dec() barrier() -#define smp_mb__before_atomic_inc() barrier() -#define smp_mb__after_atomic_inc() barrier() - #endif /* __ARCH_H8300_ATOMIC __ */ diff --git a/arch/hexagon/include/asm/atomic.h b/arch/hexagon/include/asm/atomic.h index 93d07025f183d6..811d61f6422da1 100644 --- a/arch/hexagon/include/asm/atomic.h +++ b/arch/hexagon/include/asm/atomic.h @@ -132,6 +132,10 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP diff --git a/arch/ia64/include/asm/atomic.h b/arch/ia64/include/asm/atomic.h index 0bf03501fe5ca6..be4beeb77d57c2 100644 --- a/arch/ia64/include/asm/atomic.h +++ b/arch/ia64/include/asm/atomic.h @@ -45,8 +45,6 @@ ia64_atomic_##op (int i, atomic_t *v) \ ATOMIC_OP(add, +) ATOMIC_OP(sub, -) -#undef ATOMIC_OP - #define atomic_add_return(i,v) \ ({ \ int __ia64_aar_i = (i); \ @@ -71,6 +69,16 @@ ATOMIC_OP(sub, -) : ia64_atomic_sub(__ia64_asr_i, v); \ }) +ATOMIC_OP(and, &) +ATOMIC_OP(or, |) +ATOMIC_OP(xor, ^) + +#define atomic_and(i,v) (void)ia64_atomic_and(i,v) +#define atomic_or(i,v) (void)ia64_atomic_or(i,v) +#define atomic_xor(i,v) (void)ia64_atomic_xor(i,v) + +#undef ATOMIC_OP + #define ATOMIC64_OP(op, c_op) \ static __inline__ long \ ia64_atomic64_##op (__s64 i, atomic64_t *v) \ @@ -89,8 +97,6 @@ ia64_atomic64_##op (__s64 i, atomic64_t *v) \ ATOMIC64_OP(add, +) ATOMIC64_OP(sub, -) -#undef ATOMIC64_OP - #define atomic64_add_return(i,v) \ ({ \ long __ia64_aar_i = (i); \ @@ -115,6 +121,16 @@ ATOMIC64_OP(sub, -) : ia64_atomic64_sub(__ia64_asr_i, v); \ }) +ATOMIC64_OP(and, &) +ATOMIC64_OP(or, |) +ATOMIC64_OP(xor, ^) + +#define atomic64_and(i,v) (void)ia64_atomic64_and(i,v) +#define atomic64_or(i,v) (void)ia64_atomic64_or(i,v) +#define atomic64_xor(i,v) (void)ia64_atomic64_xor(i,v) + +#undef ATOMIC64_OP + #define atomic_cmpxchg(v, old, new) (cmpxchg(&((v)->counter), old, new)) #define atomic_xchg(v, new) (xchg(&((v)->counter), new)) diff --git a/arch/ia64/include/asm/barrier.h b/arch/ia64/include/asm/barrier.h index 843ba435e43bc2..df896a1c41d348 100644 --- a/arch/ia64/include/asm/barrier.h +++ b/arch/ia64/include/asm/barrier.h @@ -66,12 +66,12 @@ do { \ compiletime_assert_atomic_type(*p); \ barrier(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ barrier(); \ ___p1; \ diff --git a/arch/m32r/include/asm/atomic.h b/arch/m32r/include/asm/atomic.h index 31bb74adba082a..025e2a17049362 100644 --- a/arch/m32r/include/asm/atomic.h +++ b/arch/m32r/include/asm/atomic.h @@ -94,6 +94,10 @@ static __inline__ int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP @@ -239,45 +243,4 @@ static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u) return c; } - -static __inline__ void atomic_clear_mask(unsigned long mask, atomic_t *addr) -{ - unsigned long flags; - unsigned long tmp; - - local_irq_save(flags); - __asm__ __volatile__ ( - "# atomic_clear_mask \n\t" - DCACHE_CLEAR("%0", "r5", "%1") - M32R_LOCK" %0, @%1; \n\t" - "and %0, %2; \n\t" - M32R_UNLOCK" %0, @%1; \n\t" - : "=&r" (tmp) - : "r" (addr), "r" (~mask) - : "memory" - __ATOMIC_CLOBBER - ); - local_irq_restore(flags); -} - -static __inline__ void atomic_set_mask(unsigned long mask, atomic_t *addr) -{ - unsigned long flags; - unsigned long tmp; - - local_irq_save(flags); - __asm__ __volatile__ ( - "# atomic_set_mask \n\t" - DCACHE_CLEAR("%0", "r5", "%1") - M32R_LOCK" %0, @%1; \n\t" - "or %0, %2; \n\t" - M32R_UNLOCK" %0, @%1; \n\t" - : "=&r" (tmp) - : "r" (addr), "r" (mask) - : "memory" - __ATOMIC_CLOBBER - ); - local_irq_restore(flags); -} - #endif /* _ASM_M32R_ATOMIC_H */ diff --git a/arch/m32r/kernel/smp.c b/arch/m32r/kernel/smp.c index c18ddc74ef9a60..62d6961e7f2b6f 100644 --- a/arch/m32r/kernel/smp.c +++ b/arch/m32r/kernel/smp.c @@ -156,7 +156,7 @@ void smp_flush_cache_all(void) cpumask_clear_cpu(smp_processor_id(), &cpumask); spin_lock(&flushcache_lock); mask=cpumask_bits(&cpumask); - atomic_set_mask(*mask, (atomic_t *)&flushcache_cpumask); + atomic_or(*mask, (atomic_t *)&flushcache_cpumask); send_IPI_mask(&cpumask, INVALIDATE_CACHE_IPI, 0); _flush_cache_copyback_all(); while (flushcache_cpumask) @@ -407,7 +407,7 @@ static void flush_tlb_others(cpumask_t cpumask, struct mm_struct *mm, flush_vma = vma; flush_va = va; mask=cpumask_bits(&cpumask); - atomic_set_mask(*mask, (atomic_t *)&flush_cpumask); + atomic_or(*mask, (atomic_t *)&flush_cpumask); /* * We have to send the IPI only to diff --git a/arch/m68k/include/asm/atomic.h b/arch/m68k/include/asm/atomic.h index e85f047fb072e8..039fac120cc0eb 100644 --- a/arch/m68k/include/asm/atomic.h +++ b/arch/m68k/include/asm/atomic.h @@ -77,6 +77,10 @@ static inline int atomic_##op##_return(int i, atomic_t * v) \ ATOMIC_OPS(add, +=, add) ATOMIC_OPS(sub, -=, sub) +ATOMIC_OP(and, &=, and) +ATOMIC_OP(or, |=, or) +ATOMIC_OP(xor, ^=, eor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP @@ -170,16 +174,6 @@ static inline int atomic_add_negative(int i, atomic_t *v) return c != 0; } -static inline void atomic_clear_mask(unsigned long mask, unsigned long *v) -{ - __asm__ __volatile__("andl %1,%0" : "+m" (*v) : ASM_DI (~(mask))); -} - -static inline void atomic_set_mask(unsigned long mask, unsigned long *v) -{ - __asm__ __volatile__("orl %1,%0" : "+m" (*v) : ASM_DI (mask)); -} - static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u) { int c, old; diff --git a/arch/metag/include/asm/atomic_lnkget.h b/arch/metag/include/asm/atomic_lnkget.h index 948d8688643c58..21c4c268b86c6e 100644 --- a/arch/metag/include/asm/atomic_lnkget.h +++ b/arch/metag/include/asm/atomic_lnkget.h @@ -74,44 +74,14 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP -static inline void atomic_clear_mask(unsigned int mask, atomic_t *v) -{ - int temp; - - asm volatile ( - "1: LNKGETD %0, [%1]\n" - " AND %0, %0, %2\n" - " LNKSETD [%1] %0\n" - " DEFR %0, TXSTAT\n" - " ANDT %0, %0, #HI(0x3f000000)\n" - " CMPT %0, #HI(0x02000000)\n" - " BNZ 1b\n" - : "=&d" (temp) - : "da" (&v->counter), "bd" (~mask) - : "cc"); -} - -static inline void atomic_set_mask(unsigned int mask, atomic_t *v) -{ - int temp; - - asm volatile ( - "1: LNKGETD %0, [%1]\n" - " OR %0, %0, %2\n" - " LNKSETD [%1], %0\n" - " DEFR %0, TXSTAT\n" - " ANDT %0, %0, #HI(0x3f000000)\n" - " CMPT %0, #HI(0x02000000)\n" - " BNZ 1b\n" - : "=&d" (temp) - : "da" (&v->counter), "bd" (mask) - : "cc"); -} - static inline int atomic_cmpxchg(atomic_t *v, int old, int new) { int result, temp; diff --git a/arch/metag/include/asm/atomic_lock1.h b/arch/metag/include/asm/atomic_lock1.h index f5d5898c10201c..f8efe380fe8b33 100644 --- a/arch/metag/include/asm/atomic_lock1.h +++ b/arch/metag/include/asm/atomic_lock1.h @@ -68,31 +68,14 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add, +=) ATOMIC_OPS(sub, -=) +ATOMIC_OP(and, &=) +ATOMIC_OP(or, |=) +ATOMIC_OP(xor, ^=) #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP -static inline void atomic_clear_mask(unsigned int mask, atomic_t *v) -{ - unsigned long flags; - - __global_lock1(flags); - fence(); - v->counter &= ~mask; - __global_unlock1(flags); -} - -static inline void atomic_set_mask(unsigned int mask, atomic_t *v) -{ - unsigned long flags; - - __global_lock1(flags); - fence(); - v->counter |= mask; - __global_unlock1(flags); -} - static inline int atomic_cmpxchg(atomic_t *v, int old, int new) { int ret; diff --git a/arch/metag/include/asm/barrier.h b/arch/metag/include/asm/barrier.h index 5a696e50793034..172b7e5efc53f9 100644 --- a/arch/metag/include/asm/barrier.h +++ b/arch/metag/include/asm/barrier.h @@ -90,12 +90,12 @@ static inline void fence(void) do { \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ ___p1; \ diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig index 4ab9a794bbcd4c..752acca8de1fa9 100644 --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -1,8 +1,10 @@ config MIPS bool default y + select ARCH_SUPPORTS_UPROBES select ARCH_MIGHT_HAVE_PC_PARPORT select ARCH_MIGHT_HAVE_PC_SERIO + select ARCH_USE_CMPXCHG_LOCKREF if 64BIT select HAVE_CONTEXT_TRACKING select HAVE_GENERIC_DMA_COHERENT select HAVE_IDE @@ -13,7 +15,6 @@ config MIPS select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_TRACEHOOK select HAVE_BPF_JIT if !CPU_MICROMIPS - select ARCH_HAVE_CUSTOM_GPIO_H select HAVE_FUNCTION_TRACER select HAVE_DYNAMIC_FTRACE select HAVE_FTRACE_MCOUNT_RECORD @@ -409,6 +410,7 @@ config MIPS_MALTA select CEVT_R4K select CSRC_R4K select CLKSRC_MIPS_GIC + select COMMON_CLK select DMA_MAYBE_COHERENT select GENERIC_ISA_DMA select HAVE_PCSPKR_PLATFORM @@ -459,6 +461,7 @@ config MIPS_SEAD3 select CEVT_R4K select CSRC_R4K select CLKSRC_MIPS_GIC + select COMMON_CLK select CPU_MIPSR2_IRQ_VI select CPU_MIPSR2_IRQ_EI select DMA_NONCOHERENT @@ -899,6 +902,7 @@ config NLM_XLP_BOARD select SYS_SUPPORTS_32BIT_KERNEL select SYS_SUPPORTS_64BIT_KERNEL select ARCH_PHYS_ADDR_T_64BIT + select ARCH_REQUIRE_GPIOLIB select SYS_SUPPORTS_BIG_ENDIAN select SYS_SUPPORTS_LITTLE_ENDIAN select SYS_SUPPORTS_HIGHMEM @@ -948,6 +952,7 @@ source "arch/mips/jazz/Kconfig" source "arch/mips/jz4740/Kconfig" source "arch/mips/lantiq/Kconfig" source "arch/mips/lasat/Kconfig" +source "arch/mips/pistachio/Kconfig" source "arch/mips/pmcs-msp71xx/Kconfig" source "arch/mips/ralink/Kconfig" source "arch/mips/sgi-ip27/Kconfig" @@ -1041,6 +1046,9 @@ config FW_CFE config ARCH_DMA_ADDR_T_64BIT def_bool (HIGHMEM && ARCH_PHYS_ADDR_T_64BIT) || 64BIT +config ARCH_SUPPORTS_UPROBES + bool + config DMA_MAYBE_COHERENT select DMA_NONCOHERENT bool @@ -1364,7 +1372,7 @@ config CPU_MIPS32_R2 otherwise CPU_MIPS32_R1 is a safe bet for any MIPS32 system. config CPU_MIPS32_R6 - bool "MIPS32 Release 6 (EXPERIMENTAL)" + bool "MIPS32 Release 6" depends on SYS_HAS_CPU_MIPS32_R6 select CPU_HAS_PREFETCH select CPU_SUPPORTS_32BIT_KERNEL @@ -1415,7 +1423,7 @@ config CPU_MIPS64_R2 otherwise CPU_MIPS64_R1 is a safe bet for any MIPS64 system. config CPU_MIPS64_R6 - bool "MIPS64 Release 6 (EXPERIMENTAL)" + bool "MIPS64 Release 6" depends on SYS_HAS_CPU_MIPS64_R6 select CPU_HAS_PREFETCH select CPU_SUPPORTS_32BIT_KERNEL @@ -1965,6 +1973,7 @@ config 32BIT select TRAD_SIGNALS help Select this option if you want to build a 32-bit kernel. + config 64BIT bool "64-bit kernel" depends on CPU_SUPPORTS_64BIT_KERNEL && SYS_SUPPORTS_64BIT_KERNEL @@ -2110,7 +2119,7 @@ config CPU_R4K_CACHE_TLB config MIPS_MT_SMP bool "MIPS MT SMP support (1 TC on each available VPE)" - depends on SYS_SUPPORTS_MULTITHREADING + depends on SYS_SUPPORTS_MULTITHREADING && !CPU_MIPSR6 select CPU_MIPSR2_IRQ_VI select CPU_MIPSR2_IRQ_EI select SYNC_R4K @@ -2211,7 +2220,7 @@ config MIPS_VPE_APSP_API_MT config MIPS_CMP bool "MIPS CMP framework support (DEPRECATED)" - depends on SYS_SUPPORTS_MIPS_CMP + depends on SYS_SUPPORTS_MIPS_CMP && !CPU_MIPSR6 select MIPS_GIC_IPI select SMP select SYNC_R4K @@ -2228,7 +2237,7 @@ config MIPS_CMP config MIPS_CPS bool "MIPS Coherent Processing System support" - depends on SYS_SUPPORTS_MIPS_CPS + depends on SYS_SUPPORTS_MIPS_CPS && !CPU_MIPSR6 select MIPS_CM select MIPS_CPC select MIPS_CPS_PM if HOTPLUG_CPU @@ -2303,7 +2312,7 @@ config CPU_MICROMIPS endchoice config CPU_HAS_MSA - bool "Support for the MIPS SIMD Architecture (EXPERIMENTAL)" + bool "Support for the MIPS SIMD Architecture" depends on CPU_SUPPORTS_MSA depends on 64BIT || MIPS_O32_FP64_SUPPORT help @@ -2643,7 +2652,7 @@ config SECCOMP If unsure, say Y. Only embedded should say N here. config MIPS_O32_FP64_SUPPORT - bool "Support for O32 binaries using 64-bit FP (EXPERIMENTAL)" + bool "Support for O32 binaries using 64-bit FP" depends on 32BIT || MIPS32_O32 help When this is enabled, the kernel will support use of 64-bit floating diff --git a/arch/mips/Kconfig.debug b/arch/mips/Kconfig.debug index 3a2b775e845893..e250524021aca8 100644 --- a/arch/mips/Kconfig.debug +++ b/arch/mips/Kconfig.debug @@ -87,15 +87,6 @@ config SB1XXX_CORELIS Select compile flags that produce code that can be processed by the Corelis mksym utility and UDB Emulator. -config RUNTIME_DEBUG - bool "Enable run-time debugging" - depends on DEBUG_KERNEL - help - If you say Y here, some debugging macros will do run-time checking. - If you say N here, those macros will mostly turn to no-ops. See - arch/mips/include/asm/debug.h for debugging macros. - If unsure, say N. - config DEBUG_ZBOOT bool "Enable compressed kernel support debugging" depends on DEBUG_KERNEL && SYS_SUPPORTS_ZBOOT diff --git a/arch/mips/alchemy/Kconfig b/arch/mips/alchemy/Kconfig index b9628983d62097..7fa24881b708ce 100644 --- a/arch/mips/alchemy/Kconfig +++ b/arch/mips/alchemy/Kconfig @@ -6,13 +6,6 @@ config ALCHEMY_GPIOINT_AU1000 config ALCHEMY_GPIOINT_AU1300 bool -# select this in your board config if you don't want to use the gpio -# namespace as documented in the manuals. In this case however you need -# to create the necessary gpio_* functions in your board code/headers! -# see arch/mips/include/asm/mach-au1x00/gpio.h for more information. -config ALCHEMY_GPIO_INDIRECT - def_bool n - choice prompt "Machine type" depends on MIPS_ALCHEMY diff --git a/arch/mips/alchemy/board-gpr.c b/arch/mips/alchemy/board-gpr.c index acf9a2a37f5a05..79efe4c6e636a3 100644 --- a/arch/mips/alchemy/board-gpr.c +++ b/arch/mips/alchemy/board-gpr.c @@ -34,6 +34,7 @@ #include #include #include +#include #include const char *get_system_type(void) diff --git a/arch/mips/alchemy/board-mtx1.c b/arch/mips/alchemy/board-mtx1.c index 1e3b102389ef51..85bb75669b0d59 100644 --- a/arch/mips/alchemy/board-mtx1.c +++ b/arch/mips/alchemy/board-mtx1.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include diff --git a/arch/mips/alchemy/common/Makefile b/arch/mips/alchemy/common/Makefile index f64744f3b59f77..23800b8e67e5e4 100644 --- a/arch/mips/alchemy/common/Makefile +++ b/arch/mips/alchemy/common/Makefile @@ -5,10 +5,5 @@ # Makefile for the Alchemy Au1xx0 CPUs, generic files. # -obj-y += prom.o time.o clock.o platform.o power.o \ +obj-y += prom.o time.o clock.o platform.o power.o gpiolib.o \ setup.o sleeper.o dma.o dbdma.o vss.o irq.o usb.o - -# optional gpiolib support -ifeq ($(CONFIG_ALCHEMY_GPIO_INDIRECT),) - obj-$(CONFIG_GPIOLIB) += gpiolib.o -endif diff --git a/arch/mips/alchemy/common/irq.c b/arch/mips/alchemy/common/irq.c index 6cb60abfdcc909..4c496c50edf699 100644 --- a/arch/mips/alchemy/common/irq.c +++ b/arch/mips/alchemy/common/irq.c @@ -491,7 +491,7 @@ static int au1x_ic_settype(struct irq_data *d, unsigned int flow_type) default: ret = -EINVAL; } - __irq_set_chip_handler_name_locked(d->irq, chip, handler, name); + irq_set_chip_handler_name_locked(d, chip, handler, name); wmb(); @@ -703,7 +703,7 @@ static int au1300_gpic_settype(struct irq_data *d, unsigned int type) return -EINVAL; } - __irq_set_chip_handler_name_locked(d->irq, &au1300_gpic, hdl, name); + irq_set_chip_handler_name_locked(d, &au1300_gpic, hdl, name); au1300_gpic_chgcfg(d->irq - ALCHEMY_GPIC_INT_BASE, GPIC_CFG_IC_MASK, s); diff --git a/arch/mips/alchemy/common/time.c b/arch/mips/alchemy/common/time.c index 50e17e13c18bbc..f99d3ec17a4592 100644 --- a/arch/mips/alchemy/common/time.c +++ b/arch/mips/alchemy/common/time.c @@ -69,11 +69,6 @@ static int au1x_rtcmatch2_set_next_event(unsigned long delta, return 0; } -static void au1x_rtcmatch2_set_mode(enum clock_event_mode mode, - struct clock_event_device *cd) -{ -} - static irqreturn_t au1x_rtcmatch2_irq(int irq, void *dev_id) { struct clock_event_device *cd = dev_id; @@ -86,7 +81,6 @@ static struct clock_event_device au1x_rtcmatch2_clockdev = { .features = CLOCK_EVT_FEAT_ONESHOT, .rating = 1500, .set_next_event = au1x_rtcmatch2_set_next_event, - .set_mode = au1x_rtcmatch2_set_mode, .cpumask = cpu_all_mask, }; diff --git a/arch/mips/alchemy/devboards/bcsr.c b/arch/mips/alchemy/devboards/bcsr.c index c98c9ea3372c53..324ad72d7c3622 100644 --- a/arch/mips/alchemy/devboards/bcsr.c +++ b/arch/mips/alchemy/devboards/bcsr.c @@ -8,6 +8,7 @@ */ #include +#include #include #include #include @@ -88,10 +89,11 @@ EXPORT_SYMBOL_GPL(bcsr_mod); static void bcsr_csc_handler(unsigned int irq, struct irq_desc *d) { unsigned short bisr = __raw_readw(bcsr_virt + BCSR_REG_INTSTAT); + struct irq_chip *chip = irq_desc_get_chip(d); - disable_irq_nosync(irq); + chained_irq_enter(chip, d); generic_handle_irq(bcsr_csc_base + __ffs(bisr)); - enable_irq(irq); + chained_irq_exit(chip, d); } static void bcsr_irq_mask(struct irq_data *d) diff --git a/arch/mips/alchemy/devboards/db1000.c b/arch/mips/alchemy/devboards/db1000.c index 001102e197f197..bdeed9d13c6fe0 100644 --- a/arch/mips/alchemy/devboards/db1000.c +++ b/arch/mips/alchemy/devboards/db1000.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include #include diff --git a/arch/mips/alchemy/devboards/db1300.c b/arch/mips/alchemy/devboards/db1300.c index 1c64fdbe4c813c..b58077008a5384 100644 --- a/arch/mips/alchemy/devboards/db1300.c +++ b/arch/mips/alchemy/devboards/db1300.c @@ -24,6 +24,7 @@ #include #include +#include #include #include #include diff --git a/arch/mips/alchemy/devboards/db1550.c b/arch/mips/alchemy/devboards/db1550.c index 0fd5177e35ab27..5740bcfdfc7f62 100644 --- a/arch/mips/alchemy/devboards/db1550.c +++ b/arch/mips/alchemy/devboards/db1550.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include diff --git a/arch/mips/alchemy/devboards/pm.c b/arch/mips/alchemy/devboards/pm.c index bfeb8f3c0be66d..93024dc6b314d9 100644 --- a/arch/mips/alchemy/devboards/pm.c +++ b/arch/mips/alchemy/devboards/pm.c @@ -9,7 +9,7 @@ #include #include #include -#include +#include #include /* diff --git a/arch/mips/ar7/gpio.c b/arch/mips/ar7/gpio.c index d8dbd8f0c1d292..f4930456eb8e81 100644 --- a/arch/mips/ar7/gpio.c +++ b/arch/mips/ar7/gpio.c @@ -21,7 +21,10 @@ #include #include -#include +#include + +#define AR7_GPIO_MAX 32 +#define TITAN_GPIO_MAX 51 struct ar7_gpio_chip { void __iomem *regs; diff --git a/arch/mips/ar7/platform.c b/arch/mips/ar7/platform.c index 298b97715d5fe1..58fca9ad5fcc16 100644 --- a/arch/mips/ar7/platform.c +++ b/arch/mips/ar7/platform.c @@ -39,7 +39,6 @@ #include #include -#include #include /***************************************************************************** diff --git a/arch/mips/ar7/setup.c b/arch/mips/ar7/setup.c index 820b7a313d9bfd..7bb9a670bb7357 100644 --- a/arch/mips/ar7/setup.c +++ b/arch/mips/ar7/setup.c @@ -23,7 +23,6 @@ #include #include #include -#include static void ar7_machine_restart(char *command) { diff --git a/arch/mips/ath79/Makefile b/arch/mips/ath79/Makefile index 5c9ff692ff3ca9..fcc382cfc77091 100644 --- a/arch/mips/ath79/Makefile +++ b/arch/mips/ath79/Makefile @@ -8,7 +8,7 @@ # under the terms of the GNU General Public License version 2 as published # by the Free Software Foundation. -obj-y := prom.o setup.o irq.o common.o clock.o gpio.o +obj-y := prom.o setup.o irq.o common.o clock.o obj-$(CONFIG_EARLY_PRINTK) += early_printk.o obj-$(CONFIG_PCI) += pci.o diff --git a/arch/mips/ath79/common.h b/arch/mips/ath79/common.h index e5ea71277f0cf9..ca7cc19adfea7f 100644 --- a/arch/mips/ath79/common.h +++ b/arch/mips/ath79/common.h @@ -25,9 +25,6 @@ unsigned long ath79_get_sys_clk_rate(const char *id); void ath79_ddr_ctrl_init(void); void ath79_ddr_wb_flush(unsigned int reg); -void ath79_gpio_function_enable(u32 mask); -void ath79_gpio_function_disable(u32 mask); -void ath79_gpio_function_setup(u32 set, u32 clear); void ath79_gpio_init(void); #endif /* __ATH79_COMMON_H */ diff --git a/arch/mips/ath79/irq.c b/arch/mips/ath79/irq.c index 2021be20d9d940..807132b838b249 100644 --- a/arch/mips/ath79/irq.c +++ b/arch/mips/ath79/irq.c @@ -123,8 +123,6 @@ static void ar934x_ip2_irq_dispatch(unsigned int irq, struct irq_desc *desc) { u32 status; - disable_irq_nosync(irq); - status = ath79_reset_rr(AR934X_RESET_REG_PCIE_WMAC_INT_STATUS); if (status & AR934X_PCIE_WMAC_INT_PCIE_ALL) { @@ -136,8 +134,6 @@ static void ar934x_ip2_irq_dispatch(unsigned int irq, struct irq_desc *desc) } else { spurious_interrupt(); } - - enable_irq(irq); } static void ar934x_ip2_irq_init(void) @@ -156,14 +152,12 @@ static void qca955x_ip2_irq_dispatch(unsigned int irq, struct irq_desc *desc) { u32 status; - disable_irq_nosync(irq); - status = ath79_reset_rr(QCA955X_RESET_REG_EXT_INT_STATUS); status &= QCA955X_EXT_INT_PCIE_RC1_ALL | QCA955X_EXT_INT_WMAC_ALL; if (status == 0) { spurious_interrupt(); - goto enable; + return; } if (status & QCA955X_EXT_INT_PCIE_RC1_ALL) { @@ -175,17 +169,12 @@ static void qca955x_ip2_irq_dispatch(unsigned int irq, struct irq_desc *desc) /* TODO: flush DDR? */ generic_handle_irq(ATH79_IP2_IRQ(1)); } - -enable: - enable_irq(irq); } static void qca955x_ip3_irq_dispatch(unsigned int irq, struct irq_desc *desc) { u32 status; - disable_irq_nosync(irq); - status = ath79_reset_rr(QCA955X_RESET_REG_EXT_INT_STATUS); status &= QCA955X_EXT_INT_PCIE_RC2_ALL | QCA955X_EXT_INT_USB1 | @@ -193,7 +182,7 @@ static void qca955x_ip3_irq_dispatch(unsigned int irq, struct irq_desc *desc) if (status == 0) { spurious_interrupt(); - goto enable; + return; } if (status & QCA955X_EXT_INT_USB1) { @@ -210,9 +199,6 @@ static void qca955x_ip3_irq_dispatch(unsigned int irq, struct irq_desc *desc) /* TODO: flush DDR? */ generic_handle_irq(ATH79_IP3_IRQ(2)); } - -enable: - enable_irq(irq); } static void qca955x_irq_init(void) diff --git a/arch/mips/bcm47xx/buttons.c b/arch/mips/bcm47xx/buttons.c index 08a4abf09a33e9..52caa75bfe4e5e 100644 --- a/arch/mips/bcm47xx/buttons.c +++ b/arch/mips/bcm47xx/buttons.c @@ -396,10 +396,9 @@ static int __init bcm47xx_buttons_copy(const struct gpio_keys_button *buttons, { size_t size = nbuttons * sizeof(*buttons); - bcm47xx_button_pdata.buttons = kmalloc(size, GFP_KERNEL); + bcm47xx_button_pdata.buttons = kmemdup(buttons, size, GFP_KERNEL); if (!bcm47xx_button_pdata.buttons) return -ENOMEM; - memcpy(bcm47xx_button_pdata.buttons, buttons, size); bcm47xx_button_pdata.nbuttons = nbuttons; return 0; diff --git a/arch/mips/bcm63xx/irq.c b/arch/mips/bcm63xx/irq.c index e3e808a6c54261..1a47ec2a09062b 100644 --- a/arch/mips/bcm63xx/irq.c +++ b/arch/mips/bcm63xx/irq.c @@ -60,7 +60,7 @@ static inline int enable_irq_for_cpu(int cpu, struct irq_data *d, if (m) enable &= cpumask_test_cpu(cpu, m); else if (irqd_affinity_was_set(d)) - enable &= cpumask_test_cpu(cpu, d->affinity); + enable &= cpumask_test_cpu(cpu, irq_data_get_affinity_mask(d)); #endif return enable; } @@ -365,9 +365,9 @@ static int bcm63xx_external_irq_set_type(struct irq_data *d, irqd_set_trigger_type(d, flow_type); if (flow_type & (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_LEVEL_HIGH)) - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); else - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); return IRQ_SET_MASK_OK_NOCOPY; } diff --git a/arch/mips/boot/compressed/Makefile b/arch/mips/boot/compressed/Makefile index dc91bde10d622e..d5bdee115f22f7 100644 --- a/arch/mips/boot/compressed/Makefile +++ b/arch/mips/boot/compressed/Makefile @@ -78,7 +78,7 @@ endif vmlinuzobjs-y += $(obj)/piggy.o -quiet_cmd_zld = LD $@ +quiet_cmd_zld = LD $@ cmd_zld = $(LD) $(LDFLAGS) -Ttext $(VMLINUZ_LOAD_ADDRESS) -T $< $(vmlinuzobjs-y) -o $@ quiet_cmd_strip = STRIP $@ cmd_strip = $(STRIP) -s $@ diff --git a/arch/mips/boot/dts/netlogic/xlp_evp.dts b/arch/mips/boot/dts/netlogic/xlp_evp.dts index 89ad04808c0247..ec16ec2d8d0276 100644 --- a/arch/mips/boot/dts/netlogic/xlp_evp.dts +++ b/arch/mips/boot/dts/netlogic/xlp_evp.dts @@ -110,6 +110,18 @@ read-only; }; }; + + gpio: xlp_gpio@34100 { + compatible = "netlogic,xlp832-gpio"; + reg = <0 0x34100 0x1000>; + #gpio-cells = <2>; + gpio-controller; + + #interrupt-cells = <2>; + interrupt-parent = <&pic>; + interrupts = <39>; + interrupt-controller; + }; }; chosen { diff --git a/arch/mips/boot/dts/netlogic/xlp_fvp.dts b/arch/mips/boot/dts/netlogic/xlp_fvp.dts index 63e62b7bd75892..4bcebe641d8e9f 100644 --- a/arch/mips/boot/dts/netlogic/xlp_fvp.dts +++ b/arch/mips/boot/dts/netlogic/xlp_fvp.dts @@ -110,6 +110,18 @@ read-only; }; }; + + gpio: xlp_gpio@34100 { + compatible = "netlogic,xlp208-gpio"; + reg = <0 0x34100 0x1000>; + #gpio-cells = <2>; + gpio-controller; + + #interrupt-cells = <2>; + interrupt-parent = <&pic>; + interrupts = <39>; + interrupt-controller; + }; }; chosen { diff --git a/arch/mips/boot/dts/netlogic/xlp_gvp.dts b/arch/mips/boot/dts/netlogic/xlp_gvp.dts index bb4ecd1d47fc29..b3ccb82ad7e4c5 100644 --- a/arch/mips/boot/dts/netlogic/xlp_gvp.dts +++ b/arch/mips/boot/dts/netlogic/xlp_gvp.dts @@ -69,6 +69,17 @@ }; }; + gpio: xlp_gpio@114100 { + compatible = "netlogic,xlp980-gpio"; + reg = <0 0x114100 0x1000>; + #gpio-cells = <2>; + gpio-controller; + + #interrupt-cells = <2>; + interrupt-parent = <&pic>; + interrupts = <39>; + interrupt-controller; + }; }; chosen { diff --git a/arch/mips/boot/dts/netlogic/xlp_rvp.dts b/arch/mips/boot/dts/netlogic/xlp_rvp.dts index 7188aed2ea2ed9..3783639a318aad 100644 --- a/arch/mips/boot/dts/netlogic/xlp_rvp.dts +++ b/arch/mips/boot/dts/netlogic/xlp_rvp.dts @@ -69,6 +69,17 @@ }; }; + gpio: xlp_gpio@114100 { + compatible = "netlogic,xlp532-gpio"; + reg = <0 0x114100 0x1000>; + #gpio-cells = <2>; + gpio-controller; + + #interrupt-cells = <2>; + interrupt-parent = <&pic>; + interrupts = <39>; + interrupt-controller; + }; }; chosen { diff --git a/arch/mips/boot/dts/netlogic/xlp_svp.dts b/arch/mips/boot/dts/netlogic/xlp_svp.dts index 1ebd00edaacc44..44d6640c144123 100644 --- a/arch/mips/boot/dts/netlogic/xlp_svp.dts +++ b/arch/mips/boot/dts/netlogic/xlp_svp.dts @@ -110,6 +110,18 @@ read-only; }; }; + + gpio: xlp_gpio@34100 { + compatible = "netlogic,xlp316-gpio"; + reg = <0 0x34100 0x1000>; + #gpio-cells = <2>; + gpio-controller; + + #interrupt-cells = <2>; + interrupt-parent = <&pic>; + interrupts = <39>; + interrupt-controller; + }; }; chosen { diff --git a/arch/mips/cavium-octeon/executive/cvmx-helper-board.c b/arch/mips/cavium-octeon/executive/cvmx-helper-board.c index 9eb0feef441721..36e30d65ba05bc 100644 --- a/arch/mips/cavium-octeon/executive/cvmx-helper-board.c +++ b/arch/mips/cavium-octeon/executive/cvmx-helper-board.c @@ -195,6 +195,12 @@ int cvmx_helper_board_get_mii_address(int ipd_port) return 8; else return -1; + case CVMX_BOARD_TYPE_KONTRON_S1901: + if (ipd_port == CVMX_HELPER_BOARD_MGMT_IPD_PORT) + return 1; + else + return -1; + } /* Some unknown board. Somebody forgot to update this function... */ diff --git a/arch/mips/cavium-octeon/executive/cvmx-helper-util.c b/arch/mips/cavium-octeon/executive/cvmx-helper-util.c index 453d7f66459aab..b45b2975746db7 100644 --- a/arch/mips/cavium-octeon/executive/cvmx-helper-util.c +++ b/arch/mips/cavium-octeon/executive/cvmx-helper-util.c @@ -95,9 +95,9 @@ int cvmx_helper_dump_packet(cvmx_wqe_t *work) uint8_t *data_address; uint8_t *end_of_data; - cvmx_dprintf("Packet Length: %u\n", work->len); - cvmx_dprintf(" Input Port: %u\n", work->ipprt); - cvmx_dprintf(" QoS: %u\n", work->qos); + cvmx_dprintf("Packet Length: %u\n", work->word1.len); + cvmx_dprintf(" Input Port: %u\n", cvmx_wqe_get_port(work)); + cvmx_dprintf(" QoS: %u\n", cvmx_wqe_get_qos(work)); cvmx_dprintf(" Buffers: %u\n", work->word2.s.bufs); if (work->word2.s.bufs == 0) { @@ -127,7 +127,7 @@ int cvmx_helper_dump_packet(cvmx_wqe_t *work) } } else buffer_ptr = work->packet_ptr; - remaining_bytes = work->len; + remaining_bytes = work->word1.len; while (remaining_bytes) { start_of_buffer = @@ -382,6 +382,10 @@ int cvmx_helper_get_ipd_port(int interface, int port) return port + 32; case 3: return port + 36; + case 4: + return port + 40; + case 5: + return port + 44; } return -1; } @@ -404,6 +408,10 @@ int cvmx_helper_get_interface_num(int ipd_port) return 2; else if (ipd_port < 40) return 3; + else if (ipd_port < 44) + return 4; + else if (ipd_port < 48) + return 5; else cvmx_dprintf("cvmx_helper_get_interface_num: Illegal IPD " "port number\n"); @@ -428,6 +436,10 @@ int cvmx_helper_get_interface_index_num(int ipd_port) return ipd_port & 3; else if (ipd_port < 40) return ipd_port & 3; + else if (ipd_port < 44) + return ipd_port & 3; + else if (ipd_port < 48) + return ipd_port & 3; else cvmx_dprintf("cvmx_helper_get_interface_index_num: " "Illegal IPD port number\n"); diff --git a/arch/mips/cavium-octeon/executive/cvmx-helper-xaui.c b/arch/mips/cavium-octeon/executive/cvmx-helper-xaui.c index 7653b7e92197f0..a56ee590de1f36 100644 --- a/arch/mips/cavium-octeon/executive/cvmx-helper-xaui.c +++ b/arch/mips/cavium-octeon/executive/cvmx-helper-xaui.c @@ -124,6 +124,13 @@ int __cvmx_helper_xaui_enable(int interface) union cvmx_gmxx_tx_int_en gmx_tx_int_en; union cvmx_pcsxx_int_en_reg pcsx_int_en_reg; + /* Setup PKND */ + if (octeon_has_feature(OCTEON_FEATURE_PKND)) { + gmx_cfg.u64 = cvmx_read_csr(CVMX_GMXX_PRTX_CFG(0, interface)); + gmx_cfg.s.pknd = cvmx_helper_get_ipd_port(interface, 0); + cvmx_write_csr(CVMX_GMXX_PRTX_CFG(0, interface), gmx_cfg.u64); + } + /* (1) Interface has already been enabled. */ /* (2) Disable GMX. */ @@ -151,7 +158,12 @@ int __cvmx_helper_xaui_enable(int interface) /* (4)c Aply reset sequence */ xauiCtl.u64 = cvmx_read_csr(CVMX_PCSXX_CONTROL1_REG(interface)); xauiCtl.s.lo_pwr = 0; - xauiCtl.s.reset = 1; + + /* Issuing a reset here seems to hang some CN68XX chips. */ + if (!OCTEON_IS_MODEL(OCTEON_CN68XX_PASS1_X) && + !OCTEON_IS_MODEL(OCTEON_CN68XX_PASS2_X)) + xauiCtl.s.reset = 1; + cvmx_write_csr(CVMX_PCSXX_CONTROL1_REG(interface), xauiCtl.u64); /* Wait for PCS to come out of reset */ diff --git a/arch/mips/cavium-octeon/executive/cvmx-helper.c b/arch/mips/cavium-octeon/executive/cvmx-helper.c index 7e5cf7a5e2f326..376701f41cc28e 100644 --- a/arch/mips/cavium-octeon/executive/cvmx-helper.c +++ b/arch/mips/cavium-octeon/executive/cvmx-helper.c @@ -83,6 +83,8 @@ static cvmx_helper_link_info_t */ int cvmx_helper_get_number_of_interfaces(void) { + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) + return 9; if (OCTEON_IS_MODEL(OCTEON_CN56XX) || OCTEON_IS_MODEL(OCTEON_CN52XX)) return 4; else @@ -656,6 +658,21 @@ static int __cvmx_helper_global_setup_pko(void) fau_to.s.tout_val = 0xfff; fau_to.s.tout_enb = 0; cvmx_write_csr(CVMX_IOB_FAU_TIMEOUT, fau_to.u64); + + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) { + union cvmx_pko_reg_min_pkt min_pkt; + + min_pkt.u64 = 0; + min_pkt.s.size1 = 59; + min_pkt.s.size2 = 59; + min_pkt.s.size3 = 59; + min_pkt.s.size4 = 59; + min_pkt.s.size5 = 59; + min_pkt.s.size6 = 59; + min_pkt.s.size7 = 59; + cvmx_write_csr(CVMX_PKO_REG_MIN_PKT, min_pkt.u64); + } + return 0; } diff --git a/arch/mips/cavium-octeon/executive/cvmx-pko.c b/arch/mips/cavium-octeon/executive/cvmx-pko.c index 008b881cdf6477..87be167a7a6ae0 100644 --- a/arch/mips/cavium-octeon/executive/cvmx-pko.c +++ b/arch/mips/cavium-octeon/executive/cvmx-pko.c @@ -39,6 +39,143 @@ * Internal state of packet output */ +static int __cvmx_pko_int(int interface, int index) +{ + switch (interface) { + case 0: + return index; + case 1: + return 4; + case 2: + return index + 0x08; + case 3: + return index + 0x0c; + case 4: + return index + 0x10; + case 5: + return 0x1c; + case 6: + return 0x1d; + case 7: + return 0x1e; + case 8: + return 0x1f; + default: + return -1; + } +} + +static void __cvmx_pko_iport_config(int pko_port) +{ + int queue; + const int num_queues = 1; + const int base_queue = pko_port; + const int static_priority_end = 1; + const int static_priority_base = 1; + + for (queue = 0; queue < num_queues; queue++) { + union cvmx_pko_mem_iqueue_ptrs config; + cvmx_cmd_queue_result_t cmd_res; + uint64_t *buf_ptr; + + config.u64 = 0; + config.s.index = queue; + config.s.qid = base_queue + queue; + config.s.ipid = pko_port; + config.s.tail = (queue == (num_queues - 1)); + config.s.s_tail = (queue == static_priority_end); + config.s.static_p = (static_priority_base >= 0); + config.s.static_q = (queue <= static_priority_end); + config.s.qos_mask = 0xff; + + cmd_res = cvmx_cmd_queue_initialize( + CVMX_CMD_QUEUE_PKO(base_queue + queue), + CVMX_PKO_MAX_QUEUE_DEPTH, + CVMX_FPA_OUTPUT_BUFFER_POOL, + (CVMX_FPA_OUTPUT_BUFFER_POOL_SIZE - + CVMX_PKO_COMMAND_BUFFER_SIZE_ADJUST * 8)); + + WARN(cmd_res, + "%s: cmd_res=%d pko_port=%d base_queue=%d num_queues=%d queue=%d\n", + __func__, (int)cmd_res, pko_port, base_queue, + num_queues, queue); + + buf_ptr = (uint64_t *)cvmx_cmd_queue_buffer( + CVMX_CMD_QUEUE_PKO(base_queue + queue)); + config.s.buf_ptr = cvmx_ptr_to_phys(buf_ptr) >> 7; + CVMX_SYNCWS; + cvmx_write_csr(CVMX_PKO_MEM_IQUEUE_PTRS, config.u64); + } +} + +static void __cvmx_pko_queue_alloc_o68(void) +{ + int port; + + for (port = 0; port < 48; port++) + __cvmx_pko_iport_config(port); +} + +static void __cvmx_pko_port_map_o68(void) +{ + int port; + int interface, index; + cvmx_helper_interface_mode_t mode; + union cvmx_pko_mem_iport_ptrs config; + + /* + * Initialize every iport with the invalid eid. + */ + config.u64 = 0; + config.s.eid = 31; /* Invalid */ + for (port = 0; port < 128; port++) { + config.s.ipid = port; + cvmx_write_csr(CVMX_PKO_MEM_IPORT_PTRS, config.u64); + } + + /* + * Set up PKO_MEM_IPORT_PTRS + */ + for (port = 0; port < 48; port++) { + interface = cvmx_helper_get_interface_num(port); + index = cvmx_helper_get_interface_index_num(port); + mode = cvmx_helper_interface_get_mode(interface); + if (mode == CVMX_HELPER_INTERFACE_MODE_DISABLED) + continue; + + config.s.ipid = port; + config.s.qos_mask = 0xff; + config.s.crc = 1; + config.s.min_pkt = 1; + config.s.intr = __cvmx_pko_int(interface, index); + config.s.eid = config.s.intr; + config.s.pipe = (mode == CVMX_HELPER_INTERFACE_MODE_LOOP) ? + index : port; + cvmx_write_csr(CVMX_PKO_MEM_IPORT_PTRS, config.u64); + } +} + +static void __cvmx_pko_chip_init(void) +{ + int i; + + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) { + __cvmx_pko_port_map_o68(); + __cvmx_pko_queue_alloc_o68(); + return; + } + + /* + * Initialize queues + */ + for (i = 0; i < CVMX_PKO_MAX_OUTPUT_QUEUES; i++) { + const uint64_t priority = 8; + + cvmx_pko_config_port(CVMX_PKO_MEM_QUEUE_PTRS_ILLEGAL_PID, i, 1, + &priority); + } +} + /** * Call before any other calls to initialize the packet * output system. This does chip global config, and should only be @@ -47,8 +184,6 @@ void cvmx_pko_initialize_global(void) { - int i; - uint64_t priority = 8; union cvmx_pko_reg_cmd_buf config; /* @@ -62,9 +197,10 @@ void cvmx_pko_initialize_global(void) cvmx_write_csr(CVMX_PKO_REG_CMD_BUF, config.u64); - for (i = 0; i < CVMX_PKO_MAX_OUTPUT_QUEUES; i++) - cvmx_pko_config_port(CVMX_PKO_MEM_QUEUE_PTRS_ILLEGAL_PID, i, 1, - &priority); + /* + * Chip-specific setup. + */ + __cvmx_pko_chip_init(); /* * If we aren't using all of the queues optimize PKO's @@ -212,6 +348,9 @@ cvmx_pko_status_t cvmx_pko_config_port(uint64_t port, uint64_t base_queue, int static_priority_base = -1; int static_priority_end = -1; + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) + return CVMX_PKO_SUCCESS; + if ((port >= CVMX_PKO_NUM_OUTPUT_PORTS) && (port != CVMX_PKO_MEM_QUEUE_PTRS_ILLEGAL_PID)) { cvmx_dprintf("ERROR: cvmx_pko_config_port: Invalid port %llu\n", diff --git a/arch/mips/cavium-octeon/octeon-irq.c b/arch/mips/cavium-octeon/octeon-irq.c index d8124a3c5a8530..f26c3c661cca9e 100644 --- a/arch/mips/cavium-octeon/octeon-irq.c +++ b/arch/mips/cavium-octeon/octeon-irq.c @@ -225,13 +225,14 @@ static int next_cpu_for_irq(struct irq_data *data) #ifdef CONFIG_SMP int cpu; - int weight = cpumask_weight(data->affinity); + struct cpumask *mask = irq_data_get_affinity_mask(data); + int weight = cpumask_weight(mask); struct octeon_ciu_chip_data *cd = irq_data_get_irq_chip_data(data); if (weight > 1) { cpu = cd->current_cpu; for (;;) { - cpu = cpumask_next(cpu, data->affinity); + cpu = cpumask_next(cpu, mask); if (cpu >= nr_cpu_ids) { cpu = -1; continue; @@ -240,7 +241,7 @@ static int next_cpu_for_irq(struct irq_data *data) } } } else if (weight == 1) { - cpu = cpumask_first(data->affinity); + cpu = cpumask_first(mask); } else { cpu = smp_processor_id(); } @@ -662,6 +663,11 @@ static int octeon_irq_ciu_gpio_set_type(struct irq_data *data, unsigned int t) irqd_set_trigger_type(data, t); octeon_irq_gpio_setup(data); + if (irqd_get_trigger_type(data) & IRQ_TYPE_EDGE_BOTH) + irq_set_handler_locked(data, handle_edge_irq); + else + irq_set_handler_locked(data, handle_level_irq); + return IRQ_SET_MASK_OK; } @@ -696,32 +702,23 @@ static void octeon_irq_ciu_gpio_ack(struct irq_data *data) cvmx_write_csr(CVMX_GPIO_INT_CLR, mask); } -static void octeon_irq_handle_trigger(unsigned int irq, struct irq_desc *desc) -{ - struct irq_data *data = irq_desc_get_irq_data(desc); - - if (irqd_get_trigger_type(data) & IRQ_TYPE_EDGE_BOTH) - handle_edge_irq(irq, desc); - else - handle_level_irq(irq, desc); -} - #ifdef CONFIG_SMP static void octeon_irq_cpu_offline_ciu(struct irq_data *data) { int cpu = smp_processor_id(); cpumask_t new_affinity; + struct cpumask *mask = irq_data_get_affinity_mask(data); - if (!cpumask_test_cpu(cpu, data->affinity)) + if (!cpumask_test_cpu(cpu, mask)) return; - if (cpumask_weight(data->affinity) > 1) { + if (cpumask_weight(mask) > 1) { /* * It has multi CPU affinity, just remove this CPU * from the affinity set. */ - cpumask_copy(&new_affinity, data->affinity); + cpumask_copy(&new_affinity, mask); cpumask_clear_cpu(cpu, &new_affinity); } else { /* Otherwise, put it on lowest numbered online CPU. */ @@ -1227,8 +1224,13 @@ static int octeon_irq_gpio_map(struct irq_domain *d, octeon_irq_ciu_to_irq[line][bit] != 0) return -EINVAL; + /* + * Default to handle_level_irq. If the DT contains a different + * trigger type, it will call the irq_set_type callback and + * the handler gets updated. + */ r = octeon_irq_set_ciu_mapping(virq, line, bit, hw, - octeon_irq_gpio_chip, octeon_irq_handle_trigger); + octeon_irq_gpio_chip, handle_level_irq); return r; } diff --git a/arch/mips/include/asm/Kbuild b/arch/mips/include/asm/Kbuild index 1f8546081d2068..40ec4ca3f946a9 100644 --- a/arch/mips/include/asm/Kbuild +++ b/arch/mips/include/asm/Kbuild @@ -16,6 +16,5 @@ generic-y += sections.h generic-y += segment.h generic-y += serial.h generic-y += trace_clock.h -generic-y += ucontext.h generic-y += user.h generic-y += xor.h diff --git a/arch/mips/include/asm/abi.h b/arch/mips/include/asm/abi.h index 7186bb51b89b2a..37f84078e78abb 100644 --- a/arch/mips/include/asm/abi.h +++ b/arch/mips/include/asm/abi.h @@ -20,6 +20,10 @@ struct mips_abi { struct pt_regs *regs, sigset_t *set); const unsigned long rt_signal_return_offset; const unsigned long restart; + + unsigned off_sc_fpregs; + unsigned off_sc_fpc_csr; + unsigned off_sc_used_math; }; #endif /* _ASM_ABI_H */ diff --git a/arch/mips/include/asm/asmmacro.h b/arch/mips/include/asm/asmmacro.h index 76317a70200d18..867f924b05c795 100644 --- a/arch/mips/include/asm/asmmacro.h +++ b/arch/mips/include/asm/asmmacro.h @@ -232,6 +232,30 @@ .set pop .endm + .macro ld_b wd, off, base + .set push + .set mips32r2 + .set msa + ld.b $w\wd, \off(\base) + .set pop + .endm + + .macro ld_h wd, off, base + .set push + .set mips32r2 + .set msa + ld.h $w\wd, \off(\base) + .set pop + .endm + + .macro ld_w wd, off, base + .set push + .set mips32r2 + .set msa + ld.w $w\wd, \off(\base) + .set pop + .endm + .macro ld_d wd, off, base .set push .set mips32r2 @@ -241,6 +265,30 @@ .set pop .endm + .macro st_b wd, off, base + .set push + .set mips32r2 + .set msa + st.b $w\wd, \off(\base) + .set pop + .endm + + .macro st_h wd, off, base + .set push + .set mips32r2 + .set msa + st.h $w\wd, \off(\base) + .set pop + .endm + + .macro st_w wd, off, base + .set push + .set mips32r2 + .set msa + st.w $w\wd, \off(\base) + .set pop + .endm + .macro st_d wd, off, base .set push .set mips32r2 @@ -290,7 +338,13 @@ #ifdef CONFIG_CPU_MICROMIPS #define CFC_MSA_INSN 0x587e0056 #define CTC_MSA_INSN 0x583e0816 +#define LDB_MSA_INSN 0x58000807 +#define LDH_MSA_INSN 0x58000817 +#define LDW_MSA_INSN 0x58000827 #define LDD_MSA_INSN 0x58000837 +#define STB_MSA_INSN 0x5800080f +#define STH_MSA_INSN 0x5800081f +#define STW_MSA_INSN 0x5800082f #define STD_MSA_INSN 0x5800083f #define COPY_UW_MSA_INSN 0x58f00056 #define COPY_UD_MSA_INSN 0x58f80056 @@ -299,7 +353,13 @@ #else #define CFC_MSA_INSN 0x787e0059 #define CTC_MSA_INSN 0x783e0819 +#define LDB_MSA_INSN 0x78000820 +#define LDH_MSA_INSN 0x78000821 +#define LDW_MSA_INSN 0x78000822 #define LDD_MSA_INSN 0x78000823 +#define STB_MSA_INSN 0x78000824 +#define STH_MSA_INSN 0x78000825 +#define STW_MSA_INSN 0x78000826 #define STD_MSA_INSN 0x78000827 #define COPY_UW_MSA_INSN 0x78f00059 #define COPY_UD_MSA_INSN 0x78f80059 @@ -329,6 +389,33 @@ .set pop .endm + .macro ld_b wd, off, base + .set push + .set noat + SET_HARDFLOAT + addu $1, \base, \off + .word LDB_MSA_INSN | (\wd << 6) + .set pop + .endm + + .macro ld_h wd, off, base + .set push + .set noat + SET_HARDFLOAT + addu $1, \base, \off + .word LDH_MSA_INSN | (\wd << 6) + .set pop + .endm + + .macro ld_w wd, off, base + .set push + .set noat + SET_HARDFLOAT + addu $1, \base, \off + .word LDW_MSA_INSN | (\wd << 6) + .set pop + .endm + .macro ld_d wd, off, base .set push .set noat @@ -338,6 +425,33 @@ .set pop .endm + .macro st_b wd, off, base + .set push + .set noat + SET_HARDFLOAT + addu $1, \base, \off + .word STB_MSA_INSN | (\wd << 6) + .set pop + .endm + + .macro st_h wd, off, base + .set push + .set noat + SET_HARDFLOAT + addu $1, \base, \off + .word STH_MSA_INSN | (\wd << 6) + .set pop + .endm + + .macro st_w wd, off, base + .set push + .set noat + SET_HARDFLOAT + addu $1, \base, \off + .word STW_MSA_INSN | (\wd << 6) + .set pop + .endm + .macro st_d wd, off, base .set push .set noat diff --git a/arch/mips/include/asm/atomic.h b/arch/mips/include/asm/atomic.h index 26d436336f2e18..4c42fd9af77784 100644 --- a/arch/mips/include/asm/atomic.h +++ b/arch/mips/include/asm/atomic.h @@ -137,6 +137,10 @@ static __inline__ int atomic_##op##_return(int i, atomic_t * v) \ ATOMIC_OPS(add, +=, addu) ATOMIC_OPS(sub, -=, subu) +ATOMIC_OP(and, &=, and) +ATOMIC_OP(or, |=, or) +ATOMIC_OP(xor, ^=, xor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP @@ -416,6 +420,9 @@ static __inline__ long atomic64_##op##_return(long i, atomic64_t * v) \ ATOMIC64_OPS(add, +=, daddu) ATOMIC64_OPS(sub, -=, dsubu) +ATOMIC64_OP(and, &=, and) +ATOMIC64_OP(or, |=, or) +ATOMIC64_OP(xor, ^=, xor) #undef ATOMIC64_OPS #undef ATOMIC64_OP_RETURN diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h index 7ecba84656d495..752e0b86c17100 100644 --- a/arch/mips/include/asm/barrier.h +++ b/arch/mips/include/asm/barrier.h @@ -133,12 +133,12 @@ do { \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ ___p1; \ diff --git a/arch/mips/include/asm/cdmm.h b/arch/mips/include/asm/cdmm.h index 16e22ce9719fdf..bece2064cc8cbf 100644 --- a/arch/mips/include/asm/cdmm.h +++ b/arch/mips/include/asm/cdmm.h @@ -53,7 +53,7 @@ struct mips_cdmm_driver { * mips_cdmm_phys_base() - Choose a physical base address for CDMM region. * * Picking a suitable physical address at which to map the CDMM region is - * platform specific, so this weak function can be defined by platform code to + * platform specific, so this function can be defined by platform code to * pick a suitable value if none is configured by the bootloader. * * This address must be 32kB aligned, and the region occupies a maximum of 32kB @@ -61,7 +61,7 @@ struct mips_cdmm_driver { * * Returns: Physical base address for CDMM region, or 0 on failure. */ -phys_addr_t __weak mips_cdmm_phys_base(void); +phys_addr_t mips_cdmm_phys_base(void); extern struct bus_type mips_cdmm_bustype; void __iomem *mips_cdmm_early_probe(unsigned int dev_type); diff --git a/arch/mips/include/asm/cevt-r4k.h b/arch/mips/include/asm/cevt-r4k.h index f0edf6fcd002b9..2e13a038d2600d 100644 --- a/arch/mips/include/asm/cevt-r4k.h +++ b/arch/mips/include/asm/cevt-r4k.h @@ -21,7 +21,6 @@ DECLARE_PER_CPU(struct clock_event_device, mips_clockevent_device); void mips_event_handler(struct clock_event_device *dev); int c0_compare_int_usable(void); -void mips_set_clock_mode(enum clock_event_mode, struct clock_event_device *); irqreturn_t c0_compare_interrupt(int, void *); extern struct irqaction c0_compare_irqaction; diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h index f25de771f7ed22..9801ac9826554c 100644 --- a/arch/mips/include/asm/cpu-features.h +++ b/arch/mips/include/asm/cpu-features.h @@ -411,4 +411,8 @@ # define cpu_has_cdmm (cpu_data[0].options & MIPS_CPU_CDMM) #endif +#ifndef cpu_has_small_pages +# define cpu_has_small_pages (cpu_data[0].options & MIPS_CPU_SP) +#endif + #endif /* __ASM_CPU_FEATURES_H */ diff --git a/arch/mips/include/asm/cpu-type.h b/arch/mips/include/asm/cpu-type.h index d41e8e28482531..abee2bfd10dc1d 100644 --- a/arch/mips/include/asm/cpu-type.h +++ b/arch/mips/include/asm/cpu-type.h @@ -77,6 +77,10 @@ static inline int __pure __get_cpu_type(const int cpu_type) */ #endif +#ifdef CONFIG_SYS_HAS_CPU_MIPS64_R6 + case CPU_I6400: +#endif + #ifdef CONFIG_SYS_HAS_CPU_R3000 case CPU_R2000: case CPU_R3000: diff --git a/arch/mips/include/asm/cpu.h b/arch/mips/include/asm/cpu.h index e46e40602af3cc..cd89e985577527 100644 --- a/arch/mips/include/asm/cpu.h +++ b/arch/mips/include/asm/cpu.h @@ -120,6 +120,7 @@ #define PRID_IMP_PROAPTIV_MP 0xa300 #define PRID_IMP_M5150 0xa700 #define PRID_IMP_P5600 0xa800 +#define PRID_IMP_I6400 0xa900 /* * These are the PRID's for when 23:16 == PRID_COMP_SIBYTE @@ -307,6 +308,7 @@ enum cpu_type_enum { CPU_ALCHEMY, CPU_PR4450, CPU_BMIPS32, CPU_BMIPS3300, CPU_BMIPS4350, CPU_BMIPS4380, CPU_BMIPS5000, CPU_JZRISC, CPU_LOONGSON1, CPU_M14KC, CPU_M14KEC, CPU_INTERAPTIV, CPU_P5600, CPU_PROAPTIV, CPU_1074K, CPU_M5150, + CPU_I6400, /* * MIPS64 class processors @@ -382,6 +384,7 @@ enum cpu_type_enum { #define MIPS_CPU_XPA 0x2000000000ull /* CPU supports Extended Physical Addressing */ #define MIPS_CPU_CDMM 0x4000000000ull /* CPU has Common Device Memory Map */ #define MIPS_CPU_BP_GHIST 0x8000000000ull /* R12K+ Branch Prediction Global History */ +#define MIPS_CPU_SP 0x10000000000ull /* Small (1KB) page support */ /* * CPU ASE encodings diff --git a/arch/mips/include/asm/debug.h b/arch/mips/include/asm/debug.h deleted file mode 100644 index 1fd5a2b39445a9..00000000000000 --- a/arch/mips/include/asm/debug.h +++ /dev/null @@ -1,48 +0,0 @@ -/* - * Debug macros for run-time debugging. - * Turned on/off with CONFIG_RUNTIME_DEBUG option. - * - * Copyright (C) 2001 MontaVista Software Inc. - * Author: Jun Sun, jsun@mvista.com or jsun@junsun.net - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License as published by the - * Free Software Foundation; either version 2 of the License, or (at your - * option) any later version. - * - */ - -#ifndef _ASM_DEBUG_H -#define _ASM_DEBUG_H - - -/* - * run-time macros for catching spurious errors. Eable CONFIG_RUNTIME_DEBUG in - * kernel hacking config menu to use them. - * - * Use them as run-time debugging aid. NEVER USE THEM AS ERROR HANDLING CODE!!! - */ - -#ifdef CONFIG_RUNTIME_DEBUG - -#include - -#define db_assert(x) if (!(x)) { \ - panic("assertion failed at %s:%d: %s", __FILE__, __LINE__, #x); } -#define db_warn(x) if (!(x)) { \ - printk(KERN_WARNING "warning at %s:%d: %s", __FILE__, __LINE__, #x); } -#define db_verify(x, y) db_assert(x y) -#define db_verify_warn(x, y) db_warn(x y) -#define db_run(x) do { x; } while (0) - -#else - -#define db_assert(x) -#define db_warn(x) -#define db_verify(x, y) x -#define db_verify_warn(x, y) x -#define db_run(x) - -#endif - -#endif /* _ASM_DEBUG_H */ diff --git a/arch/mips/include/asm/elf.h b/arch/mips/include/asm/elf.h index f19e890b99d274..53b26933b12cea 100644 --- a/arch/mips/include/asm/elf.h +++ b/arch/mips/include/asm/elf.h @@ -382,7 +382,9 @@ do { \ instruction set this cpu supports. This could be done in userspace, but it's not easy, and we've already done it here. */ -#define ELF_HWCAP (0) +#define ELF_HWCAP (elf_hwcap) +extern unsigned int elf_hwcap; +#include /* * This yields a string that ld.so will use to load implementation diff --git a/arch/mips/include/asm/fpu.h b/arch/mips/include/asm/fpu.h index 1b062518983525..9cbf383b8834c7 100644 --- a/arch/mips/include/asm/fpu.h +++ b/arch/mips/include/asm/fpu.h @@ -164,25 +164,30 @@ static inline int own_fpu(int restore) return ret; } -static inline void lose_fpu(int save) +static inline void lose_fpu_inatomic(int save, struct task_struct *tsk) { - preempt_disable(); if (is_msa_enabled()) { if (save) { - save_msa(current); - current->thread.fpu.fcr31 = + save_msa(tsk); + tsk->thread.fpu.fcr31 = read_32bit_cp1_register(CP1_STATUS); } disable_msa(); - clear_thread_flag(TIF_USEDMSA); + clear_tsk_thread_flag(tsk, TIF_USEDMSA); __disable_fpu(); } else if (is_fpu_owner()) { if (save) - _save_fp(current); + _save_fp(tsk); __disable_fpu(); } - KSTK_STATUS(current) &= ~ST0_CU1; - clear_thread_flag(TIF_USEDFPU); + KSTK_STATUS(tsk) &= ~ST0_CU1; + clear_tsk_thread_flag(tsk, TIF_USEDFPU); +} + +static inline void lose_fpu(int save) +{ + preempt_disable(); + lose_fpu_inatomic(save, current); preempt_enable(); } diff --git a/arch/mips/include/asm/gpio.h b/arch/mips/include/asm/gpio.h deleted file mode 100644 index 06e46faf862d55..00000000000000 --- a/arch/mips/include/asm/gpio.h +++ /dev/null @@ -1,6 +0,0 @@ -#ifndef __ASM_MIPS_GPIO_H -#define __ASM_MIPS_GPIO_H - -#include - -#endif /* __ASM_MIPS_GPIO_H */ diff --git a/arch/mips/include/asm/irq.h b/arch/mips/include/asm/irq.h index f0db99f8defe9b..15e0fecbc300fd 100644 --- a/arch/mips/include/asm/irq.h +++ b/arch/mips/include/asm/irq.h @@ -49,7 +49,7 @@ extern int cp0_compare_irq_shift; extern int cp0_perfcount_irq; extern int cp0_fdc_irq; -extern int __weak get_c0_fdc_int(void); +extern int get_c0_fdc_int(void); void arch_trigger_all_cpu_backtrace(bool); #define arch_trigger_all_cpu_backtrace arch_trigger_all_cpu_backtrace diff --git a/arch/mips/include/asm/jump_label.h b/arch/mips/include/asm/jump_label.h index 608aa57799c819..e77672539e8ed8 100644 --- a/arch/mips/include/asm/jump_label.h +++ b/arch/mips/include/asm/jump_label.h @@ -26,14 +26,29 @@ #define NOP_INSN "nop" #endif -static __always_inline bool arch_static_branch(struct static_key *key) +static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { asm_volatile_goto("1:\t" NOP_INSN "\n\t" "nop\n\t" ".pushsection __jump_table, \"aw\"\n\t" WORD_INSN " 1b, %l[l_yes], %0\n\t" ".popsection\n\t" - : : "i" (key) : : l_yes); + : : "i" (&((char *)key)[branch]) : : l_yes); + + return false; +l_yes: + return true; +} + +static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch) +{ + asm_volatile_goto("1:\tj %l[l_yes]\n\t" + "nop\n\t" + ".pushsection __jump_table, \"aw\"\n\t" + WORD_INSN " 1b, %l[l_yes], %0\n\t" + ".popsection\n\t" + : : "i" (&((char *)key)[branch]) : : l_yes); + return false; l_yes: return true; diff --git a/arch/mips/include/asm/kdebug.h b/arch/mips/include/asm/kdebug.h index cba22ab7ad4d5f..8e3d08e739c18d 100644 --- a/arch/mips/include/asm/kdebug.h +++ b/arch/mips/include/asm/kdebug.h @@ -11,7 +11,9 @@ enum die_val { DIE_PAGE_FAULT, DIE_BREAK, DIE_SSTEPBP, - DIE_MSAFP + DIE_MSAFP, + DIE_UPROBE, + DIE_UPROBE_XOL, }; #endif /* _ASM_MIPS_KDEBUG_H */ diff --git a/arch/mips/include/asm/linkage.h b/arch/mips/include/asm/linkage.h index 2767dda9e30910..99651b0ea7c726 100644 --- a/arch/mips/include/asm/linkage.h +++ b/arch/mips/include/asm/linkage.h @@ -5,7 +5,6 @@ #include #endif -#define __weak __attribute__((weak)) #define cond_syscall(x) asm(".weak\t" #x "\n" #x "\t=\tsys_ni_syscall") #define SYSCALL_ALIAS(alias, name) \ asm ( #alias " = " #name "\n\t.globl " #alias) diff --git a/arch/mips/include/asm/maar.h b/arch/mips/include/asm/maar.h index 6c62b0f899c0fd..b02891f9caaf1d 100644 --- a/arch/mips/include/asm/maar.h +++ b/arch/mips/include/asm/maar.h @@ -26,7 +26,7 @@ * * Return: The number of MAAR pairs configured. */ -unsigned __weak platform_maar_init(unsigned num_pairs); +unsigned platform_maar_init(unsigned num_pairs); /** * write_maar_pair() - write to a pair of MAARs diff --git a/arch/mips/include/asm/mach-ar7/ar7.h b/arch/mips/include/asm/mach-ar7/ar7.h index a47ea0c852483a..468cbd61b90671 100644 --- a/arch/mips/include/asm/mach-ar7/ar7.h +++ b/arch/mips/include/asm/mach-ar7/ar7.h @@ -203,4 +203,8 @@ static inline void ar7_device_off(u32 bit) int __init ar7_gpio_init(void); void __init ar7_init_clocks(void); +/* Board specific GPIO functions */ +int ar7_gpio_enable(unsigned gpio); +int ar7_gpio_disable(unsigned gpio); + #endif /* __AR7_H__ */ diff --git a/arch/mips/include/asm/mach-ar7/gpio.h b/arch/mips/include/asm/mach-ar7/gpio.h deleted file mode 100644 index c177cd1eed2588..00000000000000 --- a/arch/mips/include/asm/mach-ar7/gpio.h +++ /dev/null @@ -1,41 +0,0 @@ -/* - * Copyright (C) 2007-2009 Florian Fainelli - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write to the Free Software - * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - */ - -#ifndef __AR7_GPIO_H__ -#define __AR7_GPIO_H__ - -#include - -#define AR7_GPIO_MAX 32 -#define TITAN_GPIO_MAX 51 -#define NR_BUILTIN_GPIO TITAN_GPIO_MAX - -#define gpio_to_irq(gpio) -1 - -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value - -#define gpio_cansleep __gpio_cansleep - -/* Board specific GPIO functions */ -int ar7_gpio_enable(unsigned gpio); -int ar7_gpio_disable(unsigned gpio); - -#include - -#endif diff --git a/arch/mips/include/asm/mach-ath25/gpio.h b/arch/mips/include/asm/mach-ath25/gpio.h deleted file mode 100644 index 713564b8e8ef95..00000000000000 --- a/arch/mips/include/asm/mach-ath25/gpio.h +++ /dev/null @@ -1,16 +0,0 @@ -#ifndef __ASM_MACH_ATH25_GPIO_H -#define __ASM_MACH_ATH25_GPIO_H - -#include - -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value -#define gpio_cansleep __gpio_cansleep -#define gpio_to_irq __gpio_to_irq - -static inline int irq_to_gpio(unsigned irq) -{ - return -EINVAL; -} - -#endif /* __ASM_MACH_ATH25_GPIO_H */ diff --git a/arch/mips/include/asm/mach-ath79/gpio.h b/arch/mips/include/asm/mach-ath79/gpio.h deleted file mode 100644 index 60dcb62785b451..00000000000000 --- a/arch/mips/include/asm/mach-ath79/gpio.h +++ /dev/null @@ -1,26 +0,0 @@ -/* - * Atheros AR71XX/AR724X/AR913X GPIO API definitions - * - * Copyright (C) 2008-2010 Gabor Juhos - * Copyright (C) 2008 Imre Kaloz - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License version 2 as published - * by the Free Software Foundation. - * - */ - -#ifndef __ASM_MACH_ATH79_GPIO_H -#define __ASM_MACH_ATH79_GPIO_H - -#define ARCH_NR_GPIOS 64 -#include - -int gpio_to_irq(unsigned gpio); -int irq_to_gpio(unsigned irq); -int gpio_get_value(unsigned gpio); -void gpio_set_value(unsigned gpio, int value); - -#define gpio_cansleep __gpio_cansleep - -#endif /* __ASM_MACH_ATH79_GPIO_H */ diff --git a/arch/mips/include/asm/mach-au1x00/gpio-au1000.h b/arch/mips/include/asm/mach-au1x00/gpio-au1000.h index 9785e4ebb450f6..adde1fa5097e6c 100644 --- a/arch/mips/include/asm/mach-au1x00/gpio-au1000.h +++ b/arch/mips/include/asm/mach-au1x00/gpio-au1000.h @@ -266,6 +266,17 @@ static inline int alchemy_gpio1_to_irq(int gpio) return -ENXIO; } +/* On Au1000, Au1500 and Au1100 GPIOs won't work as inputs before + * SYS_PININPUTEN is written to at least once. On Au1550/Au1200/Au1300 this + * register enables use of GPIOs as wake source. + */ +static inline void alchemy_gpio1_input_enable(void) +{ + void __iomem *base = (void __iomem *)KSEG1ADDR(AU1000_SYS_PHYS_ADDR); + __raw_writel(0, base + 0x110); /* the write op is key */ + wmb(); +} + /* * GPIO2 block macros for common linux GPIO functions. The 'gpio' * parameter must be in range of ALCHEMY_GPIO2_BASE..ALCHEMY_GPIO2_MAX. @@ -518,141 +529,4 @@ static inline int alchemy_irq_to_gpio(int irq) return -ENXIO; } -/**********************************************************************/ - -/* Linux gpio framework integration. - * - * 4 use cases of Au1000-Au1200 GPIOS: - *(1) GPIOLIB=y, ALCHEMY_GPIO_INDIRECT=y: - * Board must register gpiochips. - *(2) GPIOLIB=y, ALCHEMY_GPIO_INDIRECT=n: - * 2 (1 for Au1000) gpio_chips are registered. - * - *(3) GPIOLIB=n, ALCHEMY_GPIO_INDIRECT=y: - * the boards' gpio.h must provide the linux gpio wrapper functions, - * - *(4) GPIOLIB=n, ALCHEMY_GPIO_INDIRECT=n: - * inlinable gpio functions are provided which enable access to the - * Au1000 gpios only by using the numbers straight out of the data- - * sheets. - - * Cases 1 and 3 are intended for boards which want to provide their own - * GPIO namespace and -operations (i.e. for example you have 8 GPIOs - * which are in part provided by spare Au1000 GPIO pins and in part by - * an external FPGA but you still want them to be accssible in linux - * as gpio0-7. The board can of course use the alchemy_gpioX_* functions - * as required). - */ - -#ifndef CONFIG_GPIOLIB - -#ifdef CONFIG_ALCHEMY_GPIOINT_AU1000 - -#ifndef CONFIG_ALCHEMY_GPIO_INDIRECT /* case (4) */ - -static inline int gpio_direction_input(int gpio) -{ - return alchemy_gpio_direction_input(gpio); -} - -static inline int gpio_direction_output(int gpio, int v) -{ - return alchemy_gpio_direction_output(gpio, v); -} - -static inline int gpio_get_value(int gpio) -{ - return alchemy_gpio_get_value(gpio); -} - -static inline void gpio_set_value(int gpio, int v) -{ - alchemy_gpio_set_value(gpio, v); -} - -static inline int gpio_get_value_cansleep(unsigned gpio) -{ - return gpio_get_value(gpio); -} - -static inline void gpio_set_value_cansleep(unsigned gpio, int value) -{ - gpio_set_value(gpio, value); -} - -static inline int gpio_is_valid(int gpio) -{ - return alchemy_gpio_is_valid(gpio); -} - -static inline int gpio_cansleep(int gpio) -{ - return alchemy_gpio_cansleep(gpio); -} - -static inline int gpio_to_irq(int gpio) -{ - return alchemy_gpio_to_irq(gpio); -} - -static inline int irq_to_gpio(int irq) -{ - return alchemy_irq_to_gpio(irq); -} - -static inline int gpio_request(unsigned gpio, const char *label) -{ - return 0; -} - -static inline int gpio_request_one(unsigned gpio, - unsigned long flags, const char *label) -{ - return 0; -} - -static inline int gpio_request_array(struct gpio *array, size_t num) -{ - return 0; -} - -static inline void gpio_free(unsigned gpio) -{ -} - -static inline void gpio_free_array(struct gpio *array, size_t num) -{ -} - -static inline int gpio_set_debounce(unsigned gpio, unsigned debounce) -{ - return -ENOSYS; -} - -static inline int gpio_export(unsigned gpio, bool direction_may_change) -{ - return -ENOSYS; -} - -static inline int gpio_export_link(struct device *dev, const char *name, - unsigned gpio) -{ - return -ENOSYS; -} - -static inline int gpio_sysfs_set_active_low(unsigned gpio, int value) -{ - return -ENOSYS; -} - -static inline void gpio_unexport(unsigned gpio) -{ -} - -#endif /* !CONFIG_ALCHEMY_GPIO_INDIRECT */ - -#endif /* CONFIG_ALCHEMY_GPIOINT_AU1000 */ - -#endif /* !CONFIG_GPIOLIB */ - #endif /* _ALCHEMY_GPIO_AU1000_H_ */ diff --git a/arch/mips/include/asm/mach-au1x00/gpio.h b/arch/mips/include/asm/mach-au1x00/gpio.h deleted file mode 100644 index 22e7ff17fc4829..00000000000000 --- a/arch/mips/include/asm/mach-au1x00/gpio.h +++ /dev/null @@ -1,86 +0,0 @@ -/* - * Alchemy GPIO support. - * - * With CONFIG_GPIOLIB=y different types of on-chip GPIO can be supported within - * the same kernel image. - * With CONFIG_GPIOLIB=n, your board must select ALCHEMY_GPIOINT_AU1XXX for the - * appropriate CPU type (AU1000 currently). - */ - -#ifndef _ALCHEMY_GPIO_H_ -#define _ALCHEMY_GPIO_H_ - -#include -#include -#include - -/* On Au1000, Au1500 and Au1100 GPIOs won't work as inputs before - * SYS_PININPUTEN is written to at least once. On Au1550/Au1200/Au1300 this - * register enables use of GPIOs as wake source. - */ -static inline void alchemy_gpio1_input_enable(void) -{ - void __iomem *base = (void __iomem *)KSEG1ADDR(AU1000_SYS_PHYS_ADDR); - __raw_writel(0, base + 0x110); /* the write op is key */ - wmb(); -} - - -/* Linux gpio framework integration. -* -* 4 use cases of Alchemy GPIOS: -*(1) GPIOLIB=y, ALCHEMY_GPIO_INDIRECT=y: -* Board must register gpiochips. -*(2) GPIOLIB=y, ALCHEMY_GPIO_INDIRECT=n: -* A gpiochip for the 75 GPIOs is registered. -* -*(3) GPIOLIB=n, ALCHEMY_GPIO_INDIRECT=y: -* the boards' gpio.h must provide the linux gpio wrapper functions, -* -*(4) GPIOLIB=n, ALCHEMY_GPIO_INDIRECT=n: -* inlinable gpio functions are provided which enable access to the -* Au1300 gpios only by using the numbers straight out of the data- -* sheets. - -* Cases 1 and 3 are intended for boards which want to provide their own -* GPIO namespace and -operations (i.e. for example you have 8 GPIOs -* which are in part provided by spare Au1300 GPIO pins and in part by -* an external FPGA but you still want them to be accssible in linux -* as gpio0-7. The board can of course use the alchemy_gpioX_* functions -* as required). -*/ - -#ifdef CONFIG_GPIOLIB - -/* wraps the cpu-dependent irq_to_gpio functions */ -/* FIXME: gpiolib needs an irq_to_gpio hook */ -static inline int __au_irq_to_gpio(unsigned int irq) -{ - switch (alchemy_get_cputype()) { - case ALCHEMY_CPU_AU1000...ALCHEMY_CPU_AU1200: - return alchemy_irq_to_gpio(irq); - case ALCHEMY_CPU_AU1300: - return au1300_irq_to_gpio(irq); - } - return -EINVAL; -} - - -/* using gpiolib to provide up to 2 gpio_chips for on-chip gpios */ -#ifndef CONFIG_ALCHEMY_GPIO_INDIRECT /* case (2) */ - -/* get everything through gpiolib */ -#define gpio_to_irq __gpio_to_irq -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value -#define gpio_cansleep __gpio_cansleep -#define irq_to_gpio __au_irq_to_gpio - -#include - -#endif /* !CONFIG_ALCHEMY_GPIO_INDIRECT */ - - -#endif /* CONFIG_GPIOLIB */ - -#endif /* _ALCHEMY_GPIO_H_ */ diff --git a/arch/mips/include/asm/mach-bcm47xx/gpio.h b/arch/mips/include/asm/mach-bcm47xx/gpio.h deleted file mode 100644 index 90daefa24a4dc5..00000000000000 --- a/arch/mips/include/asm/mach-bcm47xx/gpio.h +++ /dev/null @@ -1,17 +0,0 @@ -#ifndef __ASM_MIPS_MACH_BCM47XX_GPIO_H -#define __ASM_MIPS_MACH_BCM47XX_GPIO_H - -#include - -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value - -#define gpio_cansleep __gpio_cansleep -#define gpio_to_irq __gpio_to_irq - -static inline int irq_to_gpio(unsigned int irq) -{ - return -EINVAL; -} - -#endif diff --git a/arch/mips/include/asm/mach-bcm63xx/gpio.h b/arch/mips/include/asm/mach-bcm63xx/gpio.h deleted file mode 100644 index 1eb534de8e3b1e..00000000000000 --- a/arch/mips/include/asm/mach-bcm63xx/gpio.h +++ /dev/null @@ -1,15 +0,0 @@ -#ifndef __ASM_MIPS_MACH_BCM63XX_GPIO_H -#define __ASM_MIPS_MACH_BCM63XX_GPIO_H - -#include - -#define gpio_to_irq(gpio) -1 - -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value - -#define gpio_cansleep __gpio_cansleep - -#include - -#endif /* __ASM_MIPS_MACH_BCM63XX_GPIO_H */ diff --git a/arch/mips/include/asm/mach-cavium-octeon/gpio.h b/arch/mips/include/asm/mach-cavium-octeon/gpio.h deleted file mode 100644 index 34e9f7aabab44c..00000000000000 --- a/arch/mips/include/asm/mach-cavium-octeon/gpio.h +++ /dev/null @@ -1,21 +0,0 @@ -#ifndef __ASM_MACH_CAVIUM_OCTEON_GPIO_H -#define __ASM_MACH_CAVIUM_OCTEON_GPIO_H - -#ifdef CONFIG_GPIOLIB -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value -#define gpio_cansleep __gpio_cansleep -#else -int gpio_request(unsigned gpio, const char *label); -void gpio_free(unsigned gpio); -int gpio_direction_input(unsigned gpio); -int gpio_direction_output(unsigned gpio, int value); -int gpio_get_value(unsigned gpio); -void gpio_set_value(unsigned gpio, int value); -#endif - -#include - -#define gpio_to_irq __gpio_to_irq - -#endif /* __ASM_MACH_GENERIC_GPIO_H */ diff --git a/arch/mips/include/asm/mach-generic/gpio.h b/arch/mips/include/asm/mach-generic/gpio.h deleted file mode 100644 index b4e70208da643b..00000000000000 --- a/arch/mips/include/asm/mach-generic/gpio.h +++ /dev/null @@ -1,21 +0,0 @@ -#ifndef __ASM_MACH_GENERIC_GPIO_H -#define __ASM_MACH_GENERIC_GPIO_H - -#ifdef CONFIG_GPIOLIB -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value -#define gpio_cansleep __gpio_cansleep -#else -int gpio_request(unsigned gpio, const char *label); -void gpio_free(unsigned gpio); -int gpio_direction_input(unsigned gpio); -int gpio_direction_output(unsigned gpio, int value); -int gpio_get_value(unsigned gpio); -void gpio_set_value(unsigned gpio, int value); -#endif -int gpio_to_irq(unsigned gpio); -int irq_to_gpio(unsigned irq); - -#include /* cansleep wrappers */ - -#endif /* __ASM_MACH_GENERIC_GPIO_H */ diff --git a/arch/mips/include/asm/mach-jz4740/gpio.h b/arch/mips/include/asm/mach-jz4740/gpio.h index eaacba79cf18c9..bf8c3e1860e713 100644 --- a/arch/mips/include/asm/mach-jz4740/gpio.h +++ b/arch/mips/include/asm/mach-jz4740/gpio.h @@ -73,8 +73,6 @@ int jz_gpio_port_direction_output(int port, uint32_t mask); void jz_gpio_port_set_value(int port, uint32_t value, uint32_t mask); uint32_t jz_gpio_port_get_value(int port, uint32_t mask); -#include - #define JZ_GPIO_PORTA(x) ((x) + 32 * 0) #define JZ_GPIO_PORTB(x) ((x) + 32 * 1) #define JZ_GPIO_PORTC(x) ((x) + 32 * 2) diff --git a/arch/mips/include/asm/mach-lantiq/gpio.h b/arch/mips/include/asm/mach-lantiq/gpio.h deleted file mode 100644 index 9ba1caebca5f5b..00000000000000 --- a/arch/mips/include/asm/mach-lantiq/gpio.h +++ /dev/null @@ -1,13 +0,0 @@ -#ifndef __ASM_MIPS_MACH_LANTIQ_GPIO_H -#define __ASM_MIPS_MACH_LANTIQ_GPIO_H - -#define gpio_to_irq __gpio_to_irq - -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value - -#define gpio_cansleep __gpio_cansleep - -#include - -#endif diff --git a/arch/mips/include/asm/mach-loongson64/gpio.h b/arch/mips/include/asm/mach-loongson64/gpio.h deleted file mode 100644 index b3b216904a9a8a..00000000000000 --- a/arch/mips/include/asm/mach-loongson64/gpio.h +++ /dev/null @@ -1,36 +0,0 @@ -/* - * Loongson GPIO Support - * - * Copyright (c) 2008 Richard Liu, STMicroelectronics - * Copyright (c) 2008-2010 Arnaud Patard - * Copyright (c) 2014 Huacai Chen - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - */ - -#ifndef __LOONGSON_GPIO_H -#define __LOONGSON_GPIO_H - -#include - -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value -#define gpio_cansleep __gpio_cansleep - -/* The chip can do interrupt - * but it has not been tested and doc not clear - */ -static inline int gpio_to_irq(int gpio) -{ - return -EINVAL; -} - -static inline int irq_to_gpio(int gpio) -{ - return -EINVAL; -} - -#endif /* __LOONGSON_GPIO_H */ diff --git a/arch/mips/include/asm/mach-pistachio/gpio.h b/arch/mips/include/asm/mach-pistachio/gpio.h deleted file mode 100644 index 6c1649c27b8d24..00000000000000 --- a/arch/mips/include/asm/mach-pistachio/gpio.h +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Pistachio IRQ setup - * - * Copyright (C) 2014 Google, Inc. - * - * This program is free software; you can redistribute it and/or modify it - * under the terms and conditions of the GNU General Public License, - * version 2, as published by the Free Software Foundation. - */ - -#ifndef __ASM_MACH_PISTACHIO_GPIO_H -#define __ASM_MACH_PISTACHIO_GPIO_H - -#include - -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value -#define gpio_cansleep __gpio_cansleep -#define gpio_to_irq __gpio_to_irq - -#endif /* __ASM_MACH_PISTACHIO_GPIO_H */ diff --git a/arch/mips/include/asm/mach-rc32434/gpio.h b/arch/mips/include/asm/mach-rc32434/gpio.h index 4dee0a34250c39..db211212ce79ef 100644 --- a/arch/mips/include/asm/mach-rc32434/gpio.h +++ b/arch/mips/include/asm/mach-rc32434/gpio.h @@ -13,18 +13,6 @@ #ifndef _RC32434_GPIO_H_ #define _RC32434_GPIO_H_ -#include -#include - -#define NR_BUILTIN_GPIO 32 - -#define gpio_get_value __gpio_get_value -#define gpio_set_value __gpio_set_value -#define gpio_cansleep __gpio_cansleep - -#define gpio_to_irq(gpio) (8 + 4 * 32 + gpio) -#define irq_to_gpio(irq) (irq - (8 + 4 * 32)) - struct rb532_gpio_reg { u32 gpiofunc; /* GPIO Function Register * gpiofunc[x]==0 bit = gpio diff --git a/arch/mips/include/asm/mips-cm.h b/arch/mips/include/asm/mips-cm.h index edc7ee95269e73..d75b75e78ebb47 100644 --- a/arch/mips/include/asm/mips-cm.h +++ b/arch/mips/include/asm/mips-cm.h @@ -33,6 +33,29 @@ extern void __iomem *mips_cm_l2sync_base; */ extern phys_addr_t __mips_cm_phys_base(void); +/* + * mips_cm_is64 - determine CM register width + * + * The CM register width is processor and CM specific. A 64-bit processor + * usually has a 64-bit CM and a 32-bit one has a 32-bit CM but a 64-bit + * processor could come with a 32-bit CM. Moreover, accesses on 64-bit CMs + * can be done either using regular 64-bit load/store instructions, or 32-bit + * load/store instruction on 32-bit register pairs. We opt for using 64-bit + * accesses on 64-bit CMs and kernels and 32-bit in any other case. + * + * It's set to 0 for 32-bit accesses and 1 for 64-bit accesses. + */ +extern int mips_cm_is64; + +/** + * mips_cm_error_report - Report CM cache errors + */ +#ifdef CONFIG_MIPS_CM +extern void mips_cm_error_report(void); +#else +static inline void mips_cm_error_report(void) {} +#endif + /** * mips_cm_probe - probe for a Coherence Manager * @@ -90,20 +113,46 @@ static inline bool mips_cm_has_l2sync(void) /* Macros to ease the creation of register access functions */ #define BUILD_CM_R_(name, off) \ -static inline u32 __iomem *addr_gcr_##name(void) \ +static inline unsigned long __iomem *addr_gcr_##name(void) \ { \ - return (u32 __iomem *)(mips_cm_base + (off)); \ + return (unsigned long __iomem *)(mips_cm_base + (off)); \ } \ \ -static inline u32 read_gcr_##name(void) \ +static inline u32 read32_gcr_##name(void) \ { \ return __raw_readl(addr_gcr_##name()); \ +} \ + \ +static inline u64 read64_gcr_##name(void) \ +{ \ + return __raw_readq(addr_gcr_##name()); \ +} \ + \ +static inline unsigned long read_gcr_##name(void) \ +{ \ + if (mips_cm_is64) \ + return read64_gcr_##name(); \ + else \ + return read32_gcr_##name(); \ } #define BUILD_CM__W(name, off) \ -static inline void write_gcr_##name(u32 value) \ +static inline void write32_gcr_##name(u32 value) \ { \ __raw_writel(value, addr_gcr_##name()); \ +} \ + \ +static inline void write64_gcr_##name(u64 value) \ +{ \ + __raw_writeq(value, addr_gcr_##name()); \ +} \ + \ +static inline void write_gcr_##name(unsigned long value) \ +{ \ + if (mips_cm_is64) \ + write64_gcr_##name(value); \ + else \ + write32_gcr_##name(value); \ } #define BUILD_CM_RW(name, off) \ @@ -144,6 +193,7 @@ BUILD_CM_RW(reg3_base, MIPS_CM_GCB_OFS + 0xc0) BUILD_CM_RW(reg3_mask, MIPS_CM_GCB_OFS + 0xc8) BUILD_CM_R_(gic_status, MIPS_CM_GCB_OFS + 0xd0) BUILD_CM_R_(cpc_status, MIPS_CM_GCB_OFS + 0xf0) +BUILD_CM_RW(l2_config, MIPS_CM_GCB_OFS + 0x130) /* Core Local & Core Other register accessor functions */ BUILD_CM_Cx_RW(reset_release, 0x00) @@ -189,6 +239,13 @@ BUILD_CM_Cx_R_(tcid_8_priority, 0x80) #define CM_GCR_REV_MINOR_SHF 0 #define CM_GCR_REV_MINOR_MSK (_ULCAST_(0xff) << 0) +#define CM_ENCODE_REV(major, minor) \ + (((major) << CM_GCR_REV_MAJOR_SHF) | \ + ((minor) << CM_GCR_REV_MINOR_SHF)) + +#define CM_REV_CM2 CM_ENCODE_REV(6, 0) +#define CM_REV_CM3 CM_ENCODE_REV(8, 0) + /* GCR_ERROR_CAUSE register fields */ #define CM_GCR_ERROR_CAUSE_ERRTYPE_SHF 27 #define CM_GCR_ERROR_CAUSE_ERRTYPE_MSK (_ULCAST_(0x1f) << 27) @@ -249,6 +306,16 @@ BUILD_CM_Cx_R_(tcid_8_priority, 0x80) #define CM_GCR_CPC_STATUS_EX_SHF 0 #define CM_GCR_CPC_STATUS_EX_MSK (_ULCAST_(0x1) << 0) +/* GCR_L2_CONFIG register fields */ +#define CM_GCR_L2_CONFIG_BYPASS_SHF 20 +#define CM_GCR_L2_CONFIG_BYPASS_MSK (_ULCAST_(0x1) << 20) +#define CM_GCR_L2_CONFIG_SET_SIZE_SHF 12 +#define CM_GCR_L2_CONFIG_SET_SIZE_MSK (_ULCAST_(0xf) << 12) +#define CM_GCR_L2_CONFIG_LINE_SIZE_SHF 8 +#define CM_GCR_L2_CONFIG_LINE_SIZE_MSK (_ULCAST_(0xf) << 8) +#define CM_GCR_L2_CONFIG_ASSOC_SHF 0 +#define CM_GCR_L2_CONFIG_ASSOC_MSK (_ULCAST_(0xff) << 0) + /* GCR_Cx_COHERENCE register fields */ #define CM_GCR_Cx_COHERENCE_COHDOMAINEN_SHF 0 #define CM_GCR_Cx_COHERENCE_COHDOMAINEN_MSK (_ULCAST_(0xff) << 0) @@ -324,4 +391,18 @@ static inline int mips_cm_l2sync(void) return 0; } +/** + * mips_cm_revision() - return CM revision + * + * Return: The revision of the CM, from GCR_REV, or 0 if no CM is present. The + * return value should be checked against the CM_REV_* macros. + */ +static inline int mips_cm_revision(void) +{ + if (!mips_cm_present()) + return 0; + + return read_gcr_rev(); +} + #endif /* __MIPS_ASM_MIPS_CM_H__ */ diff --git a/arch/mips/include/asm/mips-cpc.h b/arch/mips/include/asm/mips-cpc.h index 1cebe8c790513e..f386f32702f17f 100644 --- a/arch/mips/include/asm/mips-cpc.h +++ b/arch/mips/include/asm/mips-cpc.h @@ -27,16 +27,6 @@ extern void __iomem *mips_cpc_base; */ extern phys_addr_t mips_cpc_default_phys_base(void); -/** - * mips_cpc_phys_base - retrieve the physical base address of the CPC - * - * This function returns the physical base address of the Cluster Power - * Controller memory mapped registers, or 0 if no Cluster Power Controller - * is present. It may be overriden by individual platforms which determine - * this address in a different way. - */ -extern phys_addr_t __weak mips_cpc_phys_base(void); - /** * mips_cpc_probe - probe for a Cluster Power Controller * diff --git a/arch/mips/include/asm/mipsregs.h b/arch/mips/include/asm/mipsregs.h index c5b0956a853027..d3cd8eac81e3a7 100644 --- a/arch/mips/include/asm/mipsregs.h +++ b/arch/mips/include/asm/mipsregs.h @@ -112,6 +112,30 @@ #define CP0_TX39_CACHE $7 +/* Generic EntryLo bit definitions */ +#define ENTRYLO_G (_ULCAST_(1) << 0) +#define ENTRYLO_V (_ULCAST_(1) << 1) +#define ENTRYLO_D (_ULCAST_(1) << 2) +#define ENTRYLO_C_SHIFT 3 +#define ENTRYLO_C (_ULCAST_(7) << ENTRYLO_C_SHIFT) + +/* R3000 EntryLo bit definitions */ +#define R3K_ENTRYLO_G (_ULCAST_(1) << 8) +#define R3K_ENTRYLO_V (_ULCAST_(1) << 9) +#define R3K_ENTRYLO_D (_ULCAST_(1) << 10) +#define R3K_ENTRYLO_N (_ULCAST_(1) << 11) + +/* MIPS32/64 EntryLo bit definitions */ +#ifdef CONFIG_64BIT +/* as read by dmfc0 */ +#define MIPS_ENTRYLO_XI (_ULCAST_(1) << 62) +#define MIPS_ENTRYLO_RI (_ULCAST_(1) << 63) +#else +/* as read by mfc0 */ +#define MIPS_ENTRYLO_XI (_ULCAST_(1) << 30) +#define MIPS_ENTRYLO_RI (_ULCAST_(1) << 31) +#endif + /* * Values for PageMask register */ @@ -203,6 +227,9 @@ #define PG_ESP (_ULCAST_(1) << 28) #define PG_IEC (_ULCAST_(1) << 27) +/* MIPS32/64 EntryHI bit definitions */ +#define MIPS_ENTRYHI_EHINV (_ULCAST_(1) << 10) + /* * R4x00 interrupt enable / cause bits */ @@ -579,6 +606,8 @@ #define MIPS_CONF7_IAR (_ULCAST_(1) << 10) #define MIPS_CONF7_AR (_ULCAST_(1) << 16) +/* FTLB probability bits for R6 */ +#define MIPS_CONF7_FTLBP_SHIFT (18) /* MAAR bit definitions */ #define MIPS_MAAR_ADDR ((BIT_ULL(BITS_PER_LONG - 12) - 1) << 12) @@ -586,31 +615,6 @@ #define MIPS_MAAR_S (_ULCAST_(1) << 1) #define MIPS_MAAR_V (_ULCAST_(1) << 0) -/* EntryHI bit definition */ -#define MIPS_ENTRYHI_EHINV (_ULCAST_(1) << 10) - -/* R3000 EntryLo bit definitions */ -#define R3K_ENTRYLO_G (_ULCAST_(1) << 8) -#define R3K_ENTRYLO_V (_ULCAST_(1) << 9) -#define R3K_ENTRYLO_D (_ULCAST_(1) << 10) -#define R3K_ENTRYLO_N (_ULCAST_(1) << 11) - -/* R4000 compatible EntryLo bit definitions */ -#define MIPS_ENTRYLO_G (_ULCAST_(1) << 0) -#define MIPS_ENTRYLO_V (_ULCAST_(1) << 1) -#define MIPS_ENTRYLO_D (_ULCAST_(1) << 2) -#define MIPS_ENTRYLO_C_SHIFT 3 -#define MIPS_ENTRYLO_C (_ULCAST_(7) << MIPS_ENTRYLO_C_SHIFT) -#ifdef CONFIG_64BIT -/* as read by dmfc0 */ -#define MIPS_ENTRYLO_XI (_ULCAST_(1) << 62) -#define MIPS_ENTRYLO_RI (_ULCAST_(1) << 63) -#else -/* as read by mfc0 */ -#define MIPS_ENTRYLO_XI (_ULCAST_(1) << 30) -#define MIPS_ENTRYLO_RI (_ULCAST_(1) << 31) -#endif - /* CMGCRBase bit definitions */ #define MIPS_CMGCRB_BASE 11 #define MIPS_CMGCRF_BASE (~_ULCAST_((1 << MIPS_CMGCRB_BASE) - 1)) @@ -932,7 +936,7 @@ do { \ */ #define __read_32bit_c0_register(source, sel) \ -({ int __res; \ +({ unsigned int __res; \ if (sel == 0) \ __asm__ __volatile__( \ "mfc0\t%0, " #source "\n\t" \ @@ -1014,7 +1018,7 @@ do { \ * On RM7000/RM9000 these are uses to access cop0 set 1 registers */ #define __read_32bit_c0_ctrl_register(source) \ -({ int __res; \ +({ unsigned int __res; \ __asm__ __volatile__( \ "cfc0\t%0, " #source "\n\t" \ : "=r" (__res)); \ @@ -1471,7 +1475,7 @@ do { \ */ #define _read_32bit_cp1_register(source, gas_hardfloat) \ ({ \ - int __res; \ + unsigned int __res; \ \ __asm__ __volatile__( \ " .set push \n" \ diff --git a/arch/mips/include/asm/msa.h b/arch/mips/include/asm/msa.h index af5638b12c7567..bbb85fe21642f0 100644 --- a/arch/mips/include/asm/msa.h +++ b/arch/mips/include/asm/msa.h @@ -14,10 +14,90 @@ #ifndef __ASSEMBLY__ +#include + extern void _save_msa(struct task_struct *); extern void _restore_msa(struct task_struct *); extern void _init_msa_upper(void); +extern void read_msa_wr_b(unsigned idx, union fpureg *to); +extern void read_msa_wr_h(unsigned idx, union fpureg *to); +extern void read_msa_wr_w(unsigned idx, union fpureg *to); +extern void read_msa_wr_d(unsigned idx, union fpureg *to); + +/** + * read_msa_wr() - Read a single MSA vector register + * @idx: The index of the vector register to read + * @to: The FPU register union to store the registers value in + * @fmt: The format of the data in the vector register + * + * Read the value of MSA vector register idx into the FPU register + * union to, using the format fmt. + */ +static inline void read_msa_wr(unsigned idx, union fpureg *to, + enum msa_2b_fmt fmt) +{ + switch (fmt) { + case msa_fmt_b: + read_msa_wr_b(idx, to); + break; + + case msa_fmt_h: + read_msa_wr_h(idx, to); + break; + + case msa_fmt_w: + read_msa_wr_w(idx, to); + break; + + case msa_fmt_d: + read_msa_wr_d(idx, to); + break; + + default: + BUG(); + } +} + +extern void write_msa_wr_b(unsigned idx, union fpureg *from); +extern void write_msa_wr_h(unsigned idx, union fpureg *from); +extern void write_msa_wr_w(unsigned idx, union fpureg *from); +extern void write_msa_wr_d(unsigned idx, union fpureg *from); + +/** + * write_msa_wr() - Write a single MSA vector register + * @idx: The index of the vector register to write + * @from: The FPU register union to take the registers value from + * @fmt: The format of the data in the vector register + * + * Write the value from the FPU register union from into MSA vector + * register idx, using the format fmt. + */ +static inline void write_msa_wr(unsigned idx, union fpureg *from, + enum msa_2b_fmt fmt) +{ + switch (fmt) { + case msa_fmt_b: + write_msa_wr_b(idx, from); + break; + + case msa_fmt_h: + write_msa_wr_h(idx, from); + break; + + case msa_fmt_w: + write_msa_wr_w(idx, from); + break; + + case msa_fmt_d: + write_msa_wr_d(idx, from); + break; + + default: + BUG(); + } +} + static inline void enable_msa(void) { if (cpu_has_msa) { diff --git a/arch/mips/include/asm/octeon/cvmx-bootinfo.h b/arch/mips/include/asm/octeon/cvmx-bootinfo.h index c373d95b5e2cfa..d92cf59bdae634 100644 --- a/arch/mips/include/asm/octeon/cvmx-bootinfo.h +++ b/arch/mips/include/asm/octeon/cvmx-bootinfo.h @@ -284,6 +284,7 @@ enum cvmx_board_types_enum { CVMX_BOARD_TYPE_CUST_PRIVATE_MIN = 20001, CVMX_BOARD_TYPE_UBNT_E100 = 20002, CVMX_BOARD_TYPE_CUST_DSR1000N = 20006, + CVMX_BOARD_TYPE_KONTRON_S1901 = 21901, CVMX_BOARD_TYPE_CUST_PRIVATE_MAX = 30000, /* The remaining range is reserved for future use. */ @@ -384,6 +385,7 @@ static inline const char *cvmx_board_type_to_string(enum ENUM_BRD_TYPE_CASE(CVMX_BOARD_TYPE_CUST_PRIVATE_MIN) ENUM_BRD_TYPE_CASE(CVMX_BOARD_TYPE_UBNT_E100) ENUM_BRD_TYPE_CASE(CVMX_BOARD_TYPE_CUST_DSR1000N) + ENUM_BRD_TYPE_CASE(CVMX_BOARD_TYPE_KONTRON_S1901) ENUM_BRD_TYPE_CASE(CVMX_BOARD_TYPE_CUST_PRIVATE_MAX) } return "Unsupported Board"; diff --git a/arch/mips/include/asm/octeon/cvmx-pip.h b/arch/mips/include/asm/octeon/cvmx-pip.h index df69bfd2b006bc..c210154ad941fa 100644 --- a/arch/mips/include/asm/octeon/cvmx-pip.h +++ b/arch/mips/include/asm/octeon/cvmx-pip.h @@ -37,7 +37,7 @@ #include #include -#define CVMX_PIP_NUM_INPUT_PORTS 40 +#define CVMX_PIP_NUM_INPUT_PORTS 48 #define CVMX_PIP_NUM_WATCHERS 4 /* diff --git a/arch/mips/include/asm/octeon/cvmx-pko.h b/arch/mips/include/asm/octeon/cvmx-pko.h index 3da59bb8ce24fb..5f47f76ed510a5 100644 --- a/arch/mips/include/asm/octeon/cvmx-pko.h +++ b/arch/mips/include/asm/octeon/cvmx-pko.h @@ -542,6 +542,9 @@ static inline int cvmx_pko_get_base_queue_per_core(int port, int core) */ static inline int cvmx_pko_get_base_queue(int port) { + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) + return port; + return cvmx_pko_get_base_queue_per_core(port, 0); } diff --git a/arch/mips/include/asm/octeon/cvmx-pow-defs.h b/arch/mips/include/asm/octeon/cvmx-pow-defs.h index 9020ef443736d4..6a3db4b068ffc2 100644 --- a/arch/mips/include/asm/octeon/cvmx-pow-defs.h +++ b/arch/mips/include/asm/octeon/cvmx-pow-defs.h @@ -52,6 +52,12 @@ #define CVMX_POW_WQ_INT_THRX(offset) (CVMX_ADD_IO_SEG(0x0001670000000080ull) + ((offset) & 15) * 8) #define CVMX_POW_WS_PCX(offset) (CVMX_ADD_IO_SEG(0x0001670000000280ull) + ((offset) & 15) * 8) +#define CVMX_SSO_WQ_INT (CVMX_ADD_IO_SEG(0x0001670000001000ull)) +#define CVMX_SSO_WQ_IQ_DIS (CVMX_ADD_IO_SEG(0x0001670000001010ull)) +#define CVMX_SSO_WQ_INT_PC (CVMX_ADD_IO_SEG(0x0001670000001020ull)) +#define CVMX_SSO_PPX_GRP_MSK(offset) (CVMX_ADD_IO_SEG(0x0001670000006000ull) + ((offset) & 31) * 8) +#define CVMX_SSO_WQ_INT_THRX(offset) (CVMX_ADD_IO_SEG(0x0001670000007000ull) + ((offset) & 63) * 8) + union cvmx_pow_bist_stat { uint64_t u64; struct cvmx_pow_bist_stat_s { @@ -1286,4 +1292,27 @@ union cvmx_pow_ws_pcx { struct cvmx_pow_ws_pcx_s cnf71xx; }; +union cvmx_sso_wq_int_thrx { + uint64_t u64; + struct { +#ifdef __BIG_ENDIAN_BITFIELD + uint64_t reserved_33_63:31; + uint64_t tc_en:1; + uint64_t tc_thr:4; + uint64_t reserved_26_27:2; + uint64_t ds_thr:12; + uint64_t reserved_12_13:2; + uint64_t iq_thr:12; +#else + uint64_t iq_thr:12; + uint64_t reserved_12_13:2; + uint64_t ds_thr:12; + uint64_t reserved_26_27:2; + uint64_t tc_thr:4; + uint64_t tc_en:1; + uint64_t reserved_33_63:31; +#endif + } s; +}; + #endif diff --git a/arch/mips/include/asm/octeon/cvmx-pow.h b/arch/mips/include/asm/octeon/cvmx-pow.h index d5565d758dddd4..51531563f8dc91 100644 --- a/arch/mips/include/asm/octeon/cvmx-pow.h +++ b/arch/mips/include/asm/octeon/cvmx-pow.h @@ -1810,10 +1810,11 @@ static inline void cvmx_pow_work_submit(cvmx_wqe_t *wqp, uint32_t tag, cvmx_addr_t ptr; cvmx_pow_tag_req_t tag_req; - wqp->qos = qos; - wqp->tag = tag; - wqp->tag_type = tag_type; - wqp->grp = grp; + wqp->word1.tag = tag; + wqp->word1.tag_type = tag_type; + + cvmx_wqe_set_qos(wqp, qos); + cvmx_wqe_set_grp(wqp, grp); tag_req.u64 = 0; tag_req.s.op = CVMX_POW_TAG_OP_ADDWQ; diff --git a/arch/mips/include/asm/octeon/cvmx-wqe.h b/arch/mips/include/asm/octeon/cvmx-wqe.h index 2d6d0c7127a755..0d697aa786d4c8 100644 --- a/arch/mips/include/asm/octeon/cvmx-wqe.h +++ b/arch/mips/include/asm/octeon/cvmx-wqe.h @@ -193,6 +193,53 @@ typedef union { uint64_t bufs:8; #endif } s; + struct { +#ifdef __BIG_ENDIAN_BITFIELD + uint64_t bufs:8; + uint64_t ip_offset:8; + uint64_t vlan_valid:1; + uint64_t vlan_stacked:1; + uint64_t unassigned:1; + uint64_t vlan_cfi:1; + uint64_t vlan_id:12; + uint64_t port:12; /* MAC/PIP port number. */ + uint64_t dec_ipcomp:1; + uint64_t tcp_or_udp:1; + uint64_t dec_ipsec:1; + uint64_t is_v6:1; + uint64_t software:1; + uint64_t L4_error:1; + uint64_t is_frag:1; + uint64_t IP_exc:1; + uint64_t is_bcast:1; + uint64_t is_mcast:1; + uint64_t not_IP:1; + uint64_t rcv_error:1; + uint64_t err_code:8; +#else + uint64_t err_code:8; + uint64_t rcv_error:1; + uint64_t not_IP:1; + uint64_t is_mcast:1; + uint64_t is_bcast:1; + uint64_t IP_exc:1; + uint64_t is_frag:1; + uint64_t L4_error:1; + uint64_t software:1; + uint64_t is_v6:1; + uint64_t dec_ipsec:1; + uint64_t tcp_or_udp:1; + uint64_t dec_ipcomp:1; + uint64_t port:12; + uint64_t vlan_id:12; + uint64_t vlan_cfi:1; + uint64_t unassigned:1; + uint64_t vlan_stacked:1; + uint64_t vlan_valid:1; + uint64_t ip_offset:8; + uint64_t bufs:8; +#endif + } s_cn68xx; /* use this to get at the 16 vlan bits */ struct { @@ -355,6 +402,146 @@ typedef union { } cvmx_pip_wqe_word2; +union cvmx_pip_wqe_word0 { + struct { +#ifdef __BIG_ENDIAN_BITFIELD + /** + * raw chksum result generated by the HW + */ + uint16_t hw_chksum; + /** + * Field unused by hardware - available for software + */ + uint8_t unused; + /** + * Next pointer used by hardware for list maintenance. + * May be written/read by HW before the work queue + * entry is scheduled to a PP (Only 36 bits used in + * Octeon 1) + */ + uint64_t next_ptr:40; +#else + uint64_t next_ptr:40; + uint8_t unused; + uint16_t hw_chksum; +#endif + } cn38xx; + struct { +#ifdef __BIG_ENDIAN_BITFIELD + uint64_t l4ptr:8; /* 56..63 */ + uint64_t unused0:8; /* 48..55 */ + uint64_t l3ptr:8; /* 40..47 */ + uint64_t l2ptr:8; /* 32..39 */ + uint64_t unused1:18; /* 14..31 */ + uint64_t bpid:6; /* 8..13 */ + uint64_t unused2:2; /* 6..7 */ + uint64_t pknd:6; /* 0..5 */ +#else + uint64_t pknd:6; /* 0..5 */ + uint64_t unused2:2; /* 6..7 */ + uint64_t bpid:6; /* 8..13 */ + uint64_t unused1:18; /* 14..31 */ + uint64_t l2ptr:8; /* 32..39 */ + uint64_t l3ptr:8; /* 40..47 */ + uint64_t unused0:8; /* 48..55 */ + uint64_t l4ptr:8; /* 56..63 */ +#endif + } cn68xx; +}; + +union cvmx_wqe_word0 { + uint64_t u64; + union cvmx_pip_wqe_word0 pip; +}; + +union cvmx_wqe_word1 { + uint64_t u64; + struct { +#ifdef __BIG_ENDIAN_BITFIELD + uint64_t len:16; + uint64_t varies:14; + /** + * the type of the tag (ORDERED, ATOMIC, NULL) + */ + uint64_t tag_type:2; + uint64_t tag:32; +#else + uint64_t tag:32; + uint64_t tag_type:2; + uint64_t varies:14; + uint64_t len:16; +#endif + }; + struct { +#ifdef __BIG_ENDIAN_BITFIELD + uint64_t len:16; + uint64_t zero_0:1; + /** + * HW sets this to what it thought the priority of + * the input packet was + */ + uint64_t qos:3; + + uint64_t zero_1:1; + /** + * the group that the work queue entry will be scheduled to + */ + uint64_t grp:6; + uint64_t zero_2:3; + uint64_t tag_type:2; + uint64_t tag:32; +#else + uint64_t tag:32; + uint64_t tag_type:2; + uint64_t zero_2:3; + uint64_t grp:6; + uint64_t zero_1:1; + uint64_t qos:3; + uint64_t zero_0:1; + uint64_t len:16; +#endif + } cn68xx; + struct { +#ifdef __BIG_ENDIAN_BITFIELD + /** + * HW sets to the total number of bytes in the packet + */ + uint64_t len:16; + /** + * HW sets this to input physical port + */ + uint64_t ipprt:6; + + /** + * HW sets this to what it thought the priority of + * the input packet was + */ + uint64_t qos:3; + + /** + * the group that the work queue entry will be scheduled to + */ + uint64_t grp:4; + /** + * the type of the tag (ORDERED, ATOMIC, NULL) + */ + uint64_t tag_type:3; + /** + * the synchronization/ordering tag + */ + uint64_t tag:32; +#else + uint64_t tag:32; + uint64_t tag_type:2; + uint64_t zero_2:1; + uint64_t grp:4; + uint64_t qos:3; + uint64_t ipprt:6; + uint64_t len:16; +#endif + } cn38xx; +}; + /** * Work queue entry format * @@ -366,70 +553,13 @@ typedef struct { * WORD 0 * HW WRITE: the following 64 bits are filled by HW when a packet arrives */ - -#ifdef __BIG_ENDIAN_BITFIELD - /** - * raw chksum result generated by the HW - */ - uint16_t hw_chksum; - /** - * Field unused by hardware - available for software - */ - uint8_t unused; - /** - * Next pointer used by hardware for list maintenance. - * May be written/read by HW before the work queue - * entry is scheduled to a PP - * (Only 36 bits used in Octeon 1) - */ - uint64_t next_ptr:40; -#else - uint64_t next_ptr:40; - uint8_t unused; - uint16_t hw_chksum; -#endif + union cvmx_wqe_word0 word0; /***************************************************************** * WORD 1 * HW WRITE: the following 64 bits are filled by HW when a packet arrives */ - -#ifdef __BIG_ENDIAN_BITFIELD - /** - * HW sets to the total number of bytes in the packet - */ - uint64_t len:16; - /** - * HW sets this to input physical port - */ - uint64_t ipprt:6; - - /** - * HW sets this to what it thought the priority of the input packet was - */ - uint64_t qos:3; - - /** - * the group that the work queue entry will be scheduled to - */ - uint64_t grp:4; - /** - * the type of the tag (ORDERED, ATOMIC, NULL) - */ - uint64_t tag_type:3; - /** - * the synchronization/ordering tag - */ - uint64_t tag:32; -#else - uint64_t tag:32; - uint64_t tag_type:2; - uint64_t zero_2:1; - uint64_t grp:4; - uint64_t qos:3; - uint64_t ipprt:6; - uint64_t len:16; -#endif + union cvmx_wqe_word1 word1; /** * WORD 2 HW WRITE: the following 64-bits are filled in by @@ -465,4 +595,64 @@ typedef struct { } CVMX_CACHE_LINE_ALIGNED cvmx_wqe_t; +static inline int cvmx_wqe_get_port(cvmx_wqe_t *work) +{ + int port; + + if (octeon_has_feature(OCTEON_FEATURE_CN68XX_WQE)) + port = work->word2.s_cn68xx.port; + else + port = work->word1.cn38xx.ipprt; + + return port; +} + +static inline void cvmx_wqe_set_port(cvmx_wqe_t *work, int port) +{ + if (octeon_has_feature(OCTEON_FEATURE_CN68XX_WQE)) + work->word2.s_cn68xx.port = port; + else + work->word1.cn38xx.ipprt = port; +} + +static inline int cvmx_wqe_get_grp(cvmx_wqe_t *work) +{ + int grp; + + if (octeon_has_feature(OCTEON_FEATURE_CN68XX_WQE)) + grp = work->word1.cn68xx.grp; + else + grp = work->word1.cn38xx.grp; + + return grp; +} + +static inline void cvmx_wqe_set_grp(cvmx_wqe_t *work, int grp) +{ + if (octeon_has_feature(OCTEON_FEATURE_CN68XX_WQE)) + work->word1.cn68xx.grp = grp; + else + work->word1.cn38xx.grp = grp; +} + +static inline int cvmx_wqe_get_qos(cvmx_wqe_t *work) +{ + int qos; + + if (octeon_has_feature(OCTEON_FEATURE_CN68XX_WQE)) + qos = work->word1.cn68xx.qos; + else + qos = work->word1.cn38xx.qos; + + return qos; +} + +static inline void cvmx_wqe_set_qos(cvmx_wqe_t *work, int qos) +{ + if (octeon_has_feature(OCTEON_FEATURE_CN68XX_WQE)) + work->word1.cn68xx.qos = qos; + else + work->word1.cn38xx.qos = qos; +} + #endif /* __CVMX_WQE_H__ */ diff --git a/arch/mips/include/asm/pgtable-bits.h b/arch/mips/include/asm/pgtable-bits.h index c28a8499aec7f4..ff7ad91c85db32 100644 --- a/arch/mips/include/asm/pgtable-bits.h +++ b/arch/mips/include/asm/pgtable-bits.h @@ -133,20 +133,13 @@ #define _PAGE_HUGE (1 << _PAGE_HUGE_SHIFT) #define _PAGE_SPLITTING_SHIFT (_PAGE_HUGE_SHIFT + 1) #define _PAGE_SPLITTING (1 << _PAGE_SPLITTING_SHIFT) - -/* Only R2 or newer cores have the XI bit */ -#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) -#define _PAGE_NO_EXEC_SHIFT (_PAGE_SPLITTING_SHIFT + 1) -#else -#define _PAGE_GLOBAL_SHIFT (_PAGE_SPLITTING_SHIFT + 1) -#define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT) -#endif /* CONFIG_CPU_MIPSR2 || CONFIG_CPU_MIPSR6 */ - #endif /* CONFIG_64BIT && CONFIG_MIPS_HUGE_TLB_SUPPORT */ #if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) /* XI - page cannot be executed */ -#ifndef _PAGE_NO_EXEC_SHIFT +#ifdef _PAGE_SPLITTING_SHIFT +#define _PAGE_NO_EXEC_SHIFT (_PAGE_SPLITTING_SHIFT + 1) +#else #define _PAGE_NO_EXEC_SHIFT (_PAGE_MODIFIED_SHIFT + 1) #endif #define _PAGE_NO_EXEC (cpu_has_rixi ? (1 << _PAGE_NO_EXEC_SHIFT) : 0) @@ -156,14 +149,16 @@ #define _PAGE_READ (cpu_has_rixi ? 0 : (1 << _PAGE_READ_SHIFT)) #define _PAGE_NO_READ_SHIFT _PAGE_READ_SHIFT #define _PAGE_NO_READ (cpu_has_rixi ? (1 << _PAGE_READ_SHIFT) : 0) +#endif /* defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) */ +#if defined(_PAGE_NO_READ_SHIFT) #define _PAGE_GLOBAL_SHIFT (_PAGE_NO_READ_SHIFT + 1) -#define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT) - -#else /* !CONFIG_CPU_MIPSR2 && !CONFIG_CPU_MIPSR6 */ +#elif defined(_PAGE_SPLITTING_SHIFT) +#define _PAGE_GLOBAL_SHIFT (_PAGE_SPLITTING_SHIFT + 1) +#else #define _PAGE_GLOBAL_SHIFT (_PAGE_MODIFIED_SHIFT + 1) +#endif #define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT) -#endif /* CONFIG_CPU_MIPSR2 || CONFIG_CPU_MIPSR6 */ #define _PAGE_VALID_SHIFT (_PAGE_GLOBAL_SHIFT + 1) #define _PAGE_VALID (1 << _PAGE_VALID_SHIFT) @@ -249,7 +244,7 @@ static inline uint64_t pte_to_entrylo(unsigned long pte_val) #define _CACHE_CACHABLE_NONCOHERENT (3<<_CACHE_SHIFT) /* LOONGSON */ #define _CACHE_CACHABLE_COHERENT (3<<_CACHE_SHIFT) /* LOONGSON-3 */ -#elif defined(CONFIG_MACH_JZ4740) +#elif defined(CONFIG_MACH_INGENIC) /* Ingenic uses the WA bit to achieve write-combine memory writes */ #define _CACHE_UNCACHED_ACCELERATED (1<<_CACHE_SHIFT) diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h index ae856947526443..8957f15e21ec4c 100644 --- a/arch/mips/include/asm/pgtable.h +++ b/arch/mips/include/asm/pgtable.h @@ -393,6 +393,8 @@ static inline pgprot_t pgprot_noncached(pgprot_t _prot) return __pgprot(prot); } +#define pgprot_writecombine pgprot_writecombine + static inline pgprot_t pgprot_writecombine(pgprot_t _prot) { unsigned long prot = pgprot_val(_prot); diff --git a/arch/mips/include/asm/processor.h b/arch/mips/include/asm/processor.h index 9b3b48e21c221f..59ee6dcf6eed9f 100644 --- a/arch/mips/include/asm/processor.h +++ b/arch/mips/include/asm/processor.h @@ -275,6 +275,7 @@ struct thread_struct { unsigned long cp0_badvaddr; /* Last user fault */ unsigned long cp0_baduaddr; /* Last kernel fault accessing USEG */ unsigned long error_code; + unsigned long trap_nr; #ifdef CONFIG_CPU_CAVIUM_OCTEON struct octeon_cop2_state cp2 __attribute__ ((__aligned__(128))); struct octeon_cvmseg_state cvmseg __attribute__ ((__aligned__(128))); @@ -341,6 +342,7 @@ struct thread_struct { .cp0_badvaddr = 0, \ .cp0_baduaddr = 0, \ .error_code = 0, \ + .trap_nr = 0, \ /* \ * Platform specific cop2 registers(null if no COP2) \ */ \ diff --git a/arch/mips/include/asm/ptrace.h b/arch/mips/include/asm/ptrace.h index ffc320389f40a0..f6fc6aac54963f 100644 --- a/arch/mips/include/asm/ptrace.h +++ b/arch/mips/include/asm/ptrace.h @@ -14,11 +14,16 @@ #include #include #include +#include +#include #include /* * This struct defines the way the registers are stored on the stack during a * system call/exception. As usual the registers k0/k1 aren't being saved. + * + * If you add a register here, also add it to regoffset_table[] in + * arch/mips/kernel/ptrace.c. */ struct pt_regs { #ifdef CONFIG_32BIT @@ -43,8 +48,83 @@ struct pt_regs { unsigned long long mpl[6]; /* MTM{0-5} */ unsigned long long mtp[6]; /* MTP{0-5} */ #endif + unsigned long __last[0]; } __aligned(8); +static inline unsigned long kernel_stack_pointer(struct pt_regs *regs) +{ + return regs->regs[31]; +} + +/* + * Don't use asm-generic/ptrace.h it defines FP accessors that don't make + * sense on MIPS. We rather want an error if they get invoked. + */ + +static inline void instruction_pointer_set(struct pt_regs *regs, + unsigned long val) +{ + regs->cp0_epc = val; +} + +/* Query offset/name of register from its name/offset */ +extern int regs_query_register_offset(const char *name); +#define MAX_REG_OFFSET (offsetof(struct pt_regs, __last)) + +/** + * regs_get_register() - get register value from its offset + * @regs: pt_regs from which register value is gotten. + * @offset: offset number of the register. + * + * regs_get_register returns the value of a register. The @offset is the + * offset of the register in struct pt_regs address which specified by @regs. + * If @offset is bigger than MAX_REG_OFFSET, this returns 0. + */ +static inline unsigned long regs_get_register(struct pt_regs *regs, + unsigned int offset) +{ + if (unlikely(offset > MAX_REG_OFFSET)) + return 0; + + return *(unsigned long *)((unsigned long)regs + offset); +} + +/** + * regs_within_kernel_stack() - check the address in the stack + * @regs: pt_regs which contains kernel stack pointer. + * @addr: address which is checked. + * + * regs_within_kernel_stack() checks @addr is within the kernel stack page(s). + * If @addr is within the kernel stack, it returns true. If not, returns false. + */ +static inline int regs_within_kernel_stack(struct pt_regs *regs, + unsigned long addr) +{ + return ((addr & ~(THREAD_SIZE - 1)) == + (kernel_stack_pointer(regs) & ~(THREAD_SIZE - 1))); +} + +/** + * regs_get_kernel_stack_nth() - get Nth entry of the stack + * @regs: pt_regs which contains kernel stack pointer. + * @n: stack entry number. + * + * regs_get_kernel_stack_nth() returns @n th entry of the kernel stack which + * is specified by @regs. If the @n th entry is NOT in the kernel stack, + * this returns 0. + */ +static inline unsigned long regs_get_kernel_stack_nth(struct pt_regs *regs, + unsigned int n) +{ + unsigned long *addr = (unsigned long *)kernel_stack_pointer(regs); + + addr += n; + if (regs_within_kernel_stack(regs, (unsigned long)addr)) + return *addr; + else + return 0; +} + struct task_struct; extern int ptrace_getregs(struct task_struct *child, diff --git a/arch/mips/include/asm/signal.h b/arch/mips/include/asm/signal.h index 8efe5a9e2c3e68..003e273eff4cef 100644 --- a/arch/mips/include/asm/signal.h +++ b/arch/mips/include/asm/signal.h @@ -23,4 +23,7 @@ #define __ARCH_HAS_IRIX_SIGACTION +extern int protected_save_fp_context(void __user *sc); +extern int protected_restore_fp_context(void __user *sc); + #endif /* _ASM_SIGNAL_H */ diff --git a/arch/mips/include/asm/spinlock.h b/arch/mips/include/asm/spinlock.h index 9de4ba43dcd11a..40196bebe849a0 100644 --- a/arch/mips/include/asm/spinlock.h +++ b/arch/mips/include/asm/spinlock.h @@ -42,6 +42,11 @@ static inline int arch_spin_is_locked(arch_spinlock_t *lock) return ((counters >> 16) ^ counters) & 0xffff; } +static inline int arch_spin_value_unlocked(arch_spinlock_t lock) +{ + return lock.h.serving_now == lock.h.ticket; +} + #define arch_spin_lock_flags(lock, flags) arch_spin_lock(lock) #define arch_spin_unlock_wait(x) \ while (arch_spin_is_locked(x)) { cpu_relax(); } diff --git a/arch/mips/include/asm/switch_to.h b/arch/mips/include/asm/switch_to.h index 9733cd0266e4b9..28b5d84a5022a1 100644 --- a/arch/mips/include/asm/switch_to.h +++ b/arch/mips/include/asm/switch_to.h @@ -16,29 +16,21 @@ #include #include #include -#include +#include struct task_struct; -enum { - FP_SAVE_NONE = 0, - FP_SAVE_VECTOR = -1, - FP_SAVE_SCALAR = 1, -}; - /** * resume - resume execution of a task * @prev: The task previously executed. * @next: The task to begin executing. * @next_ti: task_thread_info(next). - * @fp_save: Which, if any, FP context to save for prev. * * This function is used whilst scheduling to save the context of prev & load * the context of next. Returns prev. */ extern asmlinkage struct task_struct *resume(struct task_struct *prev, - struct task_struct *next, struct thread_info *next_ti, - s32 fp_save); + struct task_struct *next, struct thread_info *next_ti); extern unsigned int ll_bit; extern struct task_struct *ll_task; @@ -91,8 +83,8 @@ do { if (cpu_has_rw_llb) { \ */ #define switch_to(prev, next, last) \ do { \ - s32 __fpsave = FP_SAVE_NONE; \ __mips_mt_fpaff_switch_to(prev); \ + lose_fpu_inatomic(1, prev); \ if (cpu_has_dsp) { \ __save_dsp(prev); \ __restore_dsp(next); \ @@ -111,15 +103,10 @@ do { \ clear_c0_status(ST0_CU2); \ } \ __clear_software_ll_bit(); \ - if (test_and_clear_tsk_thread_flag(prev, TIF_USEDFPU)) \ - __fpsave = FP_SAVE_SCALAR; \ - if (test_and_clear_tsk_thread_flag(prev, TIF_USEDMSA)) \ - __fpsave = FP_SAVE_VECTOR; \ if (cpu_has_userlocal) \ write_c0_userlocal(task_thread_info(next)->tp_value); \ __restore_watch(); \ - disable_msa(); \ - (last) = resume(prev, next, task_thread_info(next), __fpsave); \ + (last) = resume(prev, next, task_thread_info(next)); \ } while (0) #endif /* _ASM_SWITCH_TO_H */ diff --git a/arch/mips/include/asm/thread_info.h b/arch/mips/include/asm/thread_info.h index 9c0014e87c1746..e309d8fcb5167b 100644 --- a/arch/mips/include/asm/thread_info.h +++ b/arch/mips/include/asm/thread_info.h @@ -99,6 +99,7 @@ static inline struct thread_info *current_thread_info(void) #define TIF_SYSCALL_AUDIT 3 /* syscall auditing active */ #define TIF_SECCOMP 4 /* secure computing */ #define TIF_NOTIFY_RESUME 5 /* callback before returning to user */ +#define TIF_UPROBE 6 /* breakpointed or singlestepping */ #define TIF_RESTORE_SIGMASK 9 /* restore signal mask in do_signal() */ #define TIF_USEDFPU 16 /* FPU was used by this task this quantum (SMP) */ #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ @@ -122,6 +123,7 @@ static inline struct thread_info *current_thread_info(void) #define _TIF_SYSCALL_AUDIT (1< +#include + +#include +#include + +/* + * We want this to be defined as union mips_instruction but that makes the + * generic code blow up. + */ +typedef u32 uprobe_opcode_t; + +/* + * Classic MIPS (note this implementation doesn't consider microMIPS yet) + * instructions are always 4 bytes but in order to deal with branches and + * their delay slots, we treat instructions as having 8 bytes maximum. + */ +#define MAX_UINSN_BYTES 8 +#define UPROBE_XOL_SLOT_BYTES 128 /* Max. cache line size */ + +#define UPROBE_BRK_UPROBE 0x000d000d /* break 13 */ +#define UPROBE_BRK_UPROBE_XOL 0x000e000d /* break 14 */ + +#define UPROBE_SWBP_INSN UPROBE_BRK_UPROBE +#define UPROBE_SWBP_INSN_SIZE 4 + +struct arch_uprobe { + unsigned long resume_epc; + u32 insn[2]; + u32 ixol[2]; + union mips_instruction orig_inst[MAX_UINSN_BYTES / 4]; +}; + +struct arch_uprobe_task { + unsigned long saved_trap_nr; +}; + +extern int arch_uprobe_analyze_insn(struct arch_uprobe *aup, + struct mm_struct *mm, unsigned long addr); +extern int arch_uprobe_pre_xol(struct arch_uprobe *aup, struct pt_regs *regs); +extern int arch_uprobe_post_xol(struct arch_uprobe *aup, struct pt_regs *regs); +extern bool arch_uprobe_xol_was_trapped(struct task_struct *tsk); +extern int arch_uprobe_exception_notify(struct notifier_block *self, + unsigned long val, void *data); +extern void arch_uprobe_abort_xol(struct arch_uprobe *aup, + struct pt_regs *regs); +extern unsigned long arch_uretprobe_hijack_return_addr( + unsigned long trampoline_vaddr, struct pt_regs *regs); + +#endif /* __ASM_UPROBES_H */ diff --git a/arch/mips/include/asm/vpe.h b/arch/mips/include/asm/vpe.h index 7849f3978feafe..80e70dbd1f6417 100644 --- a/arch/mips/include/asm/vpe.h +++ b/arch/mips/include/asm/vpe.h @@ -122,7 +122,7 @@ void release_vpe(struct vpe *v); void *alloc_progmem(unsigned long len); void release_progmem(void *ptr); -int __weak vpe_run(struct vpe *v); +int vpe_run(struct vpe *v); void cleanup_tc(struct tc *tc); int __init vpe_module_init(void); diff --git a/arch/mips/include/uapi/asm/break.h b/arch/mips/include/uapi/asm/break.h index 002c39ea20c3e2..9c4265cbf15181 100644 --- a/arch/mips/include/uapi/asm/break.h +++ b/arch/mips/include/uapi/asm/break.h @@ -21,6 +21,8 @@ #define BRK_DIVZERO 7 /* Divide by zero check */ #define BRK_RANGE 8 /* Range error check */ #define BRK_BUG 12 /* Used by BUG() */ +#define BRK_UPROBE 13 /* See */ +#define BRK_UPROBE_XOL 14 /* See */ #define BRK_MEMU 514 /* Used by FPU emulator */ #define BRK_KPROBE_BP 515 /* Kprobe break */ #define BRK_KPROBE_SSTEPBP 516 /* Kprobe single step software implementation */ diff --git a/arch/mips/include/uapi/asm/hwcap.h b/arch/mips/include/uapi/asm/hwcap.h new file mode 100644 index 00000000000000..c7484a7ca686d3 --- /dev/null +++ b/arch/mips/include/uapi/asm/hwcap.h @@ -0,0 +1,8 @@ +#ifndef _UAPI_ASM_HWCAP_H +#define _UAPI_ASM_HWCAP_H + +/* HWCAP flags */ +#define HWCAP_MIPS_R6 (1 << 0) +#define HWCAP_MIPS_MSA (1 << 1) + +#endif /* _UAPI_ASM_HWCAP_H */ diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h index fc0cf5ac0cf72c..9b44d5a816fa3e 100644 --- a/arch/mips/include/uapi/asm/inst.h +++ b/arch/mips/include/uapi/asm/inst.h @@ -26,7 +26,7 @@ enum major_op { cop0_op, cop1_op, cop2_op, cop1x_op, beql_op, bnel_op, blezl_op, bgtzl_op, daddi_op, cbcond1_op = daddi_op, daddiu_op, ldl_op, ldr_op, - spec2_op, jalx_op, mdmx_op, spec3_op, + spec2_op, jalx_op, mdmx_op, msa_op = mdmx_op, spec3_op, lb_op, lh_op, lwl_op, lw_op, lbu_op, lhu_op, lwr_op, lwu_op, sb_op, sh_op, swl_op, sw_op, @@ -167,8 +167,13 @@ enum cop1_sdw_func { fround_op = 0x0c, ftrunc_op = 0x0d, fceil_op = 0x0e, ffloor_op = 0x0f, fmovc_op = 0x11, fmovz_op = 0x12, - fmovn_op = 0x13, frecip_op = 0x15, - frsqrt_op = 0x16, fcvts_op = 0x20, + fmovn_op = 0x13, fseleqz_op = 0x14, + frecip_op = 0x15, frsqrt_op = 0x16, + fselnez_op = 0x17, fmaddf_op = 0x18, + fmsubf_op = 0x19, frint_op = 0x1a, + fclass_op = 0x1b, fmin_op = 0x1c, + fmina_op = 0x1d, fmax_op = 0x1e, + fmaxa_op = 0x1f, fcvts_op = 0x20, fcvtd_op = 0x21, fcvte_op = 0x22, fcvtw_op = 0x24, fcvtl_op = 0x25, fcmp_op = 0x30 @@ -220,6 +225,24 @@ enum bshfl_func { seh_op = 0x18, }; +/* + * func field for MSA MI10 format. + */ +enum msa_mi10_func { + msa_ld_op = 8, + msa_st_op = 9, +}; + +/* + * MSA 2 bit format fields. + */ +enum msa_2b_fmt { + msa_fmt_b = 0, + msa_fmt_h = 1, + msa_fmt_w = 2, + msa_fmt_d = 3, +}; + /* * (microMIPS) Major opcodes. */ @@ -611,6 +634,16 @@ struct v_format { /* MDMX vector format */ ;))))))) }; +struct msa_mi10_format { /* MSA MI10 */ + __BITFIELD_FIELD(unsigned int opcode : 6, + __BITFIELD_FIELD(signed int s10 : 10, + __BITFIELD_FIELD(unsigned int rs : 5, + __BITFIELD_FIELD(unsigned int wd : 5, + __BITFIELD_FIELD(unsigned int func : 4, + __BITFIELD_FIELD(unsigned int df : 2, + ;)))))) +}; + struct spec3_format { /* SPEC3 */ __BITFIELD_FIELD(unsigned int opcode:6, __BITFIELD_FIELD(unsigned int rs:5, @@ -888,6 +921,7 @@ union mips_instruction { struct p_format p_format; struct f_format f_format; struct ma_format ma_format; + struct msa_mi10_format msa_mi10_format; struct b_format b_format; struct ps_format ps_format; struct v_format v_format; diff --git a/arch/mips/include/uapi/asm/sigcontext.h b/arch/mips/include/uapi/asm/sigcontext.h index 9081d88ae44f31..5cbd9ae6421faa 100644 --- a/arch/mips/include/uapi/asm/sigcontext.h +++ b/arch/mips/include/uapi/asm/sigcontext.h @@ -12,6 +12,18 @@ #include #include +/* scalar FP context was used */ +#define USED_FP (1 << 0) + +/* the value of Status.FR when context was saved */ +#define USED_FR1 (1 << 1) + +/* FR=1, but with odd singles in bits 63:32 of preceding even double */ +#define USED_HYBRID_FPRS (1 << 2) + +/* extended context was used, see struct extcontext for details */ +#define USED_EXTCONTEXT (1 << 3) + #if _MIPS_SIM == _MIPS_SIM_ABI32 /* diff --git a/arch/mips/include/uapi/asm/swab.h b/arch/mips/include/uapi/asm/swab.h index 8f2d184dbe9f7a..c4ddc4f0d2dcb1 100644 --- a/arch/mips/include/uapi/asm/swab.h +++ b/arch/mips/include/uapi/asm/swab.h @@ -16,11 +16,13 @@ #if (defined(__mips_isa_rev) && (__mips_isa_rev >= 2)) || \ defined(_MIPS_ARCH_LOONGSON3A) -static inline __attribute_const__ __u16 __arch_swab16(__u16 x) +static inline __attribute__((nomips16)) __attribute_const__ + __u16 __arch_swab16(__u16 x) { __asm__( " .set push \n" " .set arch=mips32r2 \n" + " .set nomips16 \n" " wsbh %0, %1 \n" " .set pop \n" : "=r" (x) @@ -30,11 +32,13 @@ static inline __attribute_const__ __u16 __arch_swab16(__u16 x) } #define __arch_swab16 __arch_swab16 -static inline __attribute_const__ __u32 __arch_swab32(__u32 x) +static inline __attribute__((nomips16)) __attribute_const__ + __u32 __arch_swab32(__u32 x) { __asm__( " .set push \n" " .set arch=mips32r2 \n" + " .set nomips16 \n" " wsbh %0, %1 \n" " rotr %0, %0, 16 \n" " .set pop \n" @@ -50,11 +54,13 @@ static inline __attribute_const__ __u32 __arch_swab32(__u32 x) * 64-bit kernel on r2 CPUs. */ #ifdef __mips64 -static inline __attribute_const__ __u64 __arch_swab64(__u64 x) +static inline __attribute__((nomips16)) __attribute_const__ + __u64 __arch_swab64(__u64 x) { __asm__( " .set push \n" " .set arch=mips64r2 \n" + " .set nomips16 \n" " dsbh %0, %1 \n" " dshd %0, %0 \n" " .set pop \n" diff --git a/arch/mips/include/uapi/asm/ucontext.h b/arch/mips/include/uapi/asm/ucontext.h new file mode 100644 index 00000000000000..2320144ce85803 --- /dev/null +++ b/arch/mips/include/uapi/asm/ucontext.h @@ -0,0 +1,65 @@ +#ifndef __MIPS_UAPI_ASM_UCONTEXT_H +#define __MIPS_UAPI_ASM_UCONTEXT_H + +/** + * struct extcontext - extended context header structure + * @magic: magic value identifying the type of extended context + * @size: the size in bytes of the enclosing structure + * + * Extended context structures provide context which does not fit within struct + * sigcontext. They are placed sequentially in memory at the end of struct + * ucontext and struct sigframe, with each extended context structure beginning + * with a header defined by this struct. The type of context represented is + * indicated by the magic field. Userland may check each extended context + * structure against magic values that it recognises. The size field allows any + * unrecognised context to be skipped, allowing for future expansion. The end + * of the extended context data is indicated by the magic value + * END_EXTCONTEXT_MAGIC. + */ +struct extcontext { + unsigned int magic; + unsigned int size; +}; + +/** + * struct msa_extcontext - MSA extended context structure + * @ext: the extended context header, with magic == MSA_EXTCONTEXT_MAGIC + * @wr: the most significant 64 bits of each MSA vector register + * @csr: the value of the MSA control & status register + * + * If MSA context is live for a task at the time a signal is delivered to it, + * this structure will hold the MSA context of the task as it was prior to the + * signal delivery. + */ +struct msa_extcontext { + struct extcontext ext; +#define MSA_EXTCONTEXT_MAGIC 0x784d5341 /* xMSA */ + + unsigned long long wr[32]; + unsigned int csr; +}; + +#define END_EXTCONTEXT_MAGIC 0x78454e44 /* xEND */ + +/** + * struct ucontext - user context structure + * @uc_flags: + * @uc_link: + * @uc_stack: + * @uc_mcontext: holds basic processor state + * @uc_sigmask: + * @uc_extcontext: holds extended processor state + */ +struct ucontext { + /* Historic fields matching asm-generic */ + unsigned long uc_flags; + struct ucontext *uc_link; + stack_t uc_stack; + struct sigcontext uc_mcontext; + sigset_t uc_sigmask; + + /* Extended context structures may follow ucontext */ + unsigned long long uc_extcontext[0]; +}; + +#endif /* __MIPS_UAPI_ASM_UCONTEXT_H */ diff --git a/arch/mips/jazz/irq.c b/arch/mips/jazz/irq.c index e1ea4f625f7a45..5d6828b2a750c2 100644 --- a/arch/mips/jazz/irq.c +++ b/arch/mips/jazz/irq.c @@ -110,18 +110,11 @@ asmlinkage void plat_irq_dispatch(void) } } -static void r4030_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) -{ - /* Nothing to do ... */ -} - struct clock_event_device r4030_clockevent = { .name = "r4030", .features = CLOCK_EVT_FEAT_PERIODIC, .rating = 300, .irq = JAZZ_TIMER_IRQ, - .set_mode = r4030_set_mode, }; static irqreturn_t r4030_timer_interrupt(int irq, void *dev_id) diff --git a/arch/mips/jz4740/gpio.c b/arch/mips/jz4740/gpio.c index 54c80d42a88d60..6cd69fdaa1c5f9 100644 --- a/arch/mips/jz4740/gpio.c +++ b/arch/mips/jz4740/gpio.c @@ -231,6 +231,13 @@ static int jz_gpio_direction_input(struct gpio_chip *chip, unsigned gpio) return 0; } +static int jz_gpio_to_irq(struct gpio_chip *chip, unsigned gpio) +{ + struct jz_gpio_chip *jz_gpio = gpio_chip_to_jz_gpio_chip(chip); + + return jz_gpio->irq_base + gpio; +} + int jz_gpio_port_direction_input(int port, uint32_t mask) { writel(mask, GPIO_TO_REG(port, JZ_REG_GPIO_DIRECTION_CLEAR)); @@ -262,18 +269,6 @@ uint32_t jz_gpio_port_get_value(int port, uint32_t mask) } EXPORT_SYMBOL(jz_gpio_port_get_value); -int gpio_to_irq(unsigned gpio) -{ - return JZ4740_IRQ_GPIO(0) + gpio; -} -EXPORT_SYMBOL_GPL(gpio_to_irq); - -int irq_to_gpio(unsigned irq) -{ - return irq - JZ4740_IRQ_GPIO(0); -} -EXPORT_SYMBOL_GPL(irq_to_gpio); - #define IRQ_TO_BIT(irq) BIT(irq_to_gpio(irq) & 0x1f) static void jz_gpio_check_trigger_both(struct jz_gpio_chip *chip, unsigned int irq) @@ -403,6 +398,7 @@ static int jz_gpio_irq_set_wake(struct irq_data *data, unsigned int on) .get = jz_gpio_get_value, \ .direction_output = jz_gpio_direction_output, \ .direction_input = jz_gpio_direction_input, \ + .to_irq = jz_gpio_to_irq, \ .base = JZ4740_GPIO_BASE_ ## _bank, \ .ngpio = JZ4740_GPIO_NUM_ ## _bank, \ }, \ @@ -423,8 +419,8 @@ static void jz4740_gpio_chip_init(struct jz_gpio_chip *chip, unsigned int id) chip->base = ioremap(JZ4740_GPIO_BASE_ADDR + (id * 0x100), 0x100); chip->irq = JZ4740_IRQ_INTC_GPIO(id); - irq_set_handler_data(chip->irq, chip); - irq_set_chained_handler(chip->irq, jz_gpio_irq_demux_handler); + irq_set_chained_handler_and_data(chip->irq, + jz_gpio_irq_demux_handler, chip); gc = irq_alloc_generic_chip(chip->gpio_chip.label, 1, chip->irq_base, chip->base, handle_level_irq); diff --git a/arch/mips/jz4740/time.c b/arch/mips/jz4740/time.c index 7ab47fee1be8fd..1f7ca2c9f26259 100644 --- a/arch/mips/jz4740/time.c +++ b/arch/mips/jz4740/time.c @@ -58,7 +58,7 @@ static irqreturn_t jz4740_clockevent_irq(int irq, void *devid) jz4740_timer_ack_full(TIMER_CLOCKEVENT); - if (cd->mode != CLOCK_EVT_MODE_PERIODIC) + if (!clockevent_state_periodic(cd)) jz4740_timer_disable(TIMER_CLOCKEVENT); cd->event_handler(cd); @@ -66,24 +66,29 @@ static irqreturn_t jz4740_clockevent_irq(int irq, void *devid) return IRQ_HANDLED; } -static void jz4740_clockevent_set_mode(enum clock_event_mode mode, - struct clock_event_device *cd) +static int jz4740_clockevent_set_periodic(struct clock_event_device *evt) { - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - jz4740_timer_set_count(TIMER_CLOCKEVENT, 0); - jz4740_timer_set_period(TIMER_CLOCKEVENT, jz4740_jiffies_per_tick); - case CLOCK_EVT_MODE_RESUME: - jz4740_timer_irq_full_enable(TIMER_CLOCKEVENT); - jz4740_timer_enable(TIMER_CLOCKEVENT); - break; - case CLOCK_EVT_MODE_ONESHOT: - case CLOCK_EVT_MODE_SHUTDOWN: - jz4740_timer_disable(TIMER_CLOCKEVENT); - break; - default: - break; - } + jz4740_timer_set_count(TIMER_CLOCKEVENT, 0); + jz4740_timer_set_period(TIMER_CLOCKEVENT, jz4740_jiffies_per_tick); + jz4740_timer_irq_full_enable(TIMER_CLOCKEVENT); + jz4740_timer_enable(TIMER_CLOCKEVENT); + + return 0; +} + +static int jz4740_clockevent_resume(struct clock_event_device *evt) +{ + jz4740_timer_irq_full_enable(TIMER_CLOCKEVENT); + jz4740_timer_enable(TIMER_CLOCKEVENT); + + return 0; +} + +static int jz4740_clockevent_shutdown(struct clock_event_device *evt) +{ + jz4740_timer_disable(TIMER_CLOCKEVENT); + + return 0; } static int jz4740_clockevent_set_next(unsigned long evt, @@ -100,7 +105,10 @@ static struct clock_event_device jz4740_clockevent = { .name = "jz4740-timer", .features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT, .set_next_event = jz4740_clockevent_set_next, - .set_mode = jz4740_clockevent_set_mode, + .set_state_shutdown = jz4740_clockevent_shutdown, + .set_state_periodic = jz4740_clockevent_set_periodic, + .set_state_oneshot = jz4740_clockevent_shutdown, + .tick_resume = jz4740_clockevent_resume, .rating = 200, #ifdef CONFIG_MACH_JZ4740 .irq = JZ4740_IRQ_TCU0, diff --git a/arch/mips/kernel/Makefile b/arch/mips/kernel/Makefile index 3156c8d253c1c0..d982be1ea1c3f3 100644 --- a/arch/mips/kernel/Makefile +++ b/arch/mips/kernel/Makefile @@ -99,6 +99,7 @@ obj-$(CONFIG_PERF_EVENTS) += perf_event.o obj-$(CONFIG_HW_PERF_EVENTS) += perf_event_mipsxx.o obj-$(CONFIG_JUMP_LABEL) += jump_label.o +obj-$(CONFIG_UPROBES) += uprobes.o obj-$(CONFIG_MIPS_CM) += mips-cm.o obj-$(CONFIG_MIPS_CPC) += mips-cpc.o diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c index 072fab13645d50..154e2039ea5ef1 100644 --- a/arch/mips/kernel/asm-offsets.c +++ b/arch/mips/kernel/asm-offsets.c @@ -128,6 +128,7 @@ void output_thread_defines(void) thread.cp0_baduaddr); OFFSET(THREAD_ECODE, task_struct, \ thread.error_code); + OFFSET(THREAD_TRAPNO, task_struct, thread.trap_nr); BLANK(); } @@ -245,17 +246,6 @@ void output_sc_defines(void) } #endif -#ifdef CONFIG_MIPS32_COMPAT -void output_sc32_defines(void) -{ - COMMENT("Linux 32-bit sigcontext offsets."); - OFFSET(SC32_FPREGS, sigcontext32, sc_fpregs); - OFFSET(SC32_FPC_CSR, sigcontext32, sc_fpc_csr); - OFFSET(SC32_FPC_EIR, sigcontext32, sc_fpc_eir); - BLANK(); -} -#endif - void output_signal_defined(void) { COMMENT("Linux signal numbers."); diff --git a/arch/mips/kernel/cevt-bcm1480.c b/arch/mips/kernel/cevt-bcm1480.c index 7976457184b180..940ac00e9129ac 100644 --- a/arch/mips/kernel/cevt-bcm1480.c +++ b/arch/mips/kernel/cevt-bcm1480.c @@ -40,8 +40,8 @@ * The general purpose timer ticks at 1MHz independent if * the rest of the system */ -static void sibyte_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) + +static int sibyte_set_periodic(struct clock_event_device *evt) { unsigned int cpu = smp_processor_id(); void __iomem *cfg, *init; @@ -49,24 +49,22 @@ static void sibyte_set_mode(enum clock_event_mode mode, cfg = IOADDR(A_SCD_TIMER_REGISTER(cpu, R_SCD_TIMER_CFG)); init = IOADDR(A_SCD_TIMER_REGISTER(cpu, R_SCD_TIMER_INIT)); - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - __raw_writeq(0, cfg); - __raw_writeq((V_SCD_TIMER_FREQ / HZ) - 1, init); - __raw_writeq(M_SCD_TIMER_ENABLE | M_SCD_TIMER_MODE_CONTINUOUS, - cfg); - break; - - case CLOCK_EVT_MODE_ONESHOT: - /* Stop the timer until we actually program a shot */ - case CLOCK_EVT_MODE_SHUTDOWN: - __raw_writeq(0, cfg); - break; - - case CLOCK_EVT_MODE_UNUSED: /* shuddup gcc */ - case CLOCK_EVT_MODE_RESUME: - ; - } + __raw_writeq(0, cfg); + __raw_writeq((V_SCD_TIMER_FREQ / HZ) - 1, init); + __raw_writeq(M_SCD_TIMER_ENABLE | M_SCD_TIMER_MODE_CONTINUOUS, cfg); + return 0; +} + +static int sibyte_shutdown(struct clock_event_device *evt) +{ + unsigned int cpu = smp_processor_id(); + void __iomem *cfg; + + cfg = IOADDR(A_SCD_TIMER_REGISTER(cpu, R_SCD_TIMER_CFG)); + + /* Stop the timer until we actually program a shot */ + __raw_writeq(0, cfg); + return 0; } static int sibyte_next_event(unsigned long delta, struct clock_event_device *cd) @@ -91,7 +89,7 @@ static irqreturn_t sibyte_counter_handler(int irq, void *dev_id) void __iomem *cfg; unsigned long tmode; - if (cd->mode == CLOCK_EVT_MODE_PERIODIC) + if (clockevent_state_periodic(cd)) tmode = M_SCD_TIMER_ENABLE | M_SCD_TIMER_MODE_CONTINUOUS; else tmode = 0; @@ -130,7 +128,9 @@ void sb1480_clockevent_init(void) cd->irq = irq; cd->cpumask = cpumask_of(cpu); cd->set_next_event = sibyte_next_event; - cd->set_mode = sibyte_set_mode; + cd->set_state_shutdown = sibyte_shutdown; + cd->set_state_periodic = sibyte_set_periodic; + cd->set_state_oneshot = sibyte_shutdown; clockevents_register_device(cd); bcm1480_mask_irq(cpu, irq); diff --git a/arch/mips/kernel/cevt-ds1287.c b/arch/mips/kernel/cevt-ds1287.c index ff1f01b72270fd..77a5ddf53f57e5 100644 --- a/arch/mips/kernel/cevt-ds1287.c +++ b/arch/mips/kernel/cevt-ds1287.c @@ -59,27 +59,32 @@ static int ds1287_set_next_event(unsigned long delta, return -EINVAL; } -static void ds1287_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) +static int ds1287_shutdown(struct clock_event_device *evt) { u8 val; spin_lock(&rtc_lock); val = CMOS_READ(RTC_REG_B); + val &= ~RTC_PIE; + CMOS_WRITE(val, RTC_REG_B); - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - val |= RTC_PIE; - break; - default: - val &= ~RTC_PIE; - break; - } + spin_unlock(&rtc_lock); + return 0; +} +static int ds1287_set_periodic(struct clock_event_device *evt) +{ + u8 val; + + spin_lock(&rtc_lock); + + val = CMOS_READ(RTC_REG_B); + val |= RTC_PIE; CMOS_WRITE(val, RTC_REG_B); spin_unlock(&rtc_lock); + return 0; } static void ds1287_event_handler(struct clock_event_device *dev) @@ -87,11 +92,13 @@ static void ds1287_event_handler(struct clock_event_device *dev) } static struct clock_event_device ds1287_clockevent = { - .name = "ds1287", - .features = CLOCK_EVT_FEAT_PERIODIC, - .set_next_event = ds1287_set_next_event, - .set_mode = ds1287_set_mode, - .event_handler = ds1287_event_handler, + .name = "ds1287", + .features = CLOCK_EVT_FEAT_PERIODIC, + .set_next_event = ds1287_set_next_event, + .set_state_shutdown = ds1287_shutdown, + .set_state_periodic = ds1287_set_periodic, + .tick_resume = ds1287_shutdown, + .event_handler = ds1287_event_handler, }; static irqreturn_t ds1287_interrupt(int irq, void *dev_id) diff --git a/arch/mips/kernel/cevt-gt641xx.c b/arch/mips/kernel/cevt-gt641xx.c index f069460751ab93..66040051151d3c 100644 --- a/arch/mips/kernel/cevt-gt641xx.c +++ b/arch/mips/kernel/cevt-gt641xx.c @@ -64,8 +64,7 @@ static int gt641xx_timer0_set_next_event(unsigned long delta, return 0; } -static void gt641xx_timer0_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) +static int gt641xx_timer0_shutdown(struct clock_event_device *evt) { u32 ctrl; @@ -73,21 +72,39 @@ static void gt641xx_timer0_set_mode(enum clock_event_mode mode, ctrl = GT_READ(GT_TC_CONTROL_OFS); ctrl &= ~(GT_TC_CONTROL_ENTC0_MSK | GT_TC_CONTROL_SELTC0_MSK); + GT_WRITE(GT_TC_CONTROL_OFS, ctrl); + + raw_spin_unlock(>641xx_timer_lock); + return 0; +} + +static int gt641xx_timer0_set_oneshot(struct clock_event_device *evt) +{ + u32 ctrl; + + raw_spin_lock(>641xx_timer_lock); + + ctrl = GT_READ(GT_TC_CONTROL_OFS); + ctrl &= ~GT_TC_CONTROL_SELTC0_MSK; + ctrl |= GT_TC_CONTROL_ENTC0_MSK; + GT_WRITE(GT_TC_CONTROL_OFS, ctrl); + + raw_spin_unlock(>641xx_timer_lock); + return 0; +} - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - ctrl |= GT_TC_CONTROL_ENTC0_MSK | GT_TC_CONTROL_SELTC0_MSK; - break; - case CLOCK_EVT_MODE_ONESHOT: - ctrl |= GT_TC_CONTROL_ENTC0_MSK; - break; - default: - break; - } +static int gt641xx_timer0_set_periodic(struct clock_event_device *evt) +{ + u32 ctrl; + + raw_spin_lock(>641xx_timer_lock); + ctrl = GT_READ(GT_TC_CONTROL_OFS); + ctrl |= GT_TC_CONTROL_ENTC0_MSK | GT_TC_CONTROL_SELTC0_MSK; GT_WRITE(GT_TC_CONTROL_OFS, ctrl); raw_spin_unlock(>641xx_timer_lock); + return 0; } static void gt641xx_timer0_event_handler(struct clock_event_device *dev) @@ -95,12 +112,16 @@ static void gt641xx_timer0_event_handler(struct clock_event_device *dev) } static struct clock_event_device gt641xx_timer0_clockevent = { - .name = "gt641xx-timer0", - .features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT, - .irq = GT641XX_TIMER0_IRQ, - .set_next_event = gt641xx_timer0_set_next_event, - .set_mode = gt641xx_timer0_set_mode, - .event_handler = gt641xx_timer0_event_handler, + .name = "gt641xx-timer0", + .features = CLOCK_EVT_FEAT_PERIODIC | + CLOCK_EVT_FEAT_ONESHOT, + .irq = GT641XX_TIMER0_IRQ, + .set_next_event = gt641xx_timer0_set_next_event, + .set_state_shutdown = gt641xx_timer0_shutdown, + .set_state_periodic = gt641xx_timer0_set_periodic, + .set_state_oneshot = gt641xx_timer0_set_oneshot, + .tick_resume = gt641xx_timer0_shutdown, + .event_handler = gt641xx_timer0_event_handler, }; static irqreturn_t gt641xx_timer0_interrupt(int irq, void *dev_id) diff --git a/arch/mips/kernel/cevt-r4k.c b/arch/mips/kernel/cevt-r4k.c index d70c4d893219eb..8dfe6a6e14809b 100644 --- a/arch/mips/kernel/cevt-r4k.c +++ b/arch/mips/kernel/cevt-r4k.c @@ -28,12 +28,6 @@ static int mips_next_event(unsigned long delta, return res; } -void mips_set_clock_mode(enum clock_event_mode mode, - struct clock_event_device *evt) -{ - /* Nothing to do ... */ -} - DEFINE_PER_CPU(struct clock_event_device, mips_clockevent_device); int cp0_timer_irq_installed; @@ -174,6 +168,11 @@ int c0_compare_int_usable(void) return 1; } +unsigned int __weak get_c0_compare_int(void) +{ + return MIPS_CPU_IRQ_BASE + cp0_compare_irq; +} + int r4k_clockevent_init(void) { unsigned int cpu = smp_processor_id(); @@ -189,11 +188,9 @@ int r4k_clockevent_init(void) /* * With vectored interrupts things are getting platform specific. * get_c0_compare_int is a hook to allow a platform to return the - * interrupt number of it's liking. + * interrupt number of its liking. */ - irq = MIPS_CPU_IRQ_BASE + cp0_compare_irq; - if (get_c0_compare_int) - irq = get_c0_compare_int(); + irq = get_c0_compare_int(); cd = &per_cpu(mips_clockevent_device, cpu); @@ -212,7 +209,6 @@ int r4k_clockevent_init(void) cd->irq = irq; cd->cpumask = cpumask_of(cpu); cd->set_next_event = mips_next_event; - cd->set_mode = mips_set_clock_mode; cd->event_handler = mips_event_handler; clockevents_register_device(cd); diff --git a/arch/mips/kernel/cevt-sb1250.c b/arch/mips/kernel/cevt-sb1250.c index 5ea6d6b1de1504..3d860efd63b972 100644 --- a/arch/mips/kernel/cevt-sb1250.c +++ b/arch/mips/kernel/cevt-sb1250.c @@ -38,8 +38,20 @@ * The general purpose timer ticks at 1MHz independent if * the rest of the system */ -static void sibyte_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) + +static int sibyte_shutdown(struct clock_event_device *evt) +{ + void __iomem *cfg; + + cfg = IOADDR(A_SCD_TIMER_REGISTER(smp_processor_id(), R_SCD_TIMER_CFG)); + + /* Stop the timer until we actually program a shot */ + __raw_writeq(0, cfg); + + return 0; +} + +static int sibyte_set_periodic(struct clock_event_device *evt) { unsigned int cpu = smp_processor_id(); void __iomem *cfg, *init; @@ -47,24 +59,11 @@ static void sibyte_set_mode(enum clock_event_mode mode, cfg = IOADDR(A_SCD_TIMER_REGISTER(cpu, R_SCD_TIMER_CFG)); init = IOADDR(A_SCD_TIMER_REGISTER(cpu, R_SCD_TIMER_INIT)); - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - __raw_writeq(0, cfg); - __raw_writeq((V_SCD_TIMER_FREQ / HZ) - 1, init); - __raw_writeq(M_SCD_TIMER_ENABLE | M_SCD_TIMER_MODE_CONTINUOUS, - cfg); - break; - - case CLOCK_EVT_MODE_ONESHOT: - /* Stop the timer until we actually program a shot */ - case CLOCK_EVT_MODE_SHUTDOWN: - __raw_writeq(0, cfg); - break; - - case CLOCK_EVT_MODE_UNUSED: /* shuddup gcc */ - case CLOCK_EVT_MODE_RESUME: - ; - } + __raw_writeq(0, cfg); + __raw_writeq((V_SCD_TIMER_FREQ / HZ) - 1, init); + __raw_writeq(M_SCD_TIMER_ENABLE | M_SCD_TIMER_MODE_CONTINUOUS, cfg); + + return 0; } static int sibyte_next_event(unsigned long delta, struct clock_event_device *cd) @@ -89,7 +88,7 @@ static irqreturn_t sibyte_counter_handler(int irq, void *dev_id) void __iomem *cfg; unsigned long tmode; - if (cd->mode == CLOCK_EVT_MODE_PERIODIC) + if (clockevent_state_periodic(cd)) tmode = M_SCD_TIMER_ENABLE | M_SCD_TIMER_MODE_CONTINUOUS; else tmode = 0; @@ -129,7 +128,9 @@ void sb1250_clockevent_init(void) cd->irq = irq; cd->cpumask = cpumask_of(cpu); cd->set_next_event = sibyte_next_event; - cd->set_mode = sibyte_set_mode; + cd->set_state_shutdown = sibyte_shutdown; + cd->set_state_periodic = sibyte_set_periodic; + cd->set_state_oneshot = sibyte_shutdown; clockevents_register_device(cd); sb1250_mask_irq(cpu, irq); diff --git a/arch/mips/kernel/cevt-txx9.c b/arch/mips/kernel/cevt-txx9.c index 723932441ecc71..537eefdf838fec 100644 --- a/arch/mips/kernel/cevt-txx9.c +++ b/arch/mips/kernel/cevt-txx9.c @@ -85,36 +85,54 @@ static void txx9tmr_stop_and_clear(struct txx9_tmr_reg __iomem *tmrptr) __raw_writel(0, &tmrptr->tisr); } -static void txx9tmr_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) +static int txx9tmr_set_state_periodic(struct clock_event_device *evt) { struct txx9_clock_event_device *txx9_cd = container_of(evt, struct txx9_clock_event_device, cd); struct txx9_tmr_reg __iomem *tmrptr = txx9_cd->tmrptr; txx9tmr_stop_and_clear(tmrptr); - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - __raw_writel(TXx9_TMITMR_TIIE | TXx9_TMITMR_TZCE, - &tmrptr->itmr); - /* start timer */ - __raw_writel(((u64)(NSEC_PER_SEC / HZ) * evt->mult) >> - evt->shift, - &tmrptr->cpra); - __raw_writel(TCR_BASE | TXx9_TMTCR_TCE, &tmrptr->tcr); - break; - case CLOCK_EVT_MODE_SHUTDOWN: - case CLOCK_EVT_MODE_UNUSED: - __raw_writel(0, &tmrptr->itmr); - break; - case CLOCK_EVT_MODE_ONESHOT: - __raw_writel(TXx9_TMITMR_TIIE, &tmrptr->itmr); - break; - case CLOCK_EVT_MODE_RESUME: - __raw_writel(TIMER_CCD, &tmrptr->ccdr); - __raw_writel(0, &tmrptr->itmr); - break; - } + + __raw_writel(TXx9_TMITMR_TIIE | TXx9_TMITMR_TZCE, &tmrptr->itmr); + /* start timer */ + __raw_writel(((u64)(NSEC_PER_SEC / HZ) * evt->mult) >> evt->shift, + &tmrptr->cpra); + __raw_writel(TCR_BASE | TXx9_TMTCR_TCE, &tmrptr->tcr); + return 0; +} + +static int txx9tmr_set_state_oneshot(struct clock_event_device *evt) +{ + struct txx9_clock_event_device *txx9_cd = + container_of(evt, struct txx9_clock_event_device, cd); + struct txx9_tmr_reg __iomem *tmrptr = txx9_cd->tmrptr; + + txx9tmr_stop_and_clear(tmrptr); + __raw_writel(TXx9_TMITMR_TIIE, &tmrptr->itmr); + return 0; +} + +static int txx9tmr_set_state_shutdown(struct clock_event_device *evt) +{ + struct txx9_clock_event_device *txx9_cd = + container_of(evt, struct txx9_clock_event_device, cd); + struct txx9_tmr_reg __iomem *tmrptr = txx9_cd->tmrptr; + + txx9tmr_stop_and_clear(tmrptr); + __raw_writel(0, &tmrptr->itmr); + return 0; +} + +static int txx9tmr_tick_resume(struct clock_event_device *evt) +{ + struct txx9_clock_event_device *txx9_cd = + container_of(evt, struct txx9_clock_event_device, cd); + struct txx9_tmr_reg __iomem *tmrptr = txx9_cd->tmrptr; + + txx9tmr_stop_and_clear(tmrptr); + __raw_writel(TIMER_CCD, &tmrptr->ccdr); + __raw_writel(0, &tmrptr->itmr); + return 0; } static int txx9tmr_set_next_event(unsigned long delta, @@ -133,12 +151,15 @@ static int txx9tmr_set_next_event(unsigned long delta, static struct txx9_clock_event_device txx9_clock_event_device = { .cd = { - .name = "TXx9", - .features = CLOCK_EVT_FEAT_PERIODIC | - CLOCK_EVT_FEAT_ONESHOT, - .rating = 200, - .set_mode = txx9tmr_set_mode, - .set_next_event = txx9tmr_set_next_event, + .name = "TXx9", + .features = CLOCK_EVT_FEAT_PERIODIC | + CLOCK_EVT_FEAT_ONESHOT, + .rating = 200, + .set_state_shutdown = txx9tmr_set_state_shutdown, + .set_state_periodic = txx9tmr_set_state_periodic, + .set_state_oneshot = txx9tmr_set_state_oneshot, + .tick_resume = txx9tmr_tick_resume, + .set_next_event = txx9tmr_set_next_event, }, }; diff --git a/arch/mips/kernel/cps-vec.S b/arch/mips/kernel/cps-vec.S index 1b6ca634e6465b..9f71c06aebf631 100644 --- a/arch/mips/kernel/cps-vec.S +++ b/arch/mips/kernel/cps-vec.S @@ -152,7 +152,7 @@ dcache_done: /* Enter the coherent domain */ li t0, 0xff - PTR_S t0, GCR_CL_COHERENCE_OFS(v1) + sw t0, GCR_CL_COHERENCE_OFS(v1) ehb /* Jump to kseg0 */ @@ -302,7 +302,7 @@ LEAF(mips_cps_boot_vpes) PTR_L t0, 0(t0) /* Calculate a pointer to this cores struct core_boot_config */ - PTR_L t0, GCR_CL_ID_OFS(t0) + lw t0, GCR_CL_ID_OFS(t0) li t1, COREBOOTCFG_SIZE mul t0, t0, t1 PTR_LA t1, mips_cps_core_bootcfg diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c index dbe0792fc9c1bd..571a8e6ea5bd00 100644 --- a/arch/mips/kernel/cpu-probe.c +++ b/arch/mips/kernel/cpu-probe.c @@ -32,6 +32,9 @@ #include #include +/* Hardware capabilities */ +unsigned int elf_hwcap __read_mostly; + /* * Get the FPU Implementation/Revision. */ @@ -188,7 +191,7 @@ __setup("nohtw", htw_disable); static int mips_ftlb_disabled; static int mips_has_ftlb_configured; -static void set_ftlb_enable(struct cpuinfo_mips *c, int enable); +static int set_ftlb_enable(struct cpuinfo_mips *c, int enable); static int __init ftlb_disable(char *s) { @@ -202,7 +205,10 @@ static int __init ftlb_disable(char *s) return 1; /* Disable it in the boot cpu */ - set_ftlb_enable(&cpu_data[0], 0); + if (set_ftlb_enable(&cpu_data[0], 0)) { + pr_warn("Can't turn FTLB off\n"); + return 1; + } back_to_back_c0_hazard(); @@ -364,30 +370,41 @@ static unsigned int calculate_ftlb_probability(struct cpuinfo_mips *c) return 3; } -static void set_ftlb_enable(struct cpuinfo_mips *c, int enable) +static int set_ftlb_enable(struct cpuinfo_mips *c, int enable) { - unsigned int config6; + unsigned int config; /* It's implementation dependent how the FTLB can be enabled */ switch (c->cputype) { case CPU_PROAPTIV: case CPU_P5600: /* proAptiv & related cores use Config6 to enable the FTLB */ - config6 = read_c0_config6(); + config = read_c0_config6(); /* Clear the old probability value */ - config6 &= ~(3 << MIPS_CONF6_FTLBP_SHIFT); + config &= ~(3 << MIPS_CONF6_FTLBP_SHIFT); if (enable) /* Enable FTLB */ - write_c0_config6(config6 | + write_c0_config6(config | (calculate_ftlb_probability(c) << MIPS_CONF6_FTLBP_SHIFT) | MIPS_CONF6_FTLBEN); else /* Disable FTLB */ - write_c0_config6(config6 & ~MIPS_CONF6_FTLBEN); - back_to_back_c0_hazard(); + write_c0_config6(config & ~MIPS_CONF6_FTLBEN); break; + case CPU_I6400: + /* I6400 & related cores use Config7 to configure FTLB */ + config = read_c0_config7(); + /* Clear the old probability value */ + config &= ~(3 << MIPS_CONF7_FTLBP_SHIFT); + write_c0_config7(config | (calculate_ftlb_probability(c) + << MIPS_CONF7_FTLBP_SHIFT)); + break; + default: + return 1; } + + return 0; } static inline unsigned int decode_config0(struct cpuinfo_mips *c) @@ -524,6 +541,8 @@ static inline unsigned int decode_config3(struct cpuinfo_mips *c) } if (config3 & MIPS_CONF3_CDMM) c->options |= MIPS_CPU_CDMM; + if (config3 & MIPS_CONF3_SP) + c->options |= MIPS_CPU_SP; return config3 & MIPS_CONF_M; } @@ -540,7 +559,16 @@ static inline unsigned int decode_config4(struct cpuinfo_mips *c) if (cpu_has_tlb) { if (((config4 & MIPS_CONF4_IE) >> 29) == 2) c->options |= MIPS_CPU_TLBINV; - mmuextdef = config4 & MIPS_CONF4_MMUEXTDEF; + /* + * This is a bit ugly. R6 has dropped that field from + * config4 and the only valid configuration is VTLB+FTLB so + * set a good value for mmuextdef for that case. + */ + if (cpu_has_mips_r6) + mmuextdef = MIPS_CONF4_MMUEXTDEF_VTLBSIZEEXT; + else + mmuextdef = config4 & MIPS_CONF4_MMUEXTDEF; + switch (mmuextdef) { case MIPS_CONF4_MMUEXTDEF_MMUSIZEEXT: c->tlbsize += (config4 & MIPS_CONF4_MMUSIZEEXT) * 0x40; @@ -1121,6 +1149,10 @@ static inline void cpu_probe_mips(struct cpuinfo_mips *c, unsigned int cpu) c->cputype = CPU_P5600; __cpu_name[cpu] = "MIPS P5600"; break; + case PRID_IMP_I6400: + c->cputype = CPU_I6400; + __cpu_name[cpu] = "MIPS I6400"; + break; case PRID_IMP_M5150: c->cputype = CPU_M5150; __cpu_name[cpu] = "MIPS M5150"; @@ -1492,10 +1524,14 @@ void cpu_probe(void) else c->srsets = 1; + if (cpu_has_mips_r6) + elf_hwcap |= HWCAP_MIPS_R6; + if (cpu_has_msa) { c->msa_id = cpu_get_msa_id(); WARN(c->msa_id & MSA_IR_WRPF, "Vector register partitioning unimplemented!"); + elf_hwcap |= HWCAP_MIPS_MSA; } cpu_probe_vmbits(c); diff --git a/arch/mips/kernel/idle.c b/arch/mips/kernel/idle.c index e4f62b7875d24b..ab1478d5a4db1b 100644 --- a/arch/mips/kernel/idle.c +++ b/arch/mips/kernel/idle.c @@ -196,6 +196,7 @@ void __init check_wait(void) case CPU_INTERAPTIV: case CPU_M5150: case CPU_QEMU_GENERIC: + case CPU_I6400: cpu_wait = r4k_wait; if (read_c0_config7() & MIPS_CONF7_WII) cpu_wait = r4k_wait_irqoff; diff --git a/arch/mips/kernel/jump_label.c b/arch/mips/kernel/jump_label.c index dda800e9e73167..3e586daa3a3247 100644 --- a/arch/mips/kernel/jump_label.c +++ b/arch/mips/kernel/jump_label.c @@ -51,7 +51,7 @@ void arch_jump_label_transform(struct jump_entry *e, /* Target must have the right alignment and ISA must be preserved. */ BUG_ON((e->target & J_ALIGN_MASK) != J_ISA_BIT); - if (type == JUMP_LABEL_ENABLE) { + if (type == JUMP_LABEL_JMP) { insn.j_format.opcode = J_ISA_BIT ? mm_j32_op : j_op; insn.j_format.target = e->target >> J_RANGE_SHIFT; } else { diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c index 85bbe9b9675972..b8ceee576cdfce 100644 --- a/arch/mips/kernel/mips-cm.c +++ b/arch/mips/kernel/mips-cm.c @@ -15,11 +15,131 @@ void __iomem *mips_cm_base; void __iomem *mips_cm_l2sync_base; +int mips_cm_is64; + +static char *cm2_tr[8] = { + "mem", "gcr", "gic", "mmio", + "0x04", "cpc", "0x06", "0x07" +}; + +/* CM3 Tag ECC transation type */ +static char *cm3_tr[16] = { + [0x0] = "ReqNoData", + [0x1] = "0x1", + [0x2] = "ReqWData", + [0x3] = "0x3", + [0x4] = "IReqNoResp", + [0x5] = "IReqWResp", + [0x6] = "IReqNoRespDat", + [0x7] = "IReqWRespDat", + [0x8] = "RespNoData", + [0x9] = "RespDataFol", + [0xa] = "RespWData", + [0xb] = "RespDataOnly", + [0xc] = "IRespNoData", + [0xd] = "IRespDataFol", + [0xe] = "IRespWData", + [0xf] = "IRespDataOnly" +}; + +static char *cm2_cmd[32] = { + [0x00] = "0x00", + [0x01] = "Legacy Write", + [0x02] = "Legacy Read", + [0x03] = "0x03", + [0x04] = "0x04", + [0x05] = "0x05", + [0x06] = "0x06", + [0x07] = "0x07", + [0x08] = "Coherent Read Own", + [0x09] = "Coherent Read Share", + [0x0a] = "Coherent Read Discard", + [0x0b] = "Coherent Ready Share Always", + [0x0c] = "Coherent Upgrade", + [0x0d] = "Coherent Writeback", + [0x0e] = "0x0e", + [0x0f] = "0x0f", + [0x10] = "Coherent Copyback", + [0x11] = "Coherent Copyback Invalidate", + [0x12] = "Coherent Invalidate", + [0x13] = "Coherent Write Invalidate", + [0x14] = "Coherent Completion Sync", + [0x15] = "0x15", + [0x16] = "0x16", + [0x17] = "0x17", + [0x18] = "0x18", + [0x19] = "0x19", + [0x1a] = "0x1a", + [0x1b] = "0x1b", + [0x1c] = "0x1c", + [0x1d] = "0x1d", + [0x1e] = "0x1e", + [0x1f] = "0x1f" +}; + +/* CM3 Tag ECC command type */ +static char *cm3_cmd[16] = { + [0x0] = "Legacy Read", + [0x1] = "Legacy Write", + [0x2] = "Coherent Read Own", + [0x3] = "Coherent Read Share", + [0x4] = "Coherent Read Discard", + [0x5] = "Coherent Evicted", + [0x6] = "Coherent Upgrade", + [0x7] = "Coherent Upgrade for Store Conditional", + [0x8] = "Coherent Writeback", + [0x9] = "Coherent Write Invalidate", + [0xa] = "0xa", + [0xb] = "0xb", + [0xc] = "0xc", + [0xd] = "0xd", + [0xe] = "0xe", + [0xf] = "0xf" +}; + +/* CM3 Tag ECC command group */ +static char *cm3_cmd_group[8] = { + [0x0] = "Normal", + [0x1] = "Registers", + [0x2] = "TLB", + [0x3] = "0x3", + [0x4] = "L1I", + [0x5] = "L1D", + [0x6] = "L3", + [0x7] = "L2" +}; + +static char *cm2_core[8] = { + "Invalid/OK", "Invalid/Data", + "Shared/OK", "Shared/Data", + "Modified/OK", "Modified/Data", + "Exclusive/OK", "Exclusive/Data" +}; + +static char *cm2_causes[32] = { + "None", "GC_WR_ERR", "GC_RD_ERR", "COH_WR_ERR", + "COH_RD_ERR", "MMIO_WR_ERR", "MMIO_RD_ERR", "0x07", + "0x08", "0x09", "0x0a", "0x0b", + "0x0c", "0x0d", "0x0e", "0x0f", + "0x10", "0x11", "0x12", "0x13", + "0x14", "0x15", "0x16", "INTVN_WR_ERR", + "INTVN_RD_ERR", "0x19", "0x1a", "0x1b", + "0x1c", "0x1d", "0x1e", "0x1f" +}; + +static char *cm3_causes[32] = { + "0x0", "MP_CORRECTABLE_ECC_ERR", "MP_REQUEST_DECODE_ERR", + "MP_UNCORRECTABLE_ECC_ERR", "MP_PARITY_ERR", "MP_COHERENCE_ERR", + "CMBIU_REQUEST_DECODE_ERR", "CMBIU_PARITY_ERR", "CMBIU_AXI_RESP_ERR", + "0x9", "RBI_BUS_ERR", "0xb", "0xc", "0xd", "0xe", "0xf", "0x10", + "0x11", "0x12", "0x13", "0x14", "0x15", "0x16", "0x17", "0x18", + "0x19", "0x1a", "0x1b", "0x1c", "0x1d", "0x1e", "0x1f" +}; phys_addr_t __mips_cm_phys_base(void) { u32 config3 = read_c0_config3(); - u32 cmgcr; + unsigned long cmgcr; /* Check the CMGCRBase register is implemented */ if (!(config3 & MIPS_CONF3_CMGCR)) @@ -81,6 +201,13 @@ int mips_cm_probe(void) phys_addr_t addr; u32 base_reg; + /* + * No need to probe again if we have already been + * here before. + */ + if (mips_cm_base) + return 0; + addr = mips_cm_phys_base(); BUG_ON((addr & CM_GCR_BASE_GCRBASE_MSK) != addr); if (!addr) @@ -117,5 +244,133 @@ int mips_cm_probe(void) /* probe for an L2-only sync region */ mips_cm_probe_l2sync(); + /* determine register width for this CM */ + mips_cm_is64 = config_enabled(CONFIG_64BIT) && (mips_cm_revision() >= CM_REV_CM3); + return 0; } + +void mips_cm_error_report(void) +{ + unsigned long revision = mips_cm_revision(); + /* + * CM3 has a 64-bit Error cause register with 0:57 containing the error + * info and 63:58 the error type. For old CMs, everything is contained + * in a single 32-bit register (0:26 and 31:27 respectively). Even + * though the cm_error is u64, we will simply ignore the upper word + * for CM2. + */ + u64 cm_error = read_gcr_error_cause(); + int cm_error_cause_sft = CM_GCR_ERROR_CAUSE_ERRTYPE_SHF + + ((revision >= CM_REV_CM3) ? 31 : 0); + unsigned long cm_addr = read_gcr_error_addr(); + unsigned long cm_other = read_gcr_error_mult(); + int ocause, cause; + char buf[256]; + + if (!mips_cm_present()) + return; + + cause = cm_error >> cm_error_cause_sft; + + if (!cause) + /* All good */ + return; + + ocause = cm_other >> CM_GCR_ERROR_MULT_ERR2ND_SHF; + if (revision < CM_REV_CM3) { /* CM2 */ + if (cause < 16) { + unsigned long cca_bits = (cm_error >> 15) & 7; + unsigned long tr_bits = (cm_error >> 12) & 7; + unsigned long cmd_bits = (cm_error >> 7) & 0x1f; + unsigned long stag_bits = (cm_error >> 3) & 15; + unsigned long sport_bits = (cm_error >> 0) & 7; + + snprintf(buf, sizeof(buf), + "CCA=%lu TR=%s MCmd=%s STag=%lu " + "SPort=%lu\n", cca_bits, cm2_tr[tr_bits], + cm2_cmd[cmd_bits], stag_bits, sport_bits); + } else { + /* glob state & sresp together */ + unsigned long c3_bits = (cm_error >> 18) & 7; + unsigned long c2_bits = (cm_error >> 15) & 7; + unsigned long c1_bits = (cm_error >> 12) & 7; + unsigned long c0_bits = (cm_error >> 9) & 7; + unsigned long sc_bit = (cm_error >> 8) & 1; + unsigned long cmd_bits = (cm_error >> 3) & 0x1f; + unsigned long sport_bits = (cm_error >> 0) & 7; + + snprintf(buf, sizeof(buf), + "C3=%s C2=%s C1=%s C0=%s SC=%s " + "MCmd=%s SPort=%lu\n", + cm2_core[c3_bits], cm2_core[c2_bits], + cm2_core[c1_bits], cm2_core[c0_bits], + sc_bit ? "True" : "False", + cm2_cmd[cmd_bits], sport_bits); + } + pr_err("CM_ERROR=%08llx %s <%s>\n", cm_error, + cm2_causes[cause], buf); + pr_err("CM_ADDR =%08lx\n", cm_addr); + pr_err("CM_OTHER=%08lx %s\n", cm_other, cm2_causes[ocause]); + } else { /* CM3 */ + /* Used by cause == {1,2,3} */ + unsigned long core_id_bits = (cm_error >> 22) & 0xf; + unsigned long vp_id_bits = (cm_error >> 18) & 0xf; + unsigned long cmd_bits = (cm_error >> 14) & 0xf; + unsigned long cmd_group_bits = (cm_error >> 11) & 0xf; + unsigned long cm3_cca_bits = (cm_error >> 8) & 7; + unsigned long mcp_bits = (cm_error >> 5) & 0xf; + unsigned long cm3_tr_bits = (cm_error >> 1) & 0xf; + unsigned long sched_bit = cm_error & 0x1; + + if (cause == 1 || cause == 3) { /* Tag ECC */ + unsigned long tag_ecc = (cm_error >> 57) & 0x1; + unsigned long tag_way_bits = (cm_error >> 29) & 0xffff; + unsigned long dword_bits = (cm_error >> 49) & 0xff; + unsigned long data_way_bits = (cm_error >> 45) & 0xf; + unsigned long data_sets_bits = (cm_error >> 29) & 0xfff; + unsigned long bank_bit = (cm_error >> 28) & 0x1; + snprintf(buf, sizeof(buf), + "%s ECC Error: Way=%lu (DWORD=%lu, Sets=%lu)" + "Bank=%lu CoreID=%lu VPID=%lu Command=%s" + "Command Group=%s CCA=%lu MCP=%d" + "Transaction type=%s Scheduler=%lu\n", + tag_ecc ? "TAG" : "DATA", + tag_ecc ? (unsigned long)ffs(tag_way_bits) - 1 : + data_way_bits, bank_bit, dword_bits, + data_sets_bits, + core_id_bits, vp_id_bits, + cm3_cmd[cmd_bits], + cm3_cmd_group[cmd_group_bits], + cm3_cca_bits, 1 << mcp_bits, + cm3_tr[cm3_tr_bits], sched_bit); + } else if (cause == 2) { + unsigned long data_error_type = (cm_error >> 41) & 0xfff; + unsigned long data_decode_cmd = (cm_error >> 37) & 0xf; + unsigned long data_decode_group = (cm_error >> 34) & 0x7; + unsigned long data_decode_destination_id = (cm_error >> 28) & 0x3f; + + snprintf(buf, sizeof(buf), + "Decode Request Error: Type=%lu, Command=%lu" + "Command Group=%lu Destination ID=%lu" + "CoreID=%lu VPID=%lu Command=%s" + "Command Group=%s CCA=%lu MCP=%d" + "Transaction type=%s Scheduler=%lu\n", + data_error_type, data_decode_cmd, + data_decode_group, data_decode_destination_id, + core_id_bits, vp_id_bits, + cm3_cmd[cmd_bits], + cm3_cmd_group[cmd_group_bits], + cm3_cca_bits, 1 << mcp_bits, + cm3_tr[cm3_tr_bits], sched_bit); + } + + pr_err("CM_ERROR=%llx %s <%s>\n", cm_error, + cm3_causes[cause], buf); + pr_err("CM_ADDR =%lx\n", cm_addr); + pr_err("CM_OTHER=%lx %s\n", cm_other, cm3_causes[ocause]); + } + + /* reprime cause register */ + write_gcr_error_cause(0); +} diff --git a/arch/mips/kernel/mips-cpc.c b/arch/mips/kernel/mips-cpc.c index 11964501c4b056..8af4d627b68b96 100644 --- a/arch/mips/kernel/mips-cpc.c +++ b/arch/mips/kernel/mips-cpc.c @@ -21,9 +21,16 @@ static DEFINE_PER_CPU_ALIGNED(spinlock_t, cpc_core_lock); static DEFINE_PER_CPU_ALIGNED(unsigned long, cpc_core_lock_flags); -phys_addr_t __weak mips_cpc_phys_base(void) +/** + * mips_cpc_phys_base - retrieve the physical base address of the CPC + * + * This function returns the physical base address of the Cluster Power + * Controller memory mapped registers, or 0 if no Cluster Power Controller + * is present. + */ +static phys_addr_t mips_cpc_phys_base(void) { - u32 cpc_base; + unsigned long cpc_base; if (!mips_cm_present()) return 0; diff --git a/arch/mips/kernel/perf_event_mipsxx.c b/arch/mips/kernel/perf_event_mipsxx.c index cc1b6fadf08989..d7b8dd43147a44 100644 --- a/arch/mips/kernel/perf_event_mipsxx.c +++ b/arch/mips/kernel/perf_event_mipsxx.c @@ -1556,6 +1556,7 @@ static const struct mips_perf_event *mipsxx_pmu_map_raw_event(u64 config) #endif break; case CPU_P5600: + case CPU_I6400: /* 8-bit event numbers */ raw_id = config & 0x1ff; base_id = raw_id & 0xff; @@ -1717,6 +1718,11 @@ init_hw_perf_events(void) mipspmu.general_event_map = &mipsxxcore_event_map2; mipspmu.cache_event_map = &mipsxxcore_cache_map2; break; + case CPU_I6400: + mipspmu.name = "mips/I6400"; + mipspmu.general_event_map = &mipsxxcore_event_map2; + mipspmu.cache_event_map = &mipsxxcore_cache_map2; + break; case CPU_1004K: mipspmu.name = "mips/1004K"; mipspmu.general_event_map = &mipsxxcore_event_map; diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c index 06147179a175b7..f63a289977cc5f 100644 --- a/arch/mips/kernel/pm-cps.c +++ b/arch/mips/kernel/pm-cps.c @@ -267,6 +267,7 @@ static int __init cps_gen_flush_fsb(u32 **pp, struct uasm_label **pl, /* CPUs which do not require the workaround */ case CPU_P5600: + case CPU_I6400: return 0; default: @@ -671,6 +672,7 @@ static int __init cps_pm_init(void) case CPU_PROAPTIV: case CPU_M5150: case CPU_P5600: + case CPU_I6400: stype_intervention = 0x2; stype_memory = 0x3; stype_ordering = 0x10; diff --git a/arch/mips/kernel/ptrace.c b/arch/mips/kernel/ptrace.c index e933a309f2ea5f..4f0ac78d17f196 100644 --- a/arch/mips/kernel/ptrace.c +++ b/arch/mips/kernel/ptrace.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -490,6 +491,93 @@ enum mips_regset { REGSET_FPR, }; +struct pt_regs_offset { + const char *name; + int offset; +}; + +#define REG_OFFSET_NAME(reg, r) { \ + .name = #reg, \ + .offset = offsetof(struct pt_regs, r) \ +} + +#define REG_OFFSET_END { \ + .name = NULL, \ + .offset = 0 \ +} + +static const struct pt_regs_offset regoffset_table[] = { + REG_OFFSET_NAME(r0, regs[0]), + REG_OFFSET_NAME(r1, regs[1]), + REG_OFFSET_NAME(r2, regs[2]), + REG_OFFSET_NAME(r3, regs[3]), + REG_OFFSET_NAME(r4, regs[4]), + REG_OFFSET_NAME(r5, regs[5]), + REG_OFFSET_NAME(r6, regs[6]), + REG_OFFSET_NAME(r7, regs[7]), + REG_OFFSET_NAME(r8, regs[8]), + REG_OFFSET_NAME(r9, regs[9]), + REG_OFFSET_NAME(r10, regs[10]), + REG_OFFSET_NAME(r11, regs[11]), + REG_OFFSET_NAME(r12, regs[12]), + REG_OFFSET_NAME(r13, regs[13]), + REG_OFFSET_NAME(r14, regs[14]), + REG_OFFSET_NAME(r15, regs[15]), + REG_OFFSET_NAME(r16, regs[16]), + REG_OFFSET_NAME(r17, regs[17]), + REG_OFFSET_NAME(r18, regs[18]), + REG_OFFSET_NAME(r19, regs[19]), + REG_OFFSET_NAME(r20, regs[20]), + REG_OFFSET_NAME(r21, regs[21]), + REG_OFFSET_NAME(r22, regs[22]), + REG_OFFSET_NAME(r23, regs[23]), + REG_OFFSET_NAME(r24, regs[24]), + REG_OFFSET_NAME(r25, regs[25]), + REG_OFFSET_NAME(r26, regs[26]), + REG_OFFSET_NAME(r27, regs[27]), + REG_OFFSET_NAME(r28, regs[28]), + REG_OFFSET_NAME(r29, regs[29]), + REG_OFFSET_NAME(r30, regs[30]), + REG_OFFSET_NAME(r31, regs[31]), + REG_OFFSET_NAME(c0_status, cp0_status), + REG_OFFSET_NAME(hi, hi), + REG_OFFSET_NAME(lo, lo), +#ifdef CONFIG_CPU_HAS_SMARTMIPS + REG_OFFSET_NAME(acx, acx), +#endif + REG_OFFSET_NAME(c0_badvaddr, cp0_badvaddr), + REG_OFFSET_NAME(c0_cause, cp0_cause), + REG_OFFSET_NAME(c0_epc, cp0_epc), +#ifdef CONFIG_MIPS_MT_SMTC + REG_OFFSET_NAME(c0_tcstatus, cp0_tcstatus), +#endif +#ifdef CONFIG_CPU_CAVIUM_OCTEON + REG_OFFSET_NAME(mpl0, mpl[0]), + REG_OFFSET_NAME(mpl1, mpl[1]), + REG_OFFSET_NAME(mpl2, mpl[2]), + REG_OFFSET_NAME(mtp0, mtp[0]), + REG_OFFSET_NAME(mtp1, mtp[1]), + REG_OFFSET_NAME(mtp2, mtp[2]), +#endif + REG_OFFSET_END, +}; + +/** + * regs_query_register_offset() - query register offset from its name + * @name: the name of a register + * + * regs_query_register_offset() returns the offset of a register in struct + * pt_regs from its name. If the name is invalid, this returns -EINVAL; + */ +int regs_query_register_offset(const char *name) +{ + const struct pt_regs_offset *roff; + for (roff = regoffset_table; roff->name != NULL; roff++) + if (!strcmp(roff->name, name)) + return roff->offset; + return -EINVAL; +} + #if defined(CONFIG_32BIT) || defined(CONFIG_MIPS32_O32) static const struct user_regset mips_regsets[] = { diff --git a/arch/mips/kernel/r4k_fpu.S b/arch/mips/kernel/r4k_fpu.S index 1d88af26ba82a0..f09546ee2cdc90 100644 --- a/arch/mips/kernel/r4k_fpu.S +++ b/arch/mips/kernel/r4k_fpu.S @@ -13,6 +13,7 @@ * Copyright (C) 1999, 2001 Silicon Graphics, Inc. */ #include +#include #include #include #include @@ -35,6 +36,14 @@ .set noreorder +/** + * _save_fp_context() - save FP context from the FPU + * @a0 - pointer to fpregs field of sigcontext + * @a1 - pointer to fpc_csr field of sigcontext + * + * Save FP context, including the 32 FP data registers and the FP + * control & status register, from the FPU to signal context. + */ LEAF(_save_fp_context) .set push SET_HARDFLOAT @@ -54,117 +63,60 @@ LEAF(_save_fp_context) nop #endif /* Store the 16 odd double precision registers */ - EX sdc1 $f1, SC_FPREGS+8(a0) - EX sdc1 $f3, SC_FPREGS+24(a0) - EX sdc1 $f5, SC_FPREGS+40(a0) - EX sdc1 $f7, SC_FPREGS+56(a0) - EX sdc1 $f9, SC_FPREGS+72(a0) - EX sdc1 $f11, SC_FPREGS+88(a0) - EX sdc1 $f13, SC_FPREGS+104(a0) - EX sdc1 $f15, SC_FPREGS+120(a0) - EX sdc1 $f17, SC_FPREGS+136(a0) - EX sdc1 $f19, SC_FPREGS+152(a0) - EX sdc1 $f21, SC_FPREGS+168(a0) - EX sdc1 $f23, SC_FPREGS+184(a0) - EX sdc1 $f25, SC_FPREGS+200(a0) - EX sdc1 $f27, SC_FPREGS+216(a0) - EX sdc1 $f29, SC_FPREGS+232(a0) - EX sdc1 $f31, SC_FPREGS+248(a0) + EX sdc1 $f1, 8(a0) + EX sdc1 $f3, 24(a0) + EX sdc1 $f5, 40(a0) + EX sdc1 $f7, 56(a0) + EX sdc1 $f9, 72(a0) + EX sdc1 $f11, 88(a0) + EX sdc1 $f13, 104(a0) + EX sdc1 $f15, 120(a0) + EX sdc1 $f17, 136(a0) + EX sdc1 $f19, 152(a0) + EX sdc1 $f21, 168(a0) + EX sdc1 $f23, 184(a0) + EX sdc1 $f25, 200(a0) + EX sdc1 $f27, 216(a0) + EX sdc1 $f29, 232(a0) + EX sdc1 $f31, 248(a0) 1: .set pop #endif .set push SET_HARDFLOAT /* Store the 16 even double precision registers */ - EX sdc1 $f0, SC_FPREGS+0(a0) - EX sdc1 $f2, SC_FPREGS+16(a0) - EX sdc1 $f4, SC_FPREGS+32(a0) - EX sdc1 $f6, SC_FPREGS+48(a0) - EX sdc1 $f8, SC_FPREGS+64(a0) - EX sdc1 $f10, SC_FPREGS+80(a0) - EX sdc1 $f12, SC_FPREGS+96(a0) - EX sdc1 $f14, SC_FPREGS+112(a0) - EX sdc1 $f16, SC_FPREGS+128(a0) - EX sdc1 $f18, SC_FPREGS+144(a0) - EX sdc1 $f20, SC_FPREGS+160(a0) - EX sdc1 $f22, SC_FPREGS+176(a0) - EX sdc1 $f24, SC_FPREGS+192(a0) - EX sdc1 $f26, SC_FPREGS+208(a0) - EX sdc1 $f28, SC_FPREGS+224(a0) - EX sdc1 $f30, SC_FPREGS+240(a0) - EX sw t1, SC_FPC_CSR(a0) + EX sdc1 $f0, 0(a0) + EX sdc1 $f2, 16(a0) + EX sdc1 $f4, 32(a0) + EX sdc1 $f6, 48(a0) + EX sdc1 $f8, 64(a0) + EX sdc1 $f10, 80(a0) + EX sdc1 $f12, 96(a0) + EX sdc1 $f14, 112(a0) + EX sdc1 $f16, 128(a0) + EX sdc1 $f18, 144(a0) + EX sdc1 $f20, 160(a0) + EX sdc1 $f22, 176(a0) + EX sdc1 $f24, 192(a0) + EX sdc1 $f26, 208(a0) + EX sdc1 $f28, 224(a0) + EX sdc1 $f30, 240(a0) + EX sw t1, 0(a1) jr ra li v0, 0 # success .set pop END(_save_fp_context) -#ifdef CONFIG_MIPS32_COMPAT - /* Save 32-bit process floating point context */ -LEAF(_save_fp_context32) - .set push - .set MIPS_ISA_ARCH_LEVEL_RAW - SET_HARDFLOAT - cfc1 t1, fcr31 - -#ifndef CONFIG_CPU_MIPS64_R6 - mfc0 t0, CP0_STATUS - sll t0, t0, 5 - bgez t0, 1f # skip storing odd if FR=0 - nop -#endif - - /* Store the 16 odd double precision registers */ - EX sdc1 $f1, SC32_FPREGS+8(a0) - EX sdc1 $f3, SC32_FPREGS+24(a0) - EX sdc1 $f5, SC32_FPREGS+40(a0) - EX sdc1 $f7, SC32_FPREGS+56(a0) - EX sdc1 $f9, SC32_FPREGS+72(a0) - EX sdc1 $f11, SC32_FPREGS+88(a0) - EX sdc1 $f13, SC32_FPREGS+104(a0) - EX sdc1 $f15, SC32_FPREGS+120(a0) - EX sdc1 $f17, SC32_FPREGS+136(a0) - EX sdc1 $f19, SC32_FPREGS+152(a0) - EX sdc1 $f21, SC32_FPREGS+168(a0) - EX sdc1 $f23, SC32_FPREGS+184(a0) - EX sdc1 $f25, SC32_FPREGS+200(a0) - EX sdc1 $f27, SC32_FPREGS+216(a0) - EX sdc1 $f29, SC32_FPREGS+232(a0) - EX sdc1 $f31, SC32_FPREGS+248(a0) - - /* Store the 16 even double precision registers */ -1: EX sdc1 $f0, SC32_FPREGS+0(a0) - EX sdc1 $f2, SC32_FPREGS+16(a0) - EX sdc1 $f4, SC32_FPREGS+32(a0) - EX sdc1 $f6, SC32_FPREGS+48(a0) - EX sdc1 $f8, SC32_FPREGS+64(a0) - EX sdc1 $f10, SC32_FPREGS+80(a0) - EX sdc1 $f12, SC32_FPREGS+96(a0) - EX sdc1 $f14, SC32_FPREGS+112(a0) - EX sdc1 $f16, SC32_FPREGS+128(a0) - EX sdc1 $f18, SC32_FPREGS+144(a0) - EX sdc1 $f20, SC32_FPREGS+160(a0) - EX sdc1 $f22, SC32_FPREGS+176(a0) - EX sdc1 $f24, SC32_FPREGS+192(a0) - EX sdc1 $f26, SC32_FPREGS+208(a0) - EX sdc1 $f28, SC32_FPREGS+224(a0) - EX sdc1 $f30, SC32_FPREGS+240(a0) - EX sw t1, SC32_FPC_CSR(a0) - cfc1 t0, $0 # implementation/version - EX sw t0, SC32_FPC_EIR(a0) - .set pop - - jr ra - li v0, 0 # success - END(_save_fp_context32) -#endif - -/* - * Restore FPU state: - * - fp gp registers - * - cp1 status/control register +/** + * _restore_fp_context() - restore FP context to the FPU + * @a0 - pointer to fpregs field of sigcontext + * @a1 - pointer to fpc_csr field of sigcontext + * + * Restore FP context, including the 32 FP data registers and the FP + * control & status register, from signal context to the FPU. */ LEAF(_restore_fp_context) - EX lw t1, SC_FPC_CSR(a0) + EX lw t1, 0(a1) #if defined(CONFIG_64BIT) || defined(CONFIG_CPU_MIPS32_R2) || \ defined(CONFIG_CPU_MIPS32_R6) @@ -178,101 +130,231 @@ LEAF(_restore_fp_context) bgez t0, 1f # skip loading odd if FR=0 nop #endif - EX ldc1 $f1, SC_FPREGS+8(a0) - EX ldc1 $f3, SC_FPREGS+24(a0) - EX ldc1 $f5, SC_FPREGS+40(a0) - EX ldc1 $f7, SC_FPREGS+56(a0) - EX ldc1 $f9, SC_FPREGS+72(a0) - EX ldc1 $f11, SC_FPREGS+88(a0) - EX ldc1 $f13, SC_FPREGS+104(a0) - EX ldc1 $f15, SC_FPREGS+120(a0) - EX ldc1 $f17, SC_FPREGS+136(a0) - EX ldc1 $f19, SC_FPREGS+152(a0) - EX ldc1 $f21, SC_FPREGS+168(a0) - EX ldc1 $f23, SC_FPREGS+184(a0) - EX ldc1 $f25, SC_FPREGS+200(a0) - EX ldc1 $f27, SC_FPREGS+216(a0) - EX ldc1 $f29, SC_FPREGS+232(a0) - EX ldc1 $f31, SC_FPREGS+248(a0) + EX ldc1 $f1, 8(a0) + EX ldc1 $f3, 24(a0) + EX ldc1 $f5, 40(a0) + EX ldc1 $f7, 56(a0) + EX ldc1 $f9, 72(a0) + EX ldc1 $f11, 88(a0) + EX ldc1 $f13, 104(a0) + EX ldc1 $f15, 120(a0) + EX ldc1 $f17, 136(a0) + EX ldc1 $f19, 152(a0) + EX ldc1 $f21, 168(a0) + EX ldc1 $f23, 184(a0) + EX ldc1 $f25, 200(a0) + EX ldc1 $f27, 216(a0) + EX ldc1 $f29, 232(a0) + EX ldc1 $f31, 248(a0) 1: .set pop #endif .set push SET_HARDFLOAT - EX ldc1 $f0, SC_FPREGS+0(a0) - EX ldc1 $f2, SC_FPREGS+16(a0) - EX ldc1 $f4, SC_FPREGS+32(a0) - EX ldc1 $f6, SC_FPREGS+48(a0) - EX ldc1 $f8, SC_FPREGS+64(a0) - EX ldc1 $f10, SC_FPREGS+80(a0) - EX ldc1 $f12, SC_FPREGS+96(a0) - EX ldc1 $f14, SC_FPREGS+112(a0) - EX ldc1 $f16, SC_FPREGS+128(a0) - EX ldc1 $f18, SC_FPREGS+144(a0) - EX ldc1 $f20, SC_FPREGS+160(a0) - EX ldc1 $f22, SC_FPREGS+176(a0) - EX ldc1 $f24, SC_FPREGS+192(a0) - EX ldc1 $f26, SC_FPREGS+208(a0) - EX ldc1 $f28, SC_FPREGS+224(a0) - EX ldc1 $f30, SC_FPREGS+240(a0) + EX ldc1 $f0, 0(a0) + EX ldc1 $f2, 16(a0) + EX ldc1 $f4, 32(a0) + EX ldc1 $f6, 48(a0) + EX ldc1 $f8, 64(a0) + EX ldc1 $f10, 80(a0) + EX ldc1 $f12, 96(a0) + EX ldc1 $f14, 112(a0) + EX ldc1 $f16, 128(a0) + EX ldc1 $f18, 144(a0) + EX ldc1 $f20, 160(a0) + EX ldc1 $f22, 176(a0) + EX ldc1 $f24, 192(a0) + EX ldc1 $f26, 208(a0) + EX ldc1 $f28, 224(a0) + EX ldc1 $f30, 240(a0) ctc1 t1, fcr31 .set pop jr ra li v0, 0 # success END(_restore_fp_context) -#ifdef CONFIG_MIPS32_COMPAT -LEAF(_restore_fp_context32) - /* Restore an o32 sigcontext. */ - .set push - SET_HARDFLOAT - EX lw t1, SC32_FPC_CSR(a0) +#ifdef CONFIG_CPU_HAS_MSA -#ifndef CONFIG_CPU_MIPS64_R6 - mfc0 t0, CP0_STATUS - sll t0, t0, 5 - bgez t0, 1f # skip loading odd if FR=0 + .macro op_one_wr op, idx, base + .align 4 +\idx: \op \idx, 0, \base + jr ra nop -#endif + .endm - EX ldc1 $f1, SC32_FPREGS+8(a0) - EX ldc1 $f3, SC32_FPREGS+24(a0) - EX ldc1 $f5, SC32_FPREGS+40(a0) - EX ldc1 $f7, SC32_FPREGS+56(a0) - EX ldc1 $f9, SC32_FPREGS+72(a0) - EX ldc1 $f11, SC32_FPREGS+88(a0) - EX ldc1 $f13, SC32_FPREGS+104(a0) - EX ldc1 $f15, SC32_FPREGS+120(a0) - EX ldc1 $f17, SC32_FPREGS+136(a0) - EX ldc1 $f19, SC32_FPREGS+152(a0) - EX ldc1 $f21, SC32_FPREGS+168(a0) - EX ldc1 $f23, SC32_FPREGS+184(a0) - EX ldc1 $f25, SC32_FPREGS+200(a0) - EX ldc1 $f27, SC32_FPREGS+216(a0) - EX ldc1 $f29, SC32_FPREGS+232(a0) - EX ldc1 $f31, SC32_FPREGS+248(a0) + .macro op_msa_wr name, op +LEAF(\name) + .set push + .set noreorder + sll t0, a0, 4 + PTR_LA t1, 0f + PTR_ADDU t0, t0, t1 + jr t0 + nop + op_one_wr \op, 0, a1 + op_one_wr \op, 1, a1 + op_one_wr \op, 2, a1 + op_one_wr \op, 3, a1 + op_one_wr \op, 4, a1 + op_one_wr \op, 5, a1 + op_one_wr \op, 6, a1 + op_one_wr \op, 7, a1 + op_one_wr \op, 8, a1 + op_one_wr \op, 9, a1 + op_one_wr \op, 10, a1 + op_one_wr \op, 11, a1 + op_one_wr \op, 12, a1 + op_one_wr \op, 13, a1 + op_one_wr \op, 14, a1 + op_one_wr \op, 15, a1 + op_one_wr \op, 16, a1 + op_one_wr \op, 17, a1 + op_one_wr \op, 18, a1 + op_one_wr \op, 19, a1 + op_one_wr \op, 20, a1 + op_one_wr \op, 21, a1 + op_one_wr \op, 22, a1 + op_one_wr \op, 23, a1 + op_one_wr \op, 24, a1 + op_one_wr \op, 25, a1 + op_one_wr \op, 26, a1 + op_one_wr \op, 27, a1 + op_one_wr \op, 28, a1 + op_one_wr \op, 29, a1 + op_one_wr \op, 30, a1 + op_one_wr \op, 31, a1 + .set pop + END(\name) + .endm -1: EX ldc1 $f0, SC32_FPREGS+0(a0) - EX ldc1 $f2, SC32_FPREGS+16(a0) - EX ldc1 $f4, SC32_FPREGS+32(a0) - EX ldc1 $f6, SC32_FPREGS+48(a0) - EX ldc1 $f8, SC32_FPREGS+64(a0) - EX ldc1 $f10, SC32_FPREGS+80(a0) - EX ldc1 $f12, SC32_FPREGS+96(a0) - EX ldc1 $f14, SC32_FPREGS+112(a0) - EX ldc1 $f16, SC32_FPREGS+128(a0) - EX ldc1 $f18, SC32_FPREGS+144(a0) - EX ldc1 $f20, SC32_FPREGS+160(a0) - EX ldc1 $f22, SC32_FPREGS+176(a0) - EX ldc1 $f24, SC32_FPREGS+192(a0) - EX ldc1 $f26, SC32_FPREGS+208(a0) - EX ldc1 $f28, SC32_FPREGS+224(a0) - EX ldc1 $f30, SC32_FPREGS+240(a0) - ctc1 t1, fcr31 + op_msa_wr read_msa_wr_b, st_b + op_msa_wr read_msa_wr_h, st_h + op_msa_wr read_msa_wr_w, st_w + op_msa_wr read_msa_wr_d, st_d + + op_msa_wr write_msa_wr_b, ld_b + op_msa_wr write_msa_wr_h, ld_h + op_msa_wr write_msa_wr_w, ld_w + op_msa_wr write_msa_wr_d, ld_d + +#endif /* CONFIG_CPU_HAS_MSA */ + +#ifdef CONFIG_CPU_HAS_MSA + + .macro save_msa_upper wr, off, base + .set push + .set noat +#ifdef CONFIG_64BIT + copy_u_d \wr, 1 + EX sd $1, \off(\base) +#elif defined(CONFIG_CPU_LITTLE_ENDIAN) + copy_u_w \wr, 2 + EX sw $1, \off(\base) + copy_u_w \wr, 3 + EX sw $1, (\off+4)(\base) +#else /* CONFIG_CPU_BIG_ENDIAN */ + copy_u_w \wr, 2 + EX sw $1, (\off+4)(\base) + copy_u_w \wr, 3 + EX sw $1, \off(\base) +#endif + .set pop + .endm + +LEAF(_save_msa_all_upper) + save_msa_upper 0, 0x00, a0 + save_msa_upper 1, 0x08, a0 + save_msa_upper 2, 0x10, a0 + save_msa_upper 3, 0x18, a0 + save_msa_upper 4, 0x20, a0 + save_msa_upper 5, 0x28, a0 + save_msa_upper 6, 0x30, a0 + save_msa_upper 7, 0x38, a0 + save_msa_upper 8, 0x40, a0 + save_msa_upper 9, 0x48, a0 + save_msa_upper 10, 0x50, a0 + save_msa_upper 11, 0x58, a0 + save_msa_upper 12, 0x60, a0 + save_msa_upper 13, 0x68, a0 + save_msa_upper 14, 0x70, a0 + save_msa_upper 15, 0x78, a0 + save_msa_upper 16, 0x80, a0 + save_msa_upper 17, 0x88, a0 + save_msa_upper 18, 0x90, a0 + save_msa_upper 19, 0x98, a0 + save_msa_upper 20, 0xa0, a0 + save_msa_upper 21, 0xa8, a0 + save_msa_upper 22, 0xb0, a0 + save_msa_upper 23, 0xb8, a0 + save_msa_upper 24, 0xc0, a0 + save_msa_upper 25, 0xc8, a0 + save_msa_upper 26, 0xd0, a0 + save_msa_upper 27, 0xd8, a0 + save_msa_upper 28, 0xe0, a0 + save_msa_upper 29, 0xe8, a0 + save_msa_upper 30, 0xf0, a0 + save_msa_upper 31, 0xf8, a0 jr ra - li v0, 0 # success - .set pop - END(_restore_fp_context32) + li v0, 0 + END(_save_msa_all_upper) + + .macro restore_msa_upper wr, off, base + .set push + .set noat +#ifdef CONFIG_64BIT + EX ld $1, \off(\base) + insert_d \wr, 1 +#elif defined(CONFIG_CPU_LITTLE_ENDIAN) + EX lw $1, \off(\base) + insert_w \wr, 2 + EX lw $1, (\off+4)(\base) + insert_w \wr, 3 +#else /* CONFIG_CPU_BIG_ENDIAN */ + EX lw $1, (\off+4)(\base) + insert_w \wr, 2 + EX lw $1, \off(\base) + insert_w \wr, 3 #endif + .set pop + .endm + +LEAF(_restore_msa_all_upper) + restore_msa_upper 0, 0x00, a0 + restore_msa_upper 1, 0x08, a0 + restore_msa_upper 2, 0x10, a0 + restore_msa_upper 3, 0x18, a0 + restore_msa_upper 4, 0x20, a0 + restore_msa_upper 5, 0x28, a0 + restore_msa_upper 6, 0x30, a0 + restore_msa_upper 7, 0x38, a0 + restore_msa_upper 8, 0x40, a0 + restore_msa_upper 9, 0x48, a0 + restore_msa_upper 10, 0x50, a0 + restore_msa_upper 11, 0x58, a0 + restore_msa_upper 12, 0x60, a0 + restore_msa_upper 13, 0x68, a0 + restore_msa_upper 14, 0x70, a0 + restore_msa_upper 15, 0x78, a0 + restore_msa_upper 16, 0x80, a0 + restore_msa_upper 17, 0x88, a0 + restore_msa_upper 18, 0x90, a0 + restore_msa_upper 19, 0x98, a0 + restore_msa_upper 20, 0xa0, a0 + restore_msa_upper 21, 0xa8, a0 + restore_msa_upper 22, 0xb0, a0 + restore_msa_upper 23, 0xb8, a0 + restore_msa_upper 24, 0xc0, a0 + restore_msa_upper 25, 0xc8, a0 + restore_msa_upper 26, 0xd0, a0 + restore_msa_upper 27, 0xd8, a0 + restore_msa_upper 28, 0xe0, a0 + restore_msa_upper 29, 0xe8, a0 + restore_msa_upper 30, 0xf0, a0 + restore_msa_upper 31, 0xf8, a0 + jr ra + li v0, 0 + END(_restore_msa_all_upper) + +#endif /* CONFIG_CPU_HAS_MSA */ .set reorder diff --git a/arch/mips/kernel/r4k_switch.S b/arch/mips/kernel/r4k_switch.S index 04cbbde3521bc1..92cd0516ecf51e 100644 --- a/arch/mips/kernel/r4k_switch.S +++ b/arch/mips/kernel/r4k_switch.S @@ -34,7 +34,7 @@ #ifndef USE_ALTERNATE_RESUME_IMPL /* * task_struct *resume(task_struct *prev, task_struct *next, - * struct thread_info *next_ti, s32 fp_save) + * struct thread_info *next_ti) */ .align 5 LEAF(resume) @@ -43,45 +43,6 @@ cpu_save_nonscratch a0 LONG_S ra, THREAD_REG31(a0) - /* - * Check whether we need to save any FP context. FP context is saved - * iff the process has used the context with the scalar FPU or the MSA - * ASE in the current time slice, as indicated by _TIF_USEDFPU and - * _TIF_USEDMSA respectively. switch_to will have set fp_save - * accordingly to an FP_SAVE_ enum value. - */ - beqz a3, 2f - - /* - * We do. Clear the saved CU1 bit for prev, such that next time it is - * scheduled it will start in userland with the FPU disabled. If the - * task uses the FPU then it will be enabled again via the do_cpu trap. - * This allows us to lazily restore the FP context. - */ - PTR_L t3, TASK_THREAD_INFO(a0) - LONG_L t0, ST_OFF(t3) - li t1, ~ST0_CU1 - and t0, t0, t1 - LONG_S t0, ST_OFF(t3) - - /* Check whether we're saving scalar or vector context. */ - bgtz a3, 1f - - /* Save 128b MSA vector context + scalar FP control & status. */ - .set push - SET_HARDFLOAT - cfc1 t1, fcr31 - msa_save_all a0 - .set pop /* SET_HARDFLOAT */ - - sw t1, THREAD_FCR31(a0) - b 2f - -1: /* Save 32b/64b scalar FP context. */ - fpu_save_double a0 t0 t1 # c0_status passed in t0 - # clobbers t1 -2: - #if defined(CONFIG_CC_STACKPROTECTOR) && !defined(CONFIG_SMP) PTR_LA t8, __stack_chk_guard LONG_L t9, TASK_STACK_CANARY(a1) diff --git a/arch/mips/kernel/signal-common.h b/arch/mips/kernel/signal-common.h index 0b85f827cd1836..f50d48435c6803 100644 --- a/arch/mips/kernel/signal-common.h +++ b/arch/mips/kernel/signal-common.h @@ -31,4 +31,13 @@ extern int fpcsr_pending(unsigned int __user *fpcsr); #define lock_fpu_owner() ({ preempt_disable(); pagefault_disable(); }) #define unlock_fpu_owner() ({ pagefault_enable(); preempt_enable(); }) +/* Assembly functions to move context to/from the FPU */ +extern asmlinkage int +_save_fp_context(void __user *fpregs, void __user *csr); +extern asmlinkage int +_restore_fp_context(void __user *fpregs, void __user *csr); + +extern asmlinkage int _save_msa_all_upper(void __user *buf); +extern asmlinkage int _restore_msa_all_upper(void __user *buf); + #endif /* __SIGNAL_COMMON_H */ diff --git a/arch/mips/kernel/signal.c b/arch/mips/kernel/signal.c index 6a28c792d862c0..2fec67bfc457cc 100644 --- a/arch/mips/kernel/signal.c +++ b/arch/mips/kernel/signal.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -38,20 +39,21 @@ #include #include #include +#include #include "signal-common.h" -static int (*save_fp_context)(struct sigcontext __user *sc); -static int (*restore_fp_context)(struct sigcontext __user *sc); - -extern asmlinkage int _save_fp_context(struct sigcontext __user *sc); -extern asmlinkage int _restore_fp_context(struct sigcontext __user *sc); +static int (*save_fp_context)(void __user *sc); +static int (*restore_fp_context)(void __user *sc); struct sigframe { u32 sf_ass[4]; /* argument save space for o32 */ u32 sf_pad[2]; /* Was: signal trampoline */ + + /* Matches struct ucontext from its uc_mcontext field onwards */ struct sigcontext sf_sc; sigset_t sf_mask; + unsigned long long sf_extcontext[0]; }; struct rt_sigframe { @@ -65,43 +67,255 @@ struct rt_sigframe { * Thread saved context copy to/from a signal context presumed to be on the * user stack, and therefore accessed with appropriate macros from uaccess.h. */ -static int copy_fp_to_sigcontext(struct sigcontext __user *sc) +static int copy_fp_to_sigcontext(void __user *sc) { + struct mips_abi *abi = current->thread.abi; + uint64_t __user *fpregs = sc + abi->off_sc_fpregs; + uint32_t __user *csr = sc + abi->off_sc_fpc_csr; int i; int err = 0; + int inc = test_thread_flag(TIF_32BIT_FPREGS) ? 2 : 1; - for (i = 0; i < NUM_FPU_REGS; i++) { + for (i = 0; i < NUM_FPU_REGS; i += inc) { err |= __put_user(get_fpr64(¤t->thread.fpu.fpr[i], 0), - &sc->sc_fpregs[i]); + &fpregs[i]); } - err |= __put_user(current->thread.fpu.fcr31, &sc->sc_fpc_csr); + err |= __put_user(current->thread.fpu.fcr31, csr); return err; } -static int copy_fp_from_sigcontext(struct sigcontext __user *sc) +static int copy_fp_from_sigcontext(void __user *sc) { + struct mips_abi *abi = current->thread.abi; + uint64_t __user *fpregs = sc + abi->off_sc_fpregs; + uint32_t __user *csr = sc + abi->off_sc_fpc_csr; int i; int err = 0; + int inc = test_thread_flag(TIF_32BIT_FPREGS) ? 2 : 1; u64 fpr_val; - for (i = 0; i < NUM_FPU_REGS; i++) { - err |= __get_user(fpr_val, &sc->sc_fpregs[i]); + for (i = 0; i < NUM_FPU_REGS; i += inc) { + err |= __get_user(fpr_val, &fpregs[i]); set_fpr64(¤t->thread.fpu.fpr[i], 0, fpr_val); } - err |= __get_user(current->thread.fpu.fcr31, &sc->sc_fpc_csr); + err |= __get_user(current->thread.fpu.fcr31, csr); return err; } +/* + * Wrappers for the assembly _{save,restore}_fp_context functions. + */ +static int save_hw_fp_context(void __user *sc) +{ + struct mips_abi *abi = current->thread.abi; + uint64_t __user *fpregs = sc + abi->off_sc_fpregs; + uint32_t __user *csr = sc + abi->off_sc_fpc_csr; + + return _save_fp_context(fpregs, csr); +} + +static int restore_hw_fp_context(void __user *sc) +{ + struct mips_abi *abi = current->thread.abi; + uint64_t __user *fpregs = sc + abi->off_sc_fpregs; + uint32_t __user *csr = sc + abi->off_sc_fpc_csr; + + return _restore_fp_context(fpregs, csr); +} + +/* + * Extended context handling. + */ + +static inline void __user *sc_to_extcontext(void __user *sc) +{ + struct ucontext __user *uc; + + /* + * We can just pretend the sigcontext is always embedded in a struct + * ucontext here, because the offset from sigcontext to extended + * context is the same in the struct sigframe case. + */ + uc = container_of(sc, struct ucontext, uc_mcontext); + return &uc->uc_extcontext; +} + +static int save_msa_extcontext(void __user *buf) +{ + struct msa_extcontext __user *msa = buf; + uint64_t val; + int i, err; + + if (!thread_msa_context_live()) + return 0; + + /* + * Ensure that we can't lose the live MSA context between checking + * for it & writing it to memory. + */ + preempt_disable(); + + if (is_msa_enabled()) { + /* + * There are no EVA versions of the vector register load/store + * instructions, so MSA context has to be saved to kernel memory + * and then copied to user memory. The save to kernel memory + * should already have been done when handling scalar FP + * context. + */ + BUG_ON(config_enabled(CONFIG_EVA)); + + err = __put_user(read_msa_csr(), &msa->csr); + err |= _save_msa_all_upper(&msa->wr); + + preempt_enable(); + } else { + preempt_enable(); + + err = __put_user(current->thread.fpu.msacsr, &msa->csr); + + for (i = 0; i < NUM_FPU_REGS; i++) { + val = get_fpr64(¤t->thread.fpu.fpr[i], 1); + err |= __put_user(val, &msa->wr[i]); + } + } + + err |= __put_user(MSA_EXTCONTEXT_MAGIC, &msa->ext.magic); + err |= __put_user(sizeof(*msa), &msa->ext.size); + + return err ? -EFAULT : sizeof(*msa); +} + +static int restore_msa_extcontext(void __user *buf, unsigned int size) +{ + struct msa_extcontext __user *msa = buf; + unsigned long long val; + unsigned int csr; + int i, err; + + if (size != sizeof(*msa)) + return -EINVAL; + + err = get_user(csr, &msa->csr); + if (err) + return err; + + preempt_disable(); + + if (is_msa_enabled()) { + /* + * There are no EVA versions of the vector register load/store + * instructions, so MSA context has to be copied to kernel + * memory and later loaded to registers. The same is true of + * scalar FP context, so FPU & MSA should have already been + * disabled whilst handling scalar FP context. + */ + BUG_ON(config_enabled(CONFIG_EVA)); + + write_msa_csr(csr); + err |= _restore_msa_all_upper(&msa->wr); + preempt_enable(); + } else { + preempt_enable(); + + current->thread.fpu.msacsr = csr; + + for (i = 0; i < NUM_FPU_REGS; i++) { + err |= __get_user(val, &msa->wr[i]); + set_fpr64(¤t->thread.fpu.fpr[i], 1, val); + } + } + + return err; +} + +static int save_extcontext(void __user *buf) +{ + int sz; + + sz = save_msa_extcontext(buf); + if (sz < 0) + return sz; + buf += sz; + + /* If no context was saved then trivially return */ + if (!sz) + return 0; + + /* Write the end marker */ + if (__put_user(END_EXTCONTEXT_MAGIC, (u32 *)buf)) + return -EFAULT; + + sz += sizeof(((struct extcontext *)NULL)->magic); + return sz; +} + +static int restore_extcontext(void __user *buf) +{ + struct extcontext ext; + int err; + + while (1) { + err = __get_user(ext.magic, (unsigned int *)buf); + if (err) + return err; + + if (ext.magic == END_EXTCONTEXT_MAGIC) + return 0; + + err = __get_user(ext.size, (unsigned int *)(buf + + offsetof(struct extcontext, size))); + if (err) + return err; + + switch (ext.magic) { + case MSA_EXTCONTEXT_MAGIC: + err = restore_msa_extcontext(buf, ext.size); + break; + + default: + err = -EINVAL; + break; + } + + if (err) + return err; + + buf += ext.size; + } +} + /* * Helper routines */ -static int protected_save_fp_context(struct sigcontext __user *sc) +int protected_save_fp_context(void __user *sc) { + struct mips_abi *abi = current->thread.abi; + uint64_t __user *fpregs = sc + abi->off_sc_fpregs; + uint32_t __user *csr = sc + abi->off_sc_fpc_csr; + uint32_t __user *used_math = sc + abi->off_sc_used_math; + unsigned int used, ext_sz; int err; -#ifndef CONFIG_EVA + + used = used_math() ? USED_FP : 0; + if (!used) + goto fp_done; + + if (!test_thread_flag(TIF_32BIT_FPREGS)) + used |= USED_FR1; + if (test_thread_flag(TIF_HYBRID_FPREGS)) + used |= USED_HYBRID_FPRS; + + /* + * EVA does not have userland equivalents of ldc1 or sdc1, so + * save to the kernel FP context & copy that to userland below. + */ + if (config_enabled(CONFIG_EVA)) + lose_fpu(1); + while (1) { lock_fpu_owner(); if (is_fpu_owner()) { @@ -114,27 +328,57 @@ static int protected_save_fp_context(struct sigcontext __user *sc) if (likely(!err)) break; /* touch the sigcontext and try again */ - err = __put_user(0, &sc->sc_fpregs[0]) | - __put_user(0, &sc->sc_fpregs[31]) | - __put_user(0, &sc->sc_fpc_csr); + err = __put_user(0, &fpregs[0]) | + __put_user(0, &fpregs[31]) | + __put_user(0, csr); if (err) - break; /* really bad sigcontext */ + return err; /* really bad sigcontext */ } -#else - /* - * EVA does not have FPU EVA instructions so saving fpu context directly - * does not work. - */ - lose_fpu(1); - err = save_fp_context(sc); /* this might fail */ -#endif - return err; + +fp_done: + ext_sz = err = save_extcontext(sc_to_extcontext(sc)); + if (err < 0) + return err; + used |= ext_sz ? USED_EXTCONTEXT : 0; + + return __put_user(used, used_math); } -static int protected_restore_fp_context(struct sigcontext __user *sc) +int protected_restore_fp_context(void __user *sc) { - int err, tmp __maybe_unused; -#ifndef CONFIG_EVA + struct mips_abi *abi = current->thread.abi; + uint64_t __user *fpregs = sc + abi->off_sc_fpregs; + uint32_t __user *csr = sc + abi->off_sc_fpc_csr; + uint32_t __user *used_math = sc + abi->off_sc_used_math; + unsigned int used; + int err, sig = 0, tmp __maybe_unused; + + err = __get_user(used, used_math); + conditional_used_math(used & USED_FP); + + /* + * The signal handler may have used FPU; give it up if the program + * doesn't want it following sigreturn. + */ + if (err || !(used & USED_FP)) + lose_fpu(0); + if (err) + return err; + if (!(used & USED_FP)) + goto fp_done; + + err = sig = fpcsr_pending(csr); + if (err < 0) + return err; + + /* + * EVA does not have userland equivalents of ldc1 or sdc1, so we + * disable the FPU here such that the code below simply copies to + * the kernel FP context. + */ + if (config_enabled(CONFIG_EVA)) + lose_fpu(0); + while (1) { lock_fpu_owner(); if (is_fpu_owner()) { @@ -147,28 +391,24 @@ static int protected_restore_fp_context(struct sigcontext __user *sc) if (likely(!err)) break; /* touch the sigcontext and try again */ - err = __get_user(tmp, &sc->sc_fpregs[0]) | - __get_user(tmp, &sc->sc_fpregs[31]) | - __get_user(tmp, &sc->sc_fpc_csr); + err = __get_user(tmp, &fpregs[0]) | + __get_user(tmp, &fpregs[31]) | + __get_user(tmp, csr); if (err) break; /* really bad sigcontext */ } -#else - /* - * EVA does not have FPU EVA instructions so restoring fpu context - * directly does not work. - */ - lose_fpu(0); - err = restore_fp_context(sc); /* this might fail */ -#endif - return err; + +fp_done: + if (used & USED_EXTCONTEXT) + err |= restore_extcontext(sc_to_extcontext(sc)); + + return err ?: sig; } int setup_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc) { int err = 0; int i; - unsigned int used_math; err |= __put_user(regs->cp0_epc, &sc->sc_pc); @@ -191,19 +431,38 @@ int setup_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc) err |= __put_user(rddsp(DSP_MASK), &sc->sc_dsp); } - used_math = !!used_math(); - err |= __put_user(used_math, &sc->sc_used_math); - if (used_math) { - /* - * Save FPU state to signal context. Signal handler - * will "inherit" current FPU state. - */ - err |= protected_save_fp_context(sc); - } + /* + * Save FPU state to signal context. Signal handler + * will "inherit" current FPU state. + */ + err |= protected_save_fp_context(sc); + return err; } +static size_t extcontext_max_size(void) +{ + size_t sz = 0; + + /* + * The assumption here is that between this point & the point at which + * the extended context is saved the size of the context should only + * ever be able to shrink (if the task is preempted), but never grow. + * That is, what this function returns is an upper bound on the size of + * the extended context for the current task at the current time. + */ + + if (thread_msa_context_live()) + sz += sizeof(struct msa_extcontext); + + /* If any context is saved then we'll append the end marker */ + if (sz) + sz += sizeof(((struct extcontext *)NULL)->magic); + + return sz; +} + int fpcsr_pending(unsigned int __user *fpcsr) { int err, sig = 0; @@ -223,21 +482,8 @@ int fpcsr_pending(unsigned int __user *fpcsr) return err ?: sig; } -static int -check_and_restore_fp_context(struct sigcontext __user *sc) -{ - int err, sig; - - err = sig = fpcsr_pending(&sc->sc_fpc_csr); - if (err > 0) - err = 0; - err |= protected_restore_fp_context(sc); - return err ?: sig; -} - int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc) { - unsigned int used_math; unsigned long treg; int err = 0; int i; @@ -265,19 +511,7 @@ int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc) for (i = 1; i < 32; i++) err |= __get_user(regs->regs[i], &sc->sc_regs[i]); - err |= __get_user(used_math, &sc->sc_used_math); - conditional_used_math(used_math); - - if (used_math) { - /* restore fpu context if we have used it before */ - if (!err) - err = check_and_restore_fp_context(sc); - } else { - /* signal handler may have used FPU. Give it up. */ - lose_fpu(0); - } - - return err; + return err ?: protected_restore_fp_context(sc); } void __user *get_sigframe(struct ksignal *ksig, struct pt_regs *regs, @@ -285,6 +519,9 @@ void __user *get_sigframe(struct ksignal *ksig, struct pt_regs *regs, { unsigned long sp; + /* Leave space for potential extended context */ + frame_size += extcontext_max_size(); + /* Default to using normal stack */ sp = regs->regs[29]; @@ -520,7 +757,11 @@ struct mips_abi mips_abi = { .setup_rt_frame = setup_rt_frame, .rt_signal_return_offset = offsetof(struct mips_vdso, rt_signal_trampoline), - .restart = __NR_restart_syscall + .restart = __NR_restart_syscall, + + .off_sc_fpregs = offsetof(struct sigcontext, sc_fpregs), + .off_sc_fpc_csr = offsetof(struct sigcontext, sc_fpc_csr), + .off_sc_used_math = offsetof(struct sigcontext, sc_used_math), }; static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) @@ -616,6 +857,9 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, void *unused, user_exit(); + if (thread_info_flags & _TIF_UPROBE) + uprobe_notify_resume(regs); + /* deal with pending signal delivery */ if (thread_info_flags & _TIF_SIGPENDING) do_signal(regs); @@ -629,43 +873,46 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, void *unused, } #ifdef CONFIG_SMP -#ifndef CONFIG_EVA -static int smp_save_fp_context(struct sigcontext __user *sc) +static int smp_save_fp_context(void __user *sc) { return raw_cpu_has_fpu - ? _save_fp_context(sc) + ? save_hw_fp_context(sc) : copy_fp_to_sigcontext(sc); } -static int smp_restore_fp_context(struct sigcontext __user *sc) +static int smp_restore_fp_context(void __user *sc) { return raw_cpu_has_fpu - ? _restore_fp_context(sc) + ? restore_hw_fp_context(sc) : copy_fp_from_sigcontext(sc); } -#endif /* CONFIG_EVA */ #endif static int signal_setup(void) { -#ifndef CONFIG_EVA + /* + * The offset from sigcontext to extended context should be the same + * regardless of the type of signal, such that userland can always know + * where to look if it wishes to find the extended context structures. + */ + BUILD_BUG_ON((offsetof(struct sigframe, sf_extcontext) - + offsetof(struct sigframe, sf_sc)) != + (offsetof(struct rt_sigframe, rs_uc.uc_extcontext) - + offsetof(struct rt_sigframe, rs_uc.uc_mcontext))); + #ifdef CONFIG_SMP /* For now just do the cpu_has_fpu check when the functions are invoked */ save_fp_context = smp_save_fp_context; restore_fp_context = smp_restore_fp_context; #else if (cpu_has_fpu) { - save_fp_context = _save_fp_context; - restore_fp_context = _restore_fp_context; + save_fp_context = save_hw_fp_context; + restore_fp_context = restore_hw_fp_context; } else { save_fp_context = copy_fp_to_sigcontext; restore_fp_context = copy_fp_from_sigcontext; } #endif /* CONFIG_SMP */ -#else - save_fp_context = copy_fp_to_sigcontext; - restore_fp_context = copy_fp_from_sigcontext; -#endif return 0; } diff --git a/arch/mips/kernel/signal32.c b/arch/mips/kernel/signal32.c index 5d7f2634996fd4..f7e89524e3166f 100644 --- a/arch/mips/kernel/signal32.c +++ b/arch/mips/kernel/signal32.c @@ -36,12 +36,6 @@ #include "signal-common.h" -static int (*save_fp_context32)(struct sigcontext32 __user *sc); -static int (*restore_fp_context32)(struct sigcontext32 __user *sc); - -extern asmlinkage int _save_fp_context32(struct sigcontext32 __user *sc); -extern asmlinkage int _restore_fp_context32(struct sigcontext32 __user *sc); - /* * Including would give use the 64-bit syscall numbers ... */ @@ -74,99 +68,11 @@ struct rt_sigframe32 { struct ucontext32 rs_uc; }; -/* - * Thread saved context copy to/from a signal context presumed to be on the - * user stack, and therefore accessed with appropriate macros from uaccess.h. - */ -static int copy_fp_to_sigcontext32(struct sigcontext32 __user *sc) -{ - int i; - int err = 0; - int inc = test_thread_flag(TIF_32BIT_FPREGS) ? 2 : 1; - - for (i = 0; i < NUM_FPU_REGS; i += inc) { - err |= - __put_user(get_fpr64(¤t->thread.fpu.fpr[i], 0), - &sc->sc_fpregs[i]); - } - err |= __put_user(current->thread.fpu.fcr31, &sc->sc_fpc_csr); - - return err; -} - -static int copy_fp_from_sigcontext32(struct sigcontext32 __user *sc) -{ - int i; - int err = 0; - int inc = test_thread_flag(TIF_32BIT_FPREGS) ? 2 : 1; - u64 fpr_val; - - for (i = 0; i < NUM_FPU_REGS; i += inc) { - err |= __get_user(fpr_val, &sc->sc_fpregs[i]); - set_fpr64(¤t->thread.fpu.fpr[i], 0, fpr_val); - } - err |= __get_user(current->thread.fpu.fcr31, &sc->sc_fpc_csr); - - return err; -} - -/* - * sigcontext handlers - */ -static int protected_save_fp_context32(struct sigcontext32 __user *sc) -{ - int err; - while (1) { - lock_fpu_owner(); - if (is_fpu_owner()) { - err = save_fp_context32(sc); - unlock_fpu_owner(); - } else { - unlock_fpu_owner(); - err = copy_fp_to_sigcontext32(sc); - } - if (likely(!err)) - break; - /* touch the sigcontext and try again */ - err = __put_user(0, &sc->sc_fpregs[0]) | - __put_user(0, &sc->sc_fpregs[31]) | - __put_user(0, &sc->sc_fpc_csr); - if (err) - break; /* really bad sigcontext */ - } - return err; -} - -static int protected_restore_fp_context32(struct sigcontext32 __user *sc) -{ - int err, tmp __maybe_unused; - while (1) { - lock_fpu_owner(); - if (is_fpu_owner()) { - err = restore_fp_context32(sc); - unlock_fpu_owner(); - } else { - unlock_fpu_owner(); - err = copy_fp_from_sigcontext32(sc); - } - if (likely(!err)) - break; - /* touch the sigcontext and try again */ - err = __get_user(tmp, &sc->sc_fpregs[0]) | - __get_user(tmp, &sc->sc_fpregs[31]) | - __get_user(tmp, &sc->sc_fpc_csr); - if (err) - break; /* really bad sigcontext */ - } - return err; -} - static int setup_sigcontext32(struct pt_regs *regs, struct sigcontext32 __user *sc) { int err = 0; int i; - u32 used_math; err |= __put_user(regs->cp0_epc, &sc->sc_pc); @@ -186,35 +92,18 @@ static int setup_sigcontext32(struct pt_regs *regs, err |= __put_user(mflo3(), &sc->sc_lo3); } - used_math = !!used_math(); - err |= __put_user(used_math, &sc->sc_used_math); + /* + * Save FPU state to signal context. Signal handler + * will "inherit" current FPU state. + */ + err |= protected_save_fp_context(sc); - if (used_math) { - /* - * Save FPU state to signal context. Signal handler - * will "inherit" current FPU state. - */ - err |= protected_save_fp_context32(sc); - } return err; } -static int -check_and_restore_fp_context32(struct sigcontext32 __user *sc) -{ - int err, sig; - - err = sig = fpcsr_pending(&sc->sc_fpc_csr); - if (err > 0) - err = 0; - err |= protected_restore_fp_context32(sc); - return err ?: sig; -} - static int restore_sigcontext32(struct pt_regs *regs, struct sigcontext32 __user *sc) { - u32 used_math; int err = 0; s32 treg; int i; @@ -238,70 +127,7 @@ static int restore_sigcontext32(struct pt_regs *regs, for (i = 1; i < 32; i++) err |= __get_user(regs->regs[i], &sc->sc_regs[i]); - err |= __get_user(used_math, &sc->sc_used_math); - conditional_used_math(used_math); - - if (used_math) { - /* restore fpu context if we have used it before */ - if (!err) - err = check_and_restore_fp_context32(sc); - } else { - /* signal handler may have used FPU. Give it up. */ - lose_fpu(0); - } - - return err; -} - -/* - * - */ -extern void __put_sigset_unknown_nsig(void); -extern void __get_sigset_unknown_nsig(void); - -static inline int put_sigset(const sigset_t *kbuf, compat_sigset_t __user *ubuf) -{ - int err = 0; - - if (!access_ok(VERIFY_WRITE, ubuf, sizeof(*ubuf))) - return -EFAULT; - - switch (_NSIG_WORDS) { - default: - __put_sigset_unknown_nsig(); - case 2: - err |= __put_user(kbuf->sig[1] >> 32, &ubuf->sig[3]); - err |= __put_user(kbuf->sig[1] & 0xffffffff, &ubuf->sig[2]); - case 1: - err |= __put_user(kbuf->sig[0] >> 32, &ubuf->sig[1]); - err |= __put_user(kbuf->sig[0] & 0xffffffff, &ubuf->sig[0]); - } - - return err; -} - -static inline int get_sigset(sigset_t *kbuf, const compat_sigset_t __user *ubuf) -{ - int err = 0; - unsigned long sig[4]; - - if (!access_ok(VERIFY_READ, ubuf, sizeof(*ubuf))) - return -EFAULT; - - switch (_NSIG_WORDS) { - default: - __get_sigset_unknown_nsig(); - case 2: - err |= __get_user(sig[3], &ubuf->sig[3]); - err |= __get_user(sig[2], &ubuf->sig[2]); - kbuf->sig[1] = sig[2] | (sig[3] << 32); - case 1: - err |= __get_user(sig[1], &ubuf->sig[1]); - err |= __get_user(sig[0], &ubuf->sig[0]); - kbuf->sig[0] = sig[0] | (sig[1] << 32); - } - - return err; + return err ?: protected_restore_fp_context(sc); } /* @@ -585,20 +411,9 @@ struct mips_abi mips_abi_32 = { .setup_rt_frame = setup_rt_frame_32, .rt_signal_return_offset = offsetof(struct mips_vdso, o32_rt_signal_trampoline), - .restart = __NR_O32_restart_syscall -}; - -static int signal32_init(void) -{ - if (cpu_has_fpu) { - save_fp_context32 = _save_fp_context32; - restore_fp_context32 = _restore_fp_context32; - } else { - save_fp_context32 = copy_fp_to_sigcontext32; - restore_fp_context32 = copy_fp_from_sigcontext32; - } + .restart = __NR_O32_restart_syscall, - return 0; -} - -arch_initcall(signal32_init); + .off_sc_fpregs = offsetof(struct sigcontext32, sc_fpregs), + .off_sc_fpc_csr = offsetof(struct sigcontext32, sc_fpc_csr), + .off_sc_used_math = offsetof(struct sigcontext32, sc_used_math), +}; diff --git a/arch/mips/kernel/signal_n32.c b/arch/mips/kernel/signal_n32.c index f1d4751eead098..0d017fdcaf07aa 100644 --- a/arch/mips/kernel/signal_n32.c +++ b/arch/mips/kernel/signal_n32.c @@ -153,5 +153,9 @@ struct mips_abi mips_abi_n32 = { .setup_rt_frame = setup_rt_frame_n32, .rt_signal_return_offset = offsetof(struct mips_vdso, n32_rt_signal_trampoline), - .restart = __NR_N32_restart_syscall + .restart = __NR_N32_restart_syscall, + + .off_sc_fpregs = offsetof(struct sigcontext, sc_fpregs), + .off_sc_fpc_csr = offsetof(struct sigcontext, sc_fpc_csr), + .off_sc_used_math = offsetof(struct sigcontext, sc_used_math), }; diff --git a/arch/mips/kernel/spram.c b/arch/mips/kernel/spram.c index d1168d7c31e8ef..8489c88f993231 100644 --- a/arch/mips/kernel/spram.c +++ b/arch/mips/kernel/spram.c @@ -209,6 +209,7 @@ void spram_config(void) case CPU_PROAPTIV: case CPU_P5600: case CPU_QEMU_GENERIC: + case CPU_I6400: config0 = read_c0_config(); /* FIXME: addresses are Malta specific */ if (config0 & (1<<24)) { diff --git a/arch/mips/kernel/sysrq.c b/arch/mips/kernel/sysrq.c index 5b539f5fc9d9d6..5f055393092dec 100644 --- a/arch/mips/kernel/sysrq.c +++ b/arch/mips/kernel/sysrq.c @@ -21,24 +21,12 @@ static DEFINE_SPINLOCK(show_lock); static void sysrq_tlbdump_single(void *dummy) { - const int field = 2 * sizeof(unsigned long); unsigned long flags; spin_lock_irqsave(&show_lock, flags); pr_info("CPU%d:\n", smp_processor_id()); - pr_info("Index : %0x\n", read_c0_index()); - pr_info("Pagemask: %0x\n", read_c0_pagemask()); - pr_info("EntryHi : %0*lx\n", field, read_c0_entryhi()); - pr_info("EntryLo0: %0*lx\n", field, read_c0_entrylo0()); - pr_info("EntryLo1: %0*lx\n", field, read_c0_entrylo1()); - pr_info("Wired : %0x\n", read_c0_wired()); - pr_info("Pagegrain: %0x\n", read_c0_pagegrain()); - if (cpu_has_htw) { - pr_info("PWField : %0*lx\n", field, read_c0_pwfield()); - pr_info("PWSize : %0*lx\n", field, read_c0_pwsize()); - pr_info("PWCtl : %0x\n", read_c0_pwctl()); - } + dump_tlb_regs(); pr_info("\n"); dump_tlb_all(); pr_info("\n"); diff --git a/arch/mips/kernel/traps.c b/arch/mips/kernel/traps.c index 8ea28e6ab37dea..fdb392b27e8109 100644 --- a/arch/mips/kernel/traps.c +++ b/arch/mips/kernel/traps.c @@ -370,11 +370,6 @@ void show_registers(struct pt_regs *regs) set_fs(old_fs); } -static int regs_to_trapnr(struct pt_regs *regs) -{ - return (regs->cp0_cause >> 2) & 0x1f; -} - static DEFINE_RAW_SPINLOCK(die_lock); void __noreturn die(const char *str, struct pt_regs *regs) @@ -384,7 +379,7 @@ void __noreturn die(const char *str, struct pt_regs *regs) oops_enter(); - if (notify_die(DIE_OOPS, str, regs, 0, regs_to_trapnr(regs), + if (notify_die(DIE_OOPS, str, regs, 0, current->thread.trap_nr, SIGSEGV) == NOTIFY_STOP) sig = 0; @@ -470,7 +465,7 @@ asmlinkage void do_be(struct pt_regs *regs) printk(KERN_ALERT "%s bus error, epc == %0*lx, ra == %0*lx\n", data ? "Data" : "Instruction", field, regs->cp0_epc, field, regs->regs[31]); - if (notify_die(DIE_OOPS, "bus error", regs, 0, regs_to_trapnr(regs), + if (notify_die(DIE_OOPS, "bus error", regs, 0, current->thread.trap_nr, SIGBUS) == NOTIFY_STOP) goto out; @@ -826,7 +821,7 @@ asmlinkage void do_fpe(struct pt_regs *regs, unsigned long fcr31) int sig; prev_state = exception_enter(); - if (notify_die(DIE_FP, "FP exception", regs, 0, regs_to_trapnr(regs), + if (notify_die(DIE_FP, "FP exception", regs, 0, current->thread.trap_nr, SIGFPE) == NOTIFY_STOP) goto out; @@ -882,11 +877,12 @@ void do_trap_or_bp(struct pt_regs *regs, unsigned int code, char b[40]; #ifdef CONFIG_KGDB_LOW_LEVEL_TRAP - if (kgdb_ll_trap(DIE_TRAP, str, regs, code, regs_to_trapnr(regs), SIGTRAP) == NOTIFY_STOP) + if (kgdb_ll_trap(DIE_TRAP, str, regs, code, current->thread.trap_nr, + SIGTRAP) == NOTIFY_STOP) return; #endif /* CONFIG_KGDB_LOW_LEVEL_TRAP */ - if (notify_die(DIE_TRAP, str, regs, code, regs_to_trapnr(regs), + if (notify_die(DIE_TRAP, str, regs, code, current->thread.trap_nr, SIGTRAP) == NOTIFY_STOP) return; @@ -948,6 +944,7 @@ asmlinkage void do_bp(struct pt_regs *regs) set_fs(KERNEL_DS); prev_state = exception_enter(); + current->thread.trap_nr = (regs->cp0_cause >> 2) & 0x1f; if (get_isa16_mode(regs->cp0_epc)) { u16 instr[2]; @@ -987,15 +984,27 @@ asmlinkage void do_bp(struct pt_regs *regs) * pertain to them. */ switch (bcode) { + case BRK_UPROBE: + if (notify_die(DIE_UPROBE, "uprobe", regs, bcode, + current->thread.trap_nr, SIGTRAP) == NOTIFY_STOP) + goto out; + else + break; + case BRK_UPROBE_XOL: + if (notify_die(DIE_UPROBE_XOL, "uprobe_xol", regs, bcode, + current->thread.trap_nr, SIGTRAP) == NOTIFY_STOP) + goto out; + else + break; case BRK_KPROBE_BP: if (notify_die(DIE_BREAK, "debug", regs, bcode, - regs_to_trapnr(regs), SIGTRAP) == NOTIFY_STOP) + current->thread.trap_nr, SIGTRAP) == NOTIFY_STOP) goto out; else break; case BRK_KPROBE_SSTEPBP: if (notify_die(DIE_SSTEPBP, "single_step", regs, bcode, - regs_to_trapnr(regs), SIGTRAP) == NOTIFY_STOP) + current->thread.trap_nr, SIGTRAP) == NOTIFY_STOP) goto out; else break; @@ -1028,6 +1037,7 @@ asmlinkage void do_tr(struct pt_regs *regs) set_fs(get_ds()); prev_state = exception_enter(); + current->thread.trap_nr = (regs->cp0_cause >> 2) & 0x1f; if (get_isa16_mode(regs->cp0_epc)) { if (__get_user(instr[0], (u16 __user *)(epc + 0)) || __get_user(instr[1], (u16 __user *)(epc + 2))) @@ -1094,8 +1104,9 @@ asmlinkage void do_ri(struct pt_regs *regs) no_r2_instr: prev_state = exception_enter(); + current->thread.trap_nr = (regs->cp0_cause >> 2) & 0x1f; - if (notify_die(DIE_RI, "RI Fault", regs, 0, regs_to_trapnr(regs), + if (notify_die(DIE_RI, "RI Fault", regs, 0, current->thread.trap_nr, SIGILL) == NOTIFY_STOP) goto out; @@ -1444,8 +1455,9 @@ asmlinkage void do_msa_fpe(struct pt_regs *regs, unsigned int msacsr) enum ctx_state prev_state; prev_state = exception_enter(); + current->thread.trap_nr = (regs->cp0_cause >> 2) & 0x1f; if (notify_die(DIE_MSAFP, "MSA FP exception", regs, 0, - regs_to_trapnr(regs), SIGFPE) == NOTIFY_STOP) + current->thread.trap_nr, SIGFPE) == NOTIFY_STOP) goto out; /* Clear MSACSR.Cause before enabling interrupts */ @@ -1523,7 +1535,6 @@ asmlinkage void do_watch(struct pt_regs *regs) asmlinkage void do_mcheck(struct pt_regs *regs) { - const int field = 2 * sizeof(unsigned long); int multi_match = regs->cp0_status & ST0_TS; enum ctx_state prev_state; mm_segment_t old_fs = get_fs(); @@ -1532,19 +1543,8 @@ asmlinkage void do_mcheck(struct pt_regs *regs) show_regs(regs); if (multi_match) { - pr_err("Index : %0x\n", read_c0_index()); - pr_err("Pagemask: %0x\n", read_c0_pagemask()); - pr_err("EntryHi : %0*lx\n", field, read_c0_entryhi()); - pr_err("EntryLo0: %0*lx\n", field, read_c0_entrylo0()); - pr_err("EntryLo1: %0*lx\n", field, read_c0_entrylo1()); - pr_err("Wired : %0x\n", read_c0_wired()); - pr_err("Pagegrain: %0x\n", read_c0_pagegrain()); - if (cpu_has_htw) { - pr_err("PWField : %0*lx\n", field, read_c0_pwfield()); - pr_err("PWSize : %0*lx\n", field, read_c0_pwsize()); - pr_err("PWCtl : %0x\n", read_c0_pwctl()); - } - pr_err("\n"); + dump_tlb_regs(); + pr_info("\n"); dump_tlb_all(); } @@ -1651,6 +1651,7 @@ static inline void parity_protection_init(void) case CPU_PROAPTIV: case CPU_P5600: case CPU_QEMU_GENERIC: + case CPU_I6400: { #define ERRCTL_PE 0x80000000 #define ERRCTL_L2P 0x00800000 diff --git a/arch/mips/kernel/unaligned.c b/arch/mips/kernel/unaligned.c index eb3efd137fd17c..990354dd6bdedb 100644 --- a/arch/mips/kernel/unaligned.c +++ b/arch/mips/kernel/unaligned.c @@ -891,6 +891,9 @@ static void emulate_load_store_insn(struct pt_regs *regs, #ifdef CONFIG_EVA mm_segment_t seg; #endif + union fpureg *fpr; + enum msa_2b_fmt df; + unsigned int wd; origpc = (unsigned long)pc; orig31 = regs->regs[31]; @@ -1202,6 +1205,75 @@ static void emulate_load_store_insn(struct pt_regs *regs, break; return; + case msa_op: + if (!cpu_has_msa) + goto sigill; + + /* + * If we've reached this point then userland should have taken + * the MSA disabled exception & initialised vector context at + * some point in the past. + */ + BUG_ON(!thread_msa_context_live()); + + df = insn.msa_mi10_format.df; + wd = insn.msa_mi10_format.wd; + fpr = ¤t->thread.fpu.fpr[wd]; + + switch (insn.msa_mi10_format.func) { + case msa_ld_op: + if (!access_ok(VERIFY_READ, addr, sizeof(*fpr))) + goto sigbus; + + /* + * Disable preemption to avoid a race between copying + * state from userland, migrating to another CPU and + * updating the hardware vector register below. + */ + preempt_disable(); + + res = __copy_from_user_inatomic(fpr, addr, + sizeof(*fpr)); + if (res) + goto fault; + + /* + * Update the hardware register if it is in use by the + * task in this quantum, in order to avoid having to + * save & restore the whole vector context. + */ + if (test_thread_flag(TIF_USEDMSA)) + write_msa_wr(wd, fpr, df); + + preempt_enable(); + break; + + case msa_st_op: + if (!access_ok(VERIFY_WRITE, addr, sizeof(*fpr))) + goto sigbus; + + /* + * Update from the hardware register if it is in use by + * the task in this quantum, in order to avoid having to + * save & restore the whole vector context. + */ + preempt_disable(); + if (test_thread_flag(TIF_USEDMSA)) + read_msa_wr(wd, fpr, df); + preempt_enable(); + + res = __copy_to_user_inatomic(addr, fpr, sizeof(*fpr)); + if (res) + goto fault; + break; + + default: + goto sigbus; + } + + compute_return_epc(regs); + break; + #ifndef CONFIG_CPU_MIPSR6 /* * COP2 is available to implementor for application specific use. @@ -2240,5 +2312,5 @@ static int __init debugfs_unaligned(void) return -ENOMEM; return 0; } -__initcall(debugfs_unaligned); +arch_initcall(debugfs_unaligned); #endif diff --git a/arch/mips/kernel/uprobes.c b/arch/mips/kernel/uprobes.c new file mode 100644 index 00000000000000..8452d933a6453c --- /dev/null +++ b/arch/mips/kernel/uprobes.c @@ -0,0 +1,341 @@ +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +static inline int insn_has_delay_slot(const union mips_instruction insn) +{ + switch (insn.i_format.opcode) { + /* + * jr and jalr are in r_format format. + */ + case spec_op: + switch (insn.r_format.func) { + case jalr_op: + case jr_op: + return 1; + } + break; + + /* + * This group contains: + * bltz_op, bgez_op, bltzl_op, bgezl_op, + * bltzal_op, bgezal_op, bltzall_op, bgezall_op. + */ + case bcond_op: + switch (insn.i_format.rt) { + case bltz_op: + case bltzl_op: + case bgez_op: + case bgezl_op: + case bltzal_op: + case bltzall_op: + case bgezal_op: + case bgezall_op: + case bposge32_op: + return 1; + } + break; + + /* + * These are unconditional and in j_format. + */ + case jal_op: + case j_op: + case beq_op: + case beql_op: + case bne_op: + case bnel_op: + case blez_op: /* not really i_format */ + case blezl_op: + case bgtz_op: + case bgtzl_op: + return 1; + + /* + * And now the FPA/cp1 branch instructions. + */ + case cop1_op: +#ifdef CONFIG_CPU_CAVIUM_OCTEON + case lwc2_op: /* This is bbit0 on Octeon */ + case ldc2_op: /* This is bbit032 on Octeon */ + case swc2_op: /* This is bbit1 on Octeon */ + case sdc2_op: /* This is bbit132 on Octeon */ +#endif + return 1; + } + + return 0; +} + +/** + * arch_uprobe_analyze_insn - instruction analysis including validity and fixups. + * @mm: the probed address space. + * @arch_uprobe: the probepoint information. + * @addr: virtual address at which to install the probepoint + * Return 0 on success or a -ve number on error. + */ +int arch_uprobe_analyze_insn(struct arch_uprobe *aup, + struct mm_struct *mm, unsigned long addr) +{ + union mips_instruction inst; + + /* + * For the time being this also blocks attempts to use uprobes with + * MIPS16 and microMIPS. + */ + if (addr & 0x03) + return -EINVAL; + + inst.word = aup->insn[0]; + aup->ixol[0] = aup->insn[insn_has_delay_slot(inst)]; + aup->ixol[1] = UPROBE_BRK_UPROBE_XOL; /* NOP */ + + return 0; +} + +/** + * is_trap_insn - check if the instruction is a trap variant + * @insn: instruction to be checked. + * Returns true if @insn is a trap variant. + * + * This definition overrides the weak definition in kernel/events/uprobes.c. + * and is needed for the case where an architecture has multiple trap + * instructions (like PowerPC or MIPS). We treat BREAK just like the more + * modern conditional trap instructions. + */ +bool is_trap_insn(uprobe_opcode_t *insn) +{ + union mips_instruction inst; + + inst.word = *insn; + + switch (inst.i_format.opcode) { + case spec_op: + switch (inst.r_format.func) { + case break_op: + case teq_op: + case tge_op: + case tgeu_op: + case tlt_op: + case tltu_op: + case tne_op: + return 1; + } + break; + + case bcond_op: /* Yes, really ... */ + switch (inst.u_format.rt) { + case teqi_op: + case tgei_op: + case tgeiu_op: + case tlti_op: + case tltiu_op: + case tnei_op: + return 1; + } + break; + } + + return 0; +} + +#define UPROBE_TRAP_NR ULONG_MAX + +/* + * arch_uprobe_pre_xol - prepare to execute out of line. + * @auprobe: the probepoint information. + * @regs: reflects the saved user state of current task. + */ +int arch_uprobe_pre_xol(struct arch_uprobe *aup, struct pt_regs *regs) +{ + struct uprobe_task *utask = current->utask; + union mips_instruction insn; + + /* + * Now find the EPC where to resume after the breakpoint has been + * dealt with. This may require emulation of a branch. + */ + aup->resume_epc = regs->cp0_epc + 4; + if (insn_has_delay_slot((union mips_instruction) aup->insn[0])) { + unsigned long epc; + + epc = regs->cp0_epc; + __compute_return_epc_for_insn(regs, insn); + aup->resume_epc = regs->cp0_epc; + } + + utask->autask.saved_trap_nr = current->thread.trap_nr; + current->thread.trap_nr = UPROBE_TRAP_NR; + regs->cp0_epc = current->utask->xol_vaddr; + + return 0; +} + +int arch_uprobe_post_xol(struct arch_uprobe *aup, struct pt_regs *regs) +{ + struct uprobe_task *utask = current->utask; + + current->thread.trap_nr = utask->autask.saved_trap_nr; + regs->cp0_epc = aup->resume_epc; + + return 0; +} + +/* + * If xol insn itself traps and generates a signal(Say, + * SIGILL/SIGSEGV/etc), then detect the case where a singlestepped + * instruction jumps back to its own address. It is assumed that anything + * like do_page_fault/do_trap/etc sets thread.trap_nr != -1. + * + * arch_uprobe_pre_xol/arch_uprobe_post_xol save/restore thread.trap_nr, + * arch_uprobe_xol_was_trapped() simply checks that ->trap_nr is not equal to + * UPROBE_TRAP_NR == -1 set by arch_uprobe_pre_xol(). + */ +bool arch_uprobe_xol_was_trapped(struct task_struct *tsk) +{ + if (tsk->thread.trap_nr != UPROBE_TRAP_NR) + return true; + + return false; +} + +int arch_uprobe_exception_notify(struct notifier_block *self, + unsigned long val, void *data) +{ + struct die_args *args = data; + struct pt_regs *regs = args->regs; + + /* regs == NULL is a kernel bug */ + if (WARN_ON(!regs)) + return NOTIFY_DONE; + + /* We are only interested in userspace traps */ + if (!user_mode(regs)) + return NOTIFY_DONE; + + switch (val) { + case DIE_BREAK: + if (uprobe_pre_sstep_notifier(regs)) + return NOTIFY_STOP; + break; + case DIE_UPROBE_XOL: + if (uprobe_post_sstep_notifier(regs)) + return NOTIFY_STOP; + default: + break; + } + + return 0; +} + +/* + * This function gets called when XOL instruction either gets trapped or + * the thread has a fatal signal. Reset the instruction pointer to its + * probed address for the potential restart or for post mortem analysis. + */ +void arch_uprobe_abort_xol(struct arch_uprobe *aup, + struct pt_regs *regs) +{ + struct uprobe_task *utask = current->utask; + + instruction_pointer_set(regs, utask->vaddr); +} + +unsigned long arch_uretprobe_hijack_return_addr( + unsigned long trampoline_vaddr, struct pt_regs *regs) +{ + unsigned long ra; + + ra = regs->regs[31]; + + /* Replace the return address with the trampoline address */ + regs->regs[31] = ra; + + return ra; +} + +/** + * set_swbp - store breakpoint at a given address. + * @auprobe: arch specific probepoint information. + * @mm: the probed process address space. + * @vaddr: the virtual address to insert the opcode. + * + * For mm @mm, store the breakpoint instruction at @vaddr. + * Return 0 (success) or a negative errno. + * + * This version overrides the weak version in kernel/events/uprobes.c. + * It is required to handle MIPS16 and microMIPS. + */ +int __weak set_swbp(struct arch_uprobe *auprobe, struct mm_struct *mm, + unsigned long vaddr) +{ + return uprobe_write_opcode(mm, vaddr, UPROBE_SWBP_INSN); +} + +/** + * set_orig_insn - Restore the original instruction. + * @mm: the probed process address space. + * @auprobe: arch specific probepoint information. + * @vaddr: the virtual address to insert the opcode. + * + * For mm @mm, restore the original opcode (opcode) at @vaddr. + * Return 0 (success) or a negative errno. + * + * This overrides the weak version in kernel/events/uprobes.c. + */ +int set_orig_insn(struct arch_uprobe *auprobe, struct mm_struct *mm, + unsigned long vaddr) +{ + return uprobe_write_opcode(mm, vaddr, + *(uprobe_opcode_t *)&auprobe->orig_inst[0].word); +} + +void __weak arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr, + void *src, unsigned long len) +{ + void *kaddr; + + /* Initialize the slot */ + kaddr = kmap_atomic(page); + memcpy(kaddr + (vaddr & ~PAGE_MASK), src, len); + kunmap_atomic(kaddr); + + /* + * The MIPS version of flush_icache_range will operate safely on + * user space addresses and more importantly, it doesn't require a + * VMA argument. + */ + flush_icache_range(vaddr, vaddr + len); +} + +/** + * uprobe_get_swbp_addr - compute address of swbp given post-swbp regs + * @regs: Reflects the saved state of the task after it has hit a breakpoint + * instruction. + * Return the address of the breakpoint instruction. + * + * This overrides the weak version in kernel/events/uprobes.c. + */ +unsigned long uprobe_get_swbp_addr(struct pt_regs *regs) +{ + return instruction_pointer(regs); +} + +/* + * See if the instruction can be emulated. + * Returns true if instruction was emulated, false otherwise. + * + * For now we always emulate so this function just returns 0. + */ +bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs) +{ + return 0; +} diff --git a/arch/mips/kernel/vpe.c b/arch/mips/kernel/vpe.c index 11da314565cc33..9067b651c7a2d9 100644 --- a/arch/mips/kernel/vpe.c +++ b/arch/mips/kernel/vpe.c @@ -817,6 +817,7 @@ static int vpe_open(struct inode *inode, struct file *filp) static int vpe_release(struct inode *inode, struct file *filp) { +#if defined(CONFIG_MIPS_VPE_LOADER_MT) || defined(CONFIG_MIPS_VPE_LOADER_CMP) struct vpe *v; Elf_Ehdr *hdr; int ret = 0; @@ -827,7 +828,7 @@ static int vpe_release(struct inode *inode, struct file *filp) hdr = (Elf_Ehdr *) v->pbuffer; if (memcmp(hdr->e_ident, ELFMAG, SELFMAG) == 0) { - if ((vpe_elfload(v) >= 0) && vpe_run) { + if (vpe_elfload(v) >= 0) { vpe_run(v); } else { pr_warn("VPE loader: ELF load failed.\n"); @@ -850,6 +851,10 @@ static int vpe_release(struct inode *inode, struct file *filp) v->plen = 0; return ret; +#else + pr_warn("VPE loader: ELF load failed.\n"); + return -ENOEXEC; +#endif } static ssize_t vpe_write(struct file *file, const char __user *buffer, diff --git a/arch/mips/lasat/sysctl.c b/arch/mips/lasat/sysctl.c index a57959e648a6be..c710d969938d74 100644 --- a/arch/mips/lasat/sysctl.c +++ b/arch/mips/lasat/sysctl.c @@ -270,4 +270,4 @@ static int __init lasat_register_sysctl(void) return 0; } -__initcall(lasat_register_sysctl); +arch_initcall(lasat_register_sysctl); diff --git a/arch/mips/lib/dump_tlb.c b/arch/mips/lib/dump_tlb.c index 167f35634709b8..92a37319efbe9a 100644 --- a/arch/mips/lib/dump_tlb.c +++ b/arch/mips/lib/dump_tlb.c @@ -13,6 +13,33 @@ #include #include +void dump_tlb_regs(void) +{ + const int field = 2 * sizeof(unsigned long); + + pr_info("Index : %0x\n", read_c0_index()); + pr_info("PageMask : %0x\n", read_c0_pagemask()); + pr_info("EntryHi : %0*lx\n", field, read_c0_entryhi()); + pr_info("EntryLo0 : %0*lx\n", field, read_c0_entrylo0()); + pr_info("EntryLo1 : %0*lx\n", field, read_c0_entrylo1()); + pr_info("Wired : %0x\n", read_c0_wired()); + switch (current_cpu_type()) { + case CPU_R10000: + case CPU_R12000: + case CPU_R14000: + case CPU_R16000: + pr_info("FrameMask: %0x\n", read_c0_framemask()); + break; + } + if (cpu_has_small_pages || cpu_has_rixi || cpu_has_xpa) + pr_info("PageGrain: %0x\n", read_c0_pagegrain()); + if (cpu_has_htw) { + pr_info("PWField : %0*lx\n", field, read_c0_pwfield()); + pr_info("PWSize : %0*lx\n", field, read_c0_pwsize()); + pr_info("PWCtl : %0x\n", read_c0_pwctl()); + } +} + static inline const char *msk2str(unsigned int mask) { switch (mask) { @@ -87,7 +114,7 @@ static void dump_tlb(int first, int last) * leave only a single G bit set after a machine check exception * due to duplicate TLB entry. */ - if (!((entrylo0 | entrylo1) & MIPS_ENTRYLO_G) && + if (!((entrylo0 | entrylo1) & ENTRYLO_G) && (entryhi & 0xff) != asid) continue; @@ -96,8 +123,8 @@ static void dump_tlb(int first, int last) */ printk("Index: %2d pgmask=%s ", i, msk2str(pagemask)); - c0 = (entrylo0 & MIPS_ENTRYLO_C) >> MIPS_ENTRYLO_C_SHIFT; - c1 = (entrylo1 & MIPS_ENTRYLO_C) >> MIPS_ENTRYLO_C_SHIFT; + c0 = (entrylo0 & ENTRYLO_C) >> ENTRYLO_C_SHIFT; + c1 = (entrylo1 & ENTRYLO_C) >> ENTRYLO_C_SHIFT; printk("va=%0*lx asid=%02lx\n", vwidth, (entryhi & ~0x1fffUL), @@ -114,9 +141,9 @@ static void dump_tlb(int first, int last) (entrylo0 & MIPS_ENTRYLO_XI) ? 1 : 0); printk("pa=%0*llx c=%d d=%d v=%d g=%d] [", pwidth, pa, c0, - (entrylo0 & MIPS_ENTRYLO_D) ? 1 : 0, - (entrylo0 & MIPS_ENTRYLO_V) ? 1 : 0, - (entrylo0 & MIPS_ENTRYLO_G) ? 1 : 0); + (entrylo0 & ENTRYLO_D) ? 1 : 0, + (entrylo0 & ENTRYLO_V) ? 1 : 0, + (entrylo0 & ENTRYLO_G) ? 1 : 0); /* RI/XI are in awkward places, so mask them off separately */ pa = entrylo1 & ~(MIPS_ENTRYLO_RI | MIPS_ENTRYLO_XI); if (xpa) @@ -128,9 +155,9 @@ static void dump_tlb(int first, int last) (entrylo1 & MIPS_ENTRYLO_XI) ? 1 : 0); printk("pa=%0*llx c=%d d=%d v=%d g=%d]\n", pwidth, pa, c1, - (entrylo1 & MIPS_ENTRYLO_D) ? 1 : 0, - (entrylo1 & MIPS_ENTRYLO_V) ? 1 : 0, - (entrylo1 & MIPS_ENTRYLO_G) ? 1 : 0); + (entrylo1 & ENTRYLO_D) ? 1 : 0, + (entrylo1 & ENTRYLO_V) ? 1 : 0, + (entrylo1 & ENTRYLO_G) ? 1 : 0); } printk("\n"); diff --git a/arch/mips/lib/r3k_dump_tlb.c b/arch/mips/lib/r3k_dump_tlb.c index 8e0d3cff8ae448..cfcbb5218b5951 100644 --- a/arch/mips/lib/r3k_dump_tlb.c +++ b/arch/mips/lib/r3k_dump_tlb.c @@ -14,6 +14,17 @@ #include #include +extern int r3k_have_wired_reg; + +void dump_tlb_regs(void) +{ + pr_info("Index : %0x\n", read_c0_index()); + pr_info("EntryHi : %0lx\n", read_c0_entryhi()); + pr_info("EntryLo : %0lx\n", read_c0_entrylo0()); + if (r3k_have_wired_reg) + pr_info("Wired : %0x\n", read_c0_wired()); +} + static void dump_tlb(int first, int last) { int i; diff --git a/arch/mips/loongson32/common/time.c b/arch/mips/loongson32/common/time.c index df0f850d6a5f49..0996b025eeefb0 100644 --- a/arch/mips/loongson32/common/time.c +++ b/arch/mips/loongson32/common/time.c @@ -126,26 +126,34 @@ static irqreturn_t ls1x_clockevent_isr(int irq, void *devid) return IRQ_HANDLED; } -static void ls1x_clockevent_set_mode(enum clock_event_mode mode, - struct clock_event_device *cd) +static int ls1x_clockevent_set_state_periodic(struct clock_event_device *cd) { raw_spin_lock(&ls1x_timer_lock); - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - ls1x_pwmtimer_set_period(ls1x_jiffies_per_tick); - ls1x_pwmtimer_restart(); - case CLOCK_EVT_MODE_RESUME: - __raw_writel(INT_EN | CNT_EN, timer_base + PWM_CTRL); - break; - case CLOCK_EVT_MODE_ONESHOT: - case CLOCK_EVT_MODE_SHUTDOWN: - __raw_writel(__raw_readl(timer_base + PWM_CTRL) & ~CNT_EN, - timer_base + PWM_CTRL); - break; - default: - break; - } + ls1x_pwmtimer_set_period(ls1x_jiffies_per_tick); + ls1x_pwmtimer_restart(); + __raw_writel(INT_EN | CNT_EN, timer_base + PWM_CTRL); raw_spin_unlock(&ls1x_timer_lock); + + return 0; +} + +static int ls1x_clockevent_tick_resume(struct clock_event_device *cd) +{ + raw_spin_lock(&ls1x_timer_lock); + __raw_writel(INT_EN | CNT_EN, timer_base + PWM_CTRL); + raw_spin_unlock(&ls1x_timer_lock); + + return 0; +} + +static int ls1x_clockevent_set_state_shutdown(struct clock_event_device *cd) +{ + raw_spin_lock(&ls1x_timer_lock); + __raw_writel(__raw_readl(timer_base + PWM_CTRL) & ~CNT_EN, + timer_base + PWM_CTRL); + raw_spin_unlock(&ls1x_timer_lock); + + return 0; } static int ls1x_clockevent_set_next(unsigned long evt, @@ -160,12 +168,15 @@ static int ls1x_clockevent_set_next(unsigned long evt, } static struct clock_event_device ls1x_clockevent = { - .name = "ls1x-pwmtimer", - .features = CLOCK_EVT_FEAT_PERIODIC, - .rating = 300, - .irq = LS1X_TIMER_IRQ, - .set_next_event = ls1x_clockevent_set_next, - .set_mode = ls1x_clockevent_set_mode, + .name = "ls1x-pwmtimer", + .features = CLOCK_EVT_FEAT_PERIODIC, + .rating = 300, + .irq = LS1X_TIMER_IRQ, + .set_next_event = ls1x_clockevent_set_next, + .set_state_shutdown = ls1x_clockevent_set_state_shutdown, + .set_state_periodic = ls1x_clockevent_set_state_periodic, + .set_state_oneshot = ls1x_clockevent_set_state_shutdown, + .tick_resume = ls1x_clockevent_tick_resume, }; static struct irqaction ls1x_pwmtimer_irqaction = { diff --git a/arch/mips/loongson64/common/cs5536/cs5536_mfgpt.c b/arch/mips/loongson64/common/cs5536/cs5536_mfgpt.c index 875037063a80ec..da77d412514c03 100644 --- a/arch/mips/loongson64/common/cs5536/cs5536_mfgpt.c +++ b/arch/mips/loongson64/common/cs5536/cs5536_mfgpt.c @@ -51,40 +51,36 @@ void enable_mfgpt0_counter(void) } EXPORT_SYMBOL(enable_mfgpt0_counter); -static void init_mfgpt_timer(enum clock_event_mode mode, - struct clock_event_device *evt) +static int mfgpt_timer_set_periodic(struct clock_event_device *evt) { raw_spin_lock(&mfgpt_lock); - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - outw(COMPARE, MFGPT0_CMP2); /* set comparator2 */ - outw(0, MFGPT0_CNT); /* set counter to 0 */ - enable_mfgpt0_counter(); - break; - - case CLOCK_EVT_MODE_SHUTDOWN: - case CLOCK_EVT_MODE_UNUSED: - if (evt->mode == CLOCK_EVT_MODE_PERIODIC || - evt->mode == CLOCK_EVT_MODE_ONESHOT) - disable_mfgpt0_counter(); - break; - - case CLOCK_EVT_MODE_ONESHOT: - /* The oneshot mode have very high deviation, Not use it! */ - break; - - case CLOCK_EVT_MODE_RESUME: - /* Nothing to do here */ - break; - } + outw(COMPARE, MFGPT0_CMP2); /* set comparator2 */ + outw(0, MFGPT0_CNT); /* set counter to 0 */ + enable_mfgpt0_counter(); + raw_spin_unlock(&mfgpt_lock); + return 0; +} + +static int mfgpt_timer_shutdown(struct clock_event_device *evt) +{ + if (clockevent_state_periodic(evt) || clockevent_state_oneshot(evt)) { + raw_spin_lock(&mfgpt_lock); + disable_mfgpt0_counter(); + raw_spin_unlock(&mfgpt_lock); + } + + return 0; } static struct clock_event_device mfgpt_clockevent = { .name = "mfgpt", .features = CLOCK_EVT_FEAT_PERIODIC, - .set_mode = init_mfgpt_timer, + + /* The oneshot mode have very high deviation, don't use it! */ + .set_state_shutdown = mfgpt_timer_shutdown, + .set_state_periodic = mfgpt_timer_set_periodic, .irq = CS5536_MFGPT_INTR, }; diff --git a/arch/mips/loongson64/loongson-3/hpet.c b/arch/mips/loongson64/loongson-3/hpet.c index 5c21cd3bd339ab..bf9f1a77f0e598 100644 --- a/arch/mips/loongson64/loongson-3/hpet.c +++ b/arch/mips/loongson64/loongson-3/hpet.c @@ -78,55 +78,77 @@ static void hpet_enable_legacy_int(void) /* Do nothing on Loongson-3 */ } -static void hpet_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) +static int hpet_set_state_periodic(struct clock_event_device *evt) { - int cfg = 0; + int cfg; spin_lock(&hpet_lock); - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - pr_info("set clock event to periodic mode!\n"); - /* stop counter */ - hpet_stop_counter(); - - /* enables the timer0 to generate a periodic interrupt */ - cfg = hpet_read(HPET_T0_CFG); - cfg &= ~HPET_TN_LEVEL; - cfg |= HPET_TN_ENABLE | HPET_TN_PERIODIC | - HPET_TN_SETVAL | HPET_TN_32BIT; - hpet_write(HPET_T0_CFG, cfg); - - /* set the comparator */ - hpet_write(HPET_T0_CMP, HPET_COMPARE_VAL); - udelay(1); - hpet_write(HPET_T0_CMP, HPET_COMPARE_VAL); - - /* start counter */ - hpet_start_counter(); - break; - case CLOCK_EVT_MODE_SHUTDOWN: - case CLOCK_EVT_MODE_UNUSED: - cfg = hpet_read(HPET_T0_CFG); - cfg &= ~HPET_TN_ENABLE; - hpet_write(HPET_T0_CFG, cfg); - break; - case CLOCK_EVT_MODE_ONESHOT: - pr_info("set clock event to one shot mode!\n"); - cfg = hpet_read(HPET_T0_CFG); - /* set timer0 type - * 1 : periodic interrupt - * 0 : non-periodic(oneshot) interrupt - */ - cfg &= ~HPET_TN_PERIODIC; - cfg |= HPET_TN_ENABLE | HPET_TN_32BIT; - hpet_write(HPET_T0_CFG, cfg); - break; - case CLOCK_EVT_MODE_RESUME: - hpet_enable_legacy_int(); - break; - } + + pr_info("set clock event to periodic mode!\n"); + /* stop counter */ + hpet_stop_counter(); + + /* enables the timer0 to generate a periodic interrupt */ + cfg = hpet_read(HPET_T0_CFG); + cfg &= ~HPET_TN_LEVEL; + cfg |= HPET_TN_ENABLE | HPET_TN_PERIODIC | HPET_TN_SETVAL | + HPET_TN_32BIT; + hpet_write(HPET_T0_CFG, cfg); + + /* set the comparator */ + hpet_write(HPET_T0_CMP, HPET_COMPARE_VAL); + udelay(1); + hpet_write(HPET_T0_CMP, HPET_COMPARE_VAL); + + /* start counter */ + hpet_start_counter(); + + spin_unlock(&hpet_lock); + return 0; +} + +static int hpet_set_state_shutdown(struct clock_event_device *evt) +{ + int cfg; + + spin_lock(&hpet_lock); + + cfg = hpet_read(HPET_T0_CFG); + cfg &= ~HPET_TN_ENABLE; + hpet_write(HPET_T0_CFG, cfg); + spin_unlock(&hpet_lock); + return 0; +} + +static int hpet_set_state_oneshot(struct clock_event_device *evt) +{ + int cfg; + + spin_lock(&hpet_lock); + + pr_info("set clock event to one shot mode!\n"); + cfg = hpet_read(HPET_T0_CFG); + /* + * set timer0 type + * 1 : periodic interrupt + * 0 : non-periodic(oneshot) interrupt + */ + cfg &= ~HPET_TN_PERIODIC; + cfg |= HPET_TN_ENABLE | HPET_TN_32BIT; + hpet_write(HPET_T0_CFG, cfg); + + spin_unlock(&hpet_lock); + return 0; +} + +static int hpet_tick_resume(struct clock_event_device *evt) +{ + spin_lock(&hpet_lock); + hpet_enable_legacy_int(); + spin_unlock(&hpet_lock); + + return 0; } static int hpet_next_event(unsigned long delta, @@ -206,7 +228,10 @@ void __init setup_hpet_timer(void) cd->name = "hpet"; cd->rating = 320; cd->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT; - cd->set_mode = hpet_set_mode; + cd->set_state_shutdown = hpet_set_state_shutdown; + cd->set_state_periodic = hpet_set_state_periodic; + cd->set_state_oneshot = hpet_set_state_oneshot; + cd->tick_resume = hpet_tick_resume; cd->set_next_event = hpet_next_event; cd->irq = HPET_T0_IRQ; cd->cpumask = cpumask_of(cpu); diff --git a/arch/mips/math-emu/Makefile b/arch/mips/math-emu/Makefile index 2e5f96275c38a6..a19641d3ac23ca 100644 --- a/arch/mips/math-emu/Makefile +++ b/arch/mips/math-emu/Makefile @@ -4,9 +4,9 @@ obj-y += cp1emu.o ieee754dp.o ieee754sp.o ieee754.o \ dp_div.o dp_mul.o dp_sub.o dp_add.o dp_fsp.o dp_cmp.o dp_simple.o \ - dp_tint.o dp_fint.o \ + dp_tint.o dp_fint.o dp_maddf.o dp_msubf.o dp_2008class.o dp_fmin.o dp_fmax.o \ sp_div.o sp_mul.o sp_sub.o sp_add.o sp_fdp.o sp_cmp.o sp_simple.o \ - sp_tint.o sp_fint.o \ + sp_tint.o sp_fint.o sp_maddf.o sp_msubf.o sp_2008class.o sp_fmin.o sp_fmax.o \ dsemul.o lib-y += ieee754d.o \ diff --git a/arch/mips/math-emu/cp1emu.c b/arch/mips/math-emu/cp1emu.c index 712f17a2ecf206..32f0e19a0d7f71 100644 --- a/arch/mips/math-emu/cp1emu.c +++ b/arch/mips/math-emu/cp1emu.c @@ -1137,7 +1137,7 @@ static int cop1Emulate(struct pt_regs *xcp, struct mips_fpu_struct *ctx, break; case mfhc_op: - if (!cpu_has_mips_r2) + if (!cpu_has_mips_r2_r6) goto sigill; /* copregister rd -> gpr[rt] */ @@ -1148,7 +1148,7 @@ static int cop1Emulate(struct pt_regs *xcp, struct mips_fpu_struct *ctx, break; case mthc_op: - if (!cpu_has_mips_r2) + if (!cpu_has_mips_r2_r6) goto sigill; /* copregister rd <- gpr[rt] */ @@ -1181,6 +1181,24 @@ static int cop1Emulate(struct pt_regs *xcp, struct mips_fpu_struct *ctx, } break; + case bc1eqz_op: + case bc1nez_op: + if (!cpu_has_mips_r6 || delay_slot(xcp)) + return SIGILL; + + cond = likely = 0; + switch (MIPSInst_RS(ir)) { + case bc1eqz_op: + if (get_fpr32(¤t->thread.fpu.fpr[MIPSInst_RT(ir)], 0) & 0x1) + cond = 1; + break; + case bc1nez_op: + if (!(get_fpr32(¤t->thread.fpu.fpr[MIPSInst_RT(ir)], 0) & 0x1)) + cond = 1; + break; + } + goto branch_common; + case bc_op: if (delay_slot(xcp)) return SIGILL; @@ -1207,7 +1225,7 @@ static int cop1Emulate(struct pt_regs *xcp, struct mips_fpu_struct *ctx, case bct_op: break; } - +branch_common: set_delay_slot(xcp); if (cond) { /* @@ -1376,6 +1394,14 @@ static const unsigned char cmptab[8] = { IEEE754_CLT | IEEE754_CEQ | IEEE754_CUN, /* cmp_ule (sig) cmp_ngt */ }; +static const unsigned char negative_cmptab[8] = { + 0, /* Reserved */ + IEEE754_CLT | IEEE754_CGT | IEEE754_CEQ, + IEEE754_CLT | IEEE754_CGT | IEEE754_CUN, + IEEE754_CLT | IEEE754_CGT, + /* Reserved */ +}; + /* * Additional MIPS4 instructions @@ -1717,6 +1743,126 @@ static int fpu_emu(struct pt_regs *xcp, struct mips_fpu_struct *ctx, SPFROMREG(rv.s, MIPSInst_FS(ir)); break; + case fseleqz_op: + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(rv.s, MIPSInst_FT(ir)); + if (rv.w & 0x1) + rv.w = 0; + else + SPFROMREG(rv.s, MIPSInst_FS(ir)); + break; + + case fselnez_op: + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(rv.s, MIPSInst_FT(ir)); + if (rv.w & 0x1) + SPFROMREG(rv.s, MIPSInst_FS(ir)); + else + rv.w = 0; + break; + + case fmaddf_op: { + union ieee754sp ft, fs, fd; + + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(ft, MIPSInst_FT(ir)); + SPFROMREG(fs, MIPSInst_FS(ir)); + SPFROMREG(fd, MIPSInst_FD(ir)); + rv.s = ieee754sp_maddf(fd, fs, ft); + break; + } + + case fmsubf_op: { + union ieee754sp ft, fs, fd; + + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(ft, MIPSInst_FT(ir)); + SPFROMREG(fs, MIPSInst_FS(ir)); + SPFROMREG(fd, MIPSInst_FD(ir)); + rv.s = ieee754sp_msubf(fd, fs, ft); + break; + } + + case frint_op: { + union ieee754sp fs; + + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(fs, MIPSInst_FS(ir)); + rv.l = ieee754sp_tlong(fs); + rv.s = ieee754sp_flong(rv.l); + goto copcsr; + } + + case fclass_op: { + union ieee754sp fs; + + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(fs, MIPSInst_FS(ir)); + rv.w = ieee754sp_2008class(fs); + rfmt = w_fmt; + break; + } + + case fmin_op: { + union ieee754sp fs, ft; + + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(ft, MIPSInst_FT(ir)); + SPFROMREG(fs, MIPSInst_FS(ir)); + rv.s = ieee754sp_fmin(fs, ft); + break; + } + + case fmina_op: { + union ieee754sp fs, ft; + + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(ft, MIPSInst_FT(ir)); + SPFROMREG(fs, MIPSInst_FS(ir)); + rv.s = ieee754sp_fmina(fs, ft); + break; + } + + case fmax_op: { + union ieee754sp fs, ft; + + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(ft, MIPSInst_FT(ir)); + SPFROMREG(fs, MIPSInst_FS(ir)); + rv.s = ieee754sp_fmax(fs, ft); + break; + } + + case fmaxa_op: { + union ieee754sp fs, ft; + + if (!cpu_has_mips_r6) + return SIGILL; + + SPFROMREG(ft, MIPSInst_FT(ir)); + SPFROMREG(fs, MIPSInst_FS(ir)); + rv.s = ieee754sp_fmaxa(fs, ft); + break; + } + case fabs_op: handler.u = ieee754sp_abs; goto scopuop; @@ -1820,7 +1966,7 @@ static int fpu_emu(struct pt_regs *xcp, struct mips_fpu_struct *ctx, goto copcsr; default: - if (MIPSInst_FUNC(ir) >= fcmp_op) { + if (!NO_R6EMU && MIPSInst_FUNC(ir) >= fcmp_op) { unsigned cmpop = MIPSInst_FUNC(ir) - fcmp_op; union ieee754sp fs, ft; @@ -1914,6 +2060,127 @@ static int fpu_emu(struct pt_regs *xcp, struct mips_fpu_struct *ctx, return 0; DPFROMREG(rv.d, MIPSInst_FS(ir)); break; + + case fseleqz_op: + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(rv.d, MIPSInst_FT(ir)); + if (rv.l & 0x1) + rv.l = 0; + else + DPFROMREG(rv.d, MIPSInst_FS(ir)); + break; + + case fselnez_op: + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(rv.d, MIPSInst_FT(ir)); + if (rv.l & 0x1) + DPFROMREG(rv.d, MIPSInst_FS(ir)); + else + rv.l = 0; + break; + + case fmaddf_op: { + union ieee754dp ft, fs, fd; + + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(ft, MIPSInst_FT(ir)); + DPFROMREG(fs, MIPSInst_FS(ir)); + DPFROMREG(fd, MIPSInst_FD(ir)); + rv.d = ieee754dp_maddf(fd, fs, ft); + break; + } + + case fmsubf_op: { + union ieee754dp ft, fs, fd; + + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(ft, MIPSInst_FT(ir)); + DPFROMREG(fs, MIPSInst_FS(ir)); + DPFROMREG(fd, MIPSInst_FD(ir)); + rv.d = ieee754dp_msubf(fd, fs, ft); + break; + } + + case frint_op: { + union ieee754dp fs; + + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(fs, MIPSInst_FS(ir)); + rv.l = ieee754dp_tlong(fs); + rv.d = ieee754dp_flong(rv.l); + goto copcsr; + } + + case fclass_op: { + union ieee754dp fs; + + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(fs, MIPSInst_FS(ir)); + rv.w = ieee754dp_2008class(fs); + rfmt = w_fmt; + break; + } + + case fmin_op: { + union ieee754dp fs, ft; + + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(ft, MIPSInst_FT(ir)); + DPFROMREG(fs, MIPSInst_FS(ir)); + rv.d = ieee754dp_fmin(fs, ft); + break; + } + + case fmina_op: { + union ieee754dp fs, ft; + + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(ft, MIPSInst_FT(ir)); + DPFROMREG(fs, MIPSInst_FS(ir)); + rv.d = ieee754dp_fmina(fs, ft); + break; + } + + case fmax_op: { + union ieee754dp fs, ft; + + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(ft, MIPSInst_FT(ir)); + DPFROMREG(fs, MIPSInst_FS(ir)); + rv.d = ieee754dp_fmax(fs, ft); + break; + } + + case fmaxa_op: { + union ieee754dp fs, ft; + + if (!cpu_has_mips_r6) + return SIGILL; + + DPFROMREG(ft, MIPSInst_FT(ir)); + DPFROMREG(fs, MIPSInst_FS(ir)); + rv.d = ieee754dp_fmaxa(fs, ft); + break; + } + case fabs_op: handler.u = ieee754dp_abs; goto dcopuop; @@ -1997,7 +2264,7 @@ static int fpu_emu(struct pt_regs *xcp, struct mips_fpu_struct *ctx, goto copcsr; default: - if (MIPSInst_FUNC(ir) >= fcmp_op) { + if (!NO_R6EMU && MIPSInst_FUNC(ir) >= fcmp_op) { unsigned cmpop = MIPSInst_FUNC(ir) - fcmp_op; union ieee754dp fs, ft; @@ -2021,8 +2288,11 @@ static int fpu_emu(struct pt_regs *xcp, struct mips_fpu_struct *ctx, break; } break; + } + + case w_fmt: { + union ieee754dp fs; - case w_fmt: switch (MIPSInst_FUNC(ir)) { case fcvts_op: /* convert word to single precision real */ @@ -2036,10 +2306,65 @@ static int fpu_emu(struct pt_regs *xcp, struct mips_fpu_struct *ctx, rv.d = ieee754dp_fint(fs.bits); rfmt = d_fmt; goto copcsr; - default: - return SIGILL; + default: { + /* Emulating the new CMP.condn.fmt R6 instruction */ +#define CMPOP_MASK 0x7 +#define SIGN_BIT (0x1 << 3) +#define PREDICATE_BIT (0x1 << 4) + + int cmpop = MIPSInst_FUNC(ir) & CMPOP_MASK; + int sig = MIPSInst_FUNC(ir) & SIGN_BIT; + union ieee754sp fs, ft; + + /* This is an R6 only instruction */ + if (!cpu_has_mips_r6 || + (MIPSInst_FUNC(ir) & 0x20)) + return SIGILL; + + /* fmt is w_fmt for single precision so fix it */ + rfmt = s_fmt; + /* default to false */ + rv.w = 0; + + /* CMP.condn.S */ + SPFROMREG(fs, MIPSInst_FS(ir)); + SPFROMREG(ft, MIPSInst_FT(ir)); + + /* positive predicates */ + if (!(MIPSInst_FUNC(ir) & PREDICATE_BIT)) { + if (ieee754sp_cmp(fs, ft, cmptab[cmpop], + sig)) + rv.w = -1; /* true, all 1s */ + if ((sig) && + ieee754_cxtest(IEEE754_INVALID_OPERATION)) + rcsr = FPU_CSR_INV_X | FPU_CSR_INV_S; + else + goto copcsr; + } else { + /* negative predicates */ + switch (cmpop) { + case 1: + case 2: + case 3: + if (ieee754sp_cmp(fs, ft, + negative_cmptab[cmpop], + sig)) + rv.w = -1; /* true, all 1s */ + if (sig && + ieee754_cxtest(IEEE754_INVALID_OPERATION)) + rcsr = FPU_CSR_INV_X | FPU_CSR_INV_S; + else + goto copcsr; + break; + default: + /* Reserved R6 ops */ + pr_err("Reserved MIPS R6 CMP.condn.S operation\n"); + return SIGILL; + } + } + break; + } } - break; } case l_fmt: @@ -2060,11 +2385,60 @@ static int fpu_emu(struct pt_regs *xcp, struct mips_fpu_struct *ctx, rv.d = ieee754dp_flong(bits); rfmt = d_fmt; goto copcsr; - default: - return SIGILL; - } - break; + default: { + /* Emulating the new CMP.condn.fmt R6 instruction */ + int cmpop = MIPSInst_FUNC(ir) & CMPOP_MASK; + int sig = MIPSInst_FUNC(ir) & SIGN_BIT; + union ieee754dp fs, ft; + + if (!cpu_has_mips_r6 || + (MIPSInst_FUNC(ir) & 0x20)) + return SIGILL; + + /* fmt is l_fmt for double precision so fix it */ + rfmt = d_fmt; + /* default to false */ + rv.l = 0; + /* CMP.condn.D */ + DPFROMREG(fs, MIPSInst_FS(ir)); + DPFROMREG(ft, MIPSInst_FT(ir)); + + /* positive predicates */ + if (!(MIPSInst_FUNC(ir) & PREDICATE_BIT)) { + if (ieee754dp_cmp(fs, ft, + cmptab[cmpop], sig)) + rv.l = -1LL; /* true, all 1s */ + if (sig && + ieee754_cxtest(IEEE754_INVALID_OPERATION)) + rcsr = FPU_CSR_INV_X | FPU_CSR_INV_S; + else + goto copcsr; + } else { + /* negative predicates */ + switch (cmpop) { + case 1: + case 2: + case 3: + if (ieee754dp_cmp(fs, ft, + negative_cmptab[cmpop], + sig)) + rv.l = -1LL; /* true, all 1s */ + if (sig && + ieee754_cxtest(IEEE754_INVALID_OPERATION)) + rcsr = FPU_CSR_INV_X | FPU_CSR_INV_S; + else + goto copcsr; + break; + default: + /* Reserved R6 ops */ + pr_err("Reserved MIPS R6 CMP.condn.D operation\n"); + return SIGILL; + } + } + break; + } + } default: return SIGILL; } diff --git a/arch/mips/math-emu/dp_2008class.c b/arch/mips/math-emu/dp_2008class.c new file mode 100644 index 00000000000000..9dc39fc4835e11 --- /dev/null +++ b/arch/mips/math-emu/dp_2008class.c @@ -0,0 +1,55 @@ +/* + * IEEE754 floating point arithmetic + * double precision: CLASS.f + * FPR[fd] = class(FPR[fs]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754dp.h" + +int ieee754dp_2008class(union ieee754dp x) +{ + COMPXDP; + + EXPLODEXDP; + + /* + * 10 bit mask as follows: + * + * bit0 = SNAN + * bit1 = QNAN + * bit2 = -INF + * bit3 = -NORM + * bit4 = -DNORM + * bit5 = -ZERO + * bit6 = INF + * bit7 = NORM + * bit8 = DNORM + * bit9 = ZERO + */ + + switch(xc) { + case IEEE754_CLASS_SNAN: + return 0x01; + case IEEE754_CLASS_QNAN: + return 0x02; + case IEEE754_CLASS_INF: + return 0x04 << (xs ? 0 : 4); + case IEEE754_CLASS_NORM: + return 0x08 << (xs ? 0 : 4); + case IEEE754_CLASS_DNORM: + return 0x10 << (xs ? 0 : 4); + case IEEE754_CLASS_ZERO: + return 0x20 << (xs ? 0 : 4); + default: + pr_err("Unknown class: %d\n", xc); + return 0; + } +} diff --git a/arch/mips/math-emu/dp_fmax.c b/arch/mips/math-emu/dp_fmax.c new file mode 100644 index 00000000000000..fd71b8daaaf205 --- /dev/null +++ b/arch/mips/math-emu/dp_fmax.c @@ -0,0 +1,213 @@ +/* + * IEEE754 floating point arithmetic + * double precision: MIN{,A}.f + * MIN : Scalar Floating-Point Minimum + * MINA: Scalar Floating-Point argument with Minimum Absolute Value + * + * MIN.D : FPR[fd] = minNum(FPR[fs],FPR[ft]) + * MINA.D: FPR[fd] = maxNumMag(FPR[fs],FPR[ft]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754dp.h" + +union ieee754dp ieee754dp_fmax(union ieee754dp x, union ieee754dp y) +{ + COMPXDP; + COMPYDP; + + EXPLODEXDP; + EXPLODEYDP; + + FLUSHXDP; + FLUSHYDP; + + ieee754_clearcx(); + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754dp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754dp_nanxcpt(x); + + /* numbers are preferred to NaNs */ + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return x; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return y; + + /* + * Infinity and zero handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + return xs ? y : x; + + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + return ys ? x : y; + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + if (xs == ys) + return x; + return ieee754dp_zero(1); + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + DPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + DPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + DPDNORMX; + } + + /* Finally get to do some computation */ + + assert(xm & DP_HIDDEN_BIT); + assert(ym & DP_HIDDEN_BIT); + + /* Compare signs */ + if (xs > ys) + return y; + else if (xs < ys) + return x; + + /* Compare exponent */ + if (xe > ye) + return x; + else if (xe < ye) + return y; + + /* Compare mantissa */ + if (xm <= ym) + return y; + return x; +} + +union ieee754dp ieee754dp_fmaxa(union ieee754dp x, union ieee754dp y) +{ + COMPXDP; + COMPYDP; + + EXPLODEXDP; + EXPLODEYDP; + + FLUSHXDP; + FLUSHYDP; + + ieee754_clearcx(); + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754dp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754dp_nanxcpt(x); + + /* numbers are preferred to NaNs */ + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return x; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return y; + + /* + * Infinity and zero handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + return x; + + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + return y; + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + if (xs == ys) + return x; + return ieee754dp_zero(1); + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + DPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + DPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + DPDNORMX; + } + + /* Finally get to do some computation */ + + assert(xm & DP_HIDDEN_BIT); + assert(ym & DP_HIDDEN_BIT); + + /* Compare exponent */ + if (xe > ye) + return x; + else if (xe < ye) + return y; + + /* Compare mantissa */ + if (xm <= ym) + return y; + return x; +} diff --git a/arch/mips/math-emu/dp_fmin.c b/arch/mips/math-emu/dp_fmin.c new file mode 100644 index 00000000000000..c1072b0dfb9519 --- /dev/null +++ b/arch/mips/math-emu/dp_fmin.c @@ -0,0 +1,213 @@ +/* + * IEEE754 floating point arithmetic + * double precision: MIN{,A}.f + * MIN : Scalar Floating-Point Minimum + * MINA: Scalar Floating-Point argument with Minimum Absolute Value + * + * MIN.D : FPR[fd] = minNum(FPR[fs],FPR[ft]) + * MINA.D: FPR[fd] = maxNumMag(FPR[fs],FPR[ft]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754dp.h" + +union ieee754dp ieee754dp_fmin(union ieee754dp x, union ieee754dp y) +{ + COMPXDP; + COMPYDP; + + EXPLODEXDP; + EXPLODEYDP; + + FLUSHXDP; + FLUSHYDP; + + ieee754_clearcx(); + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754dp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754dp_nanxcpt(x); + + /* numbers are preferred to NaNs */ + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return x; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return y; + + /* + * Infinity and zero handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + return xs ? x : y; + + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + return ys ? y : x; + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + if (xs == ys) + return x; + return ieee754dp_zero(1); + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + DPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + DPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + DPDNORMX; + } + + /* Finally get to do some computation */ + + assert(xm & DP_HIDDEN_BIT); + assert(ym & DP_HIDDEN_BIT); + + /* Compare signs */ + if (xs > ys) + return x; + else if (xs < ys) + return y; + + /* Compare exponent */ + if (xe > ye) + return y; + else if (xe < ye) + return x; + + /* Compare mantissa */ + if (xm <= ym) + return x; + return y; +} + +union ieee754dp ieee754dp_fmina(union ieee754dp x, union ieee754dp y) +{ + COMPXDP; + COMPYDP; + + EXPLODEXDP; + EXPLODEYDP; + + FLUSHXDP; + FLUSHYDP; + + ieee754_clearcx(); + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754dp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754dp_nanxcpt(x); + + /* numbers are preferred to NaNs */ + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return x; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return y; + + /* + * Infinity and zero handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + return x; + + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + return y; + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + if (xs == ys) + return x; + return ieee754dp_zero(1); + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + DPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + DPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + DPDNORMX; + } + + /* Finally get to do some computation */ + + assert(xm & DP_HIDDEN_BIT); + assert(ym & DP_HIDDEN_BIT); + + /* Compare exponent */ + if (xe > ye) + return y; + else if (xe < ye) + return x; + + /* Compare mantissa */ + if (xm <= ym) + return x; + return y; +} diff --git a/arch/mips/math-emu/dp_maddf.c b/arch/mips/math-emu/dp_maddf.c new file mode 100644 index 00000000000000..119eda9fa1ea8f --- /dev/null +++ b/arch/mips/math-emu/dp_maddf.c @@ -0,0 +1,265 @@ +/* + * IEEE754 floating point arithmetic + * double precision: MADDF.f (Fused Multiply Add) + * MADDF.fmt: FPR[fd] = FPR[fd] + (FPR[fs] x FPR[ft]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754dp.h" + +union ieee754dp ieee754dp_maddf(union ieee754dp z, union ieee754dp x, + union ieee754dp y) +{ + int re; + int rs; + u64 rm; + unsigned lxm; + unsigned hxm; + unsigned lym; + unsigned hym; + u64 lrm; + u64 hrm; + u64 t; + u64 at; + int s; + + COMPXDP; + COMPYDP; + + u64 zm; int ze; int zs __maybe_unused; int zc; + + EXPLODEXDP; + EXPLODEYDP; + EXPLODEDP(z, zc, zs, ze, zm) + + FLUSHXDP; + FLUSHYDP; + FLUSHDP(z, zc, zs, ze, zm); + + ieee754_clearcx(); + + switch (zc) { + case IEEE754_CLASS_SNAN: + ieee754_setcx(IEEE754_INVALID_OPERATION); + return ieee754dp_nanxcpt(z); + case IEEE754_CLASS_DNORM: + DPDNORMx(zm, ze); + /* QNAN is handled separately below */ + } + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754dp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754dp_nanxcpt(x); + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return y; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return x; + + + /* + * Infinity handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + if (zc == IEEE754_CLASS_QNAN) + return z; + ieee754_setcx(IEEE754_INVALID_OPERATION); + return ieee754dp_indef(); + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + if (zc == IEEE754_CLASS_QNAN) + return z; + return ieee754dp_inf(xs ^ ys); + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + if (zc == IEEE754_CLASS_INF) + return ieee754dp_inf(zs); + /* Multiplication is 0 so just return z */ + return z; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + DPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754dp_inf(zs); + DPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754dp_inf(zs); + DPDNORMX; + break; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_NORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754dp_inf(zs); + /* fall through to real computations */ + } + + /* Finally get to do some computation */ + + /* + * Do the multiplication bit first + * + * rm = xm * ym, re = xe + ye basically + * + * At this point xm and ym should have been normalized. + */ + assert(xm & DP_HIDDEN_BIT); + assert(ym & DP_HIDDEN_BIT); + + re = xe + ye; + rs = xs ^ ys; + + /* shunt to top of word */ + xm <<= 64 - (DP_FBITS + 1); + ym <<= 64 - (DP_FBITS + 1); + + /* + * Multiply 32 bits xm, ym to give high 32 bits rm with stickness. + */ + + /* 32 * 32 => 64 */ +#define DPXMULT(x, y) ((u64)(x) * (u64)y) + + lxm = xm; + hxm = xm >> 32; + lym = ym; + hym = ym >> 32; + + lrm = DPXMULT(lxm, lym); + hrm = DPXMULT(hxm, hym); + + t = DPXMULT(lxm, hym); + + at = lrm + (t << 32); + hrm += at < lrm; + lrm = at; + + hrm = hrm + (t >> 32); + + t = DPXMULT(hxm, lym); + + at = lrm + (t << 32); + hrm += at < lrm; + lrm = at; + + hrm = hrm + (t >> 32); + + rm = hrm | (lrm != 0); + + /* + * Sticky shift down to normal rounding precision. + */ + if ((s64) rm < 0) { + rm = (rm >> (64 - (DP_FBITS + 1 + 3))) | + ((rm << (DP_FBITS + 1 + 3)) != 0); + re++; + } else { + rm = (rm >> (64 - (DP_FBITS + 1 + 3 + 1))) | + ((rm << (DP_FBITS + 1 + 3 + 1)) != 0); + } + assert(rm & (DP_HIDDEN_BIT << 3)); + + /* And now the addition */ + assert(zm & DP_HIDDEN_BIT); + + /* + * Provide guard,round and stick bit space. + */ + zm <<= 3; + + if (ze > re) { + /* + * Have to shift y fraction right to align. + */ + s = ze - re; + rm = XDPSRS(rm, s); + re += s; + } else if (re > ze) { + /* + * Have to shift x fraction right to align. + */ + s = re - ze; + zm = XDPSRS(zm, s); + ze += s; + } + assert(ze == re); + assert(ze <= DP_EMAX); + + if (zs == rs) { + /* + * Generate 28 bit result of adding two 27 bit numbers + * leaving result in xm, xs and xe. + */ + zm = zm + rm; + + if (zm >> (DP_FBITS + 1 + 3)) { /* carry out */ + zm = XDPSRS1(zm); + ze++; + } + } else { + if (zm >= rm) { + zm = zm - rm; + } else { + zm = rm - zm; + zs = rs; + } + if (zm == 0) + return ieee754dp_zero(ieee754_csr.rm == FPU_CSR_RD); + + /* + * Normalize to rounding precision. + */ + while ((zm >> (DP_FBITS + 3)) == 0) { + zm <<= 1; + ze--; + } + } + + return ieee754dp_format(zs, ze, zm); +} diff --git a/arch/mips/math-emu/dp_msubf.c b/arch/mips/math-emu/dp_msubf.c new file mode 100644 index 00000000000000..12241262f85612 --- /dev/null +++ b/arch/mips/math-emu/dp_msubf.c @@ -0,0 +1,269 @@ +/* + * IEEE754 floating point arithmetic + * double precision: MSUB.f (Fused Multiply Subtract) + * MSUBF.fmt: FPR[fd] = FPR[fd] - (FPR[fs] x FPR[ft]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754dp.h" + +union ieee754dp ieee754dp_msubf(union ieee754dp z, union ieee754dp x, + union ieee754dp y) +{ + int re; + int rs; + u64 rm; + unsigned lxm; + unsigned hxm; + unsigned lym; + unsigned hym; + u64 lrm; + u64 hrm; + u64 t; + u64 at; + int s; + + COMPXDP; + COMPYDP; + + u64 zm; int ze; int zs __maybe_unused; int zc; + + EXPLODEXDP; + EXPLODEYDP; + EXPLODEDP(z, zc, zs, ze, zm) + + FLUSHXDP; + FLUSHYDP; + FLUSHDP(z, zc, zs, ze, zm); + + ieee754_clearcx(); + + switch (zc) { + case IEEE754_CLASS_SNAN: + ieee754_setcx(IEEE754_INVALID_OPERATION); + return ieee754dp_nanxcpt(z); + case IEEE754_CLASS_DNORM: + DPDNORMx(zm, ze); + /* QNAN is handled separately below */ + } + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754dp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754dp_nanxcpt(x); + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return y; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return x; + + + /* + * Infinity handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + if (zc == IEEE754_CLASS_QNAN) + return z; + ieee754_setcx(IEEE754_INVALID_OPERATION); + return ieee754dp_indef(); + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + if (zc == IEEE754_CLASS_QNAN) + return z; + return ieee754dp_inf(xs ^ ys); + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + if (zc == IEEE754_CLASS_INF) + return ieee754dp_inf(zs); + /* Multiplication is 0 so just return z */ + return z; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + DPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754dp_inf(zs); + DPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754dp_inf(zs); + DPDNORMX; + break; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_NORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754dp_inf(zs); + /* fall through to real computations */ + } + + /* Finally get to do some computation */ + + /* + * Do the multiplication bit first + * + * rm = xm * ym, re = xe + ye basically + * + * At this point xm and ym should have been normalized. + */ + assert(xm & DP_HIDDEN_BIT); + assert(ym & DP_HIDDEN_BIT); + + re = xe + ye; + rs = xs ^ ys; + + /* shunt to top of word */ + xm <<= 64 - (DP_FBITS + 1); + ym <<= 64 - (DP_FBITS + 1); + + /* + * Multiply 32 bits xm, ym to give high 32 bits rm with stickness. + */ + + /* 32 * 32 => 64 */ +#define DPXMULT(x, y) ((u64)(x) * (u64)y) + + lxm = xm; + hxm = xm >> 32; + lym = ym; + hym = ym >> 32; + + lrm = DPXMULT(lxm, lym); + hrm = DPXMULT(hxm, hym); + + t = DPXMULT(lxm, hym); + + at = lrm + (t << 32); + hrm += at < lrm; + lrm = at; + + hrm = hrm + (t >> 32); + + t = DPXMULT(hxm, lym); + + at = lrm + (t << 32); + hrm += at < lrm; + lrm = at; + + hrm = hrm + (t >> 32); + + rm = hrm | (lrm != 0); + + /* + * Sticky shift down to normal rounding precision. + */ + if ((s64) rm < 0) { + rm = (rm >> (64 - (DP_FBITS + 1 + 3))) | + ((rm << (DP_FBITS + 1 + 3)) != 0); + re++; + } else { + rm = (rm >> (64 - (DP_FBITS + 1 + 3 + 1))) | + ((rm << (DP_FBITS + 1 + 3 + 1)) != 0); + } + assert(rm & (DP_HIDDEN_BIT << 3)); + + /* And now the subtraction */ + + /* flip sign of r and handle as add */ + rs ^= 1; + + assert(zm & DP_HIDDEN_BIT); + + /* + * Provide guard,round and stick bit space. + */ + zm <<= 3; + + if (ze > re) { + /* + * Have to shift y fraction right to align. + */ + s = ze - re; + rm = XDPSRS(rm, s); + re += s; + } else if (re > ze) { + /* + * Have to shift x fraction right to align. + */ + s = re - ze; + zm = XDPSRS(zm, s); + ze += s; + } + assert(ze == re); + assert(ze <= DP_EMAX); + + if (zs == rs) { + /* + * Generate 28 bit result of adding two 27 bit numbers + * leaving result in xm, xs and xe. + */ + zm = zm + rm; + + if (zm >> (DP_FBITS + 1 + 3)) { /* carry out */ + zm = XDPSRS1(zm); + ze++; + } + } else { + if (zm >= rm) { + zm = zm - rm; + } else { + zm = rm - zm; + zs = rs; + } + if (zm == 0) + return ieee754dp_zero(ieee754_csr.rm == FPU_CSR_RD); + + /* + * Normalize to rounding precision. + */ + while ((zm >> (DP_FBITS + 3)) == 0) { + zm <<= 1; + ze--; + } + } + + return ieee754dp_format(zs, ze, zm); +} diff --git a/arch/mips/math-emu/dsemul.c b/arch/mips/math-emu/dsemul.c index e0b5cc27d78b0d..cbb36c14b155ad 100644 --- a/arch/mips/math-emu/dsemul.c +++ b/arch/mips/math-emu/dsemul.c @@ -33,7 +33,6 @@ struct emuframe { int mips_dsemul(struct pt_regs *regs, mips_instruction ir, unsigned long cpc) { - extern asmlinkage void handle_dsemulret(void); struct emuframe __user *fr; int err; diff --git a/arch/mips/math-emu/ieee754.h b/arch/mips/math-emu/ieee754.h index a5ca108ce4679a..df94720714c73c 100644 --- a/arch/mips/math-emu/ieee754.h +++ b/arch/mips/math-emu/ieee754.h @@ -75,6 +75,16 @@ int ieee754sp_cmp(union ieee754sp x, union ieee754sp y, int cop, int sig); union ieee754sp ieee754sp_sqrt(union ieee754sp x); +union ieee754sp ieee754sp_maddf(union ieee754sp z, union ieee754sp x, + union ieee754sp y); +union ieee754sp ieee754sp_msubf(union ieee754sp z, union ieee754sp x, + union ieee754sp y); +int ieee754sp_2008class(union ieee754sp x); +union ieee754sp ieee754sp_fmin(union ieee754sp x, union ieee754sp y); +union ieee754sp ieee754sp_fmina(union ieee754sp x, union ieee754sp y); +union ieee754sp ieee754sp_fmax(union ieee754sp x, union ieee754sp y); +union ieee754sp ieee754sp_fmaxa(union ieee754sp x, union ieee754sp y); + /* * double precision (often aka double) */ @@ -99,6 +109,15 @@ int ieee754dp_cmp(union ieee754dp x, union ieee754dp y, int cop, int sig); union ieee754dp ieee754dp_sqrt(union ieee754dp x); +union ieee754dp ieee754dp_maddf(union ieee754dp z, union ieee754dp x, + union ieee754dp y); +union ieee754dp ieee754dp_msubf(union ieee754dp z, union ieee754dp x, + union ieee754dp y); +int ieee754dp_2008class(union ieee754dp x); +union ieee754dp ieee754dp_fmin(union ieee754dp x, union ieee754dp y); +union ieee754dp ieee754dp_fmina(union ieee754dp x, union ieee754dp y); +union ieee754dp ieee754dp_fmax(union ieee754dp x, union ieee754dp y); +union ieee754dp ieee754dp_fmaxa(union ieee754dp x, union ieee754dp y); /* 5 types of floating point number diff --git a/arch/mips/math-emu/ieee754int.h b/arch/mips/math-emu/ieee754int.h index 05389d5e3a933d..6383e2c5c1adb5 100644 --- a/arch/mips/math-emu/ieee754int.h +++ b/arch/mips/math-emu/ieee754int.h @@ -65,8 +65,8 @@ static inline int ieee754_class_nan(int xc) vc = IEEE754_CLASS_INF; \ else if (vm & SP_MBIT(SP_FBITS-1)) \ vc = IEEE754_CLASS_SNAN; \ - else \ - vc = IEEE754_CLASS_QNAN; \ + else \ + vc = IEEE754_CLASS_QNAN; \ } else if (ve == SP_EMIN-1+SP_EBIAS) { \ if (vm) { \ ve = SP_EMIN; \ @@ -105,8 +105,8 @@ static inline int ieee754_class_nan(int xc) if (vm) { \ ve = DP_EMIN; \ vc = IEEE754_CLASS_DNORM; \ - } else \ - vc = IEEE754_CLASS_ZERO; \ + } else \ + vc = IEEE754_CLASS_ZERO; \ } else { \ ve -= DP_EBIAS; \ vm |= DP_HIDDEN_BIT; \ diff --git a/arch/mips/math-emu/me-debugfs.c b/arch/mips/math-emu/me-debugfs.c index f308e0f05fc585..506a67a98cdf5c 100644 --- a/arch/mips/math-emu/me-debugfs.c +++ b/arch/mips/math-emu/me-debugfs.c @@ -65,4 +65,4 @@ do { \ return 0; } -__initcall(debugfs_fpuemu); +arch_initcall(debugfs_fpuemu); diff --git a/arch/mips/math-emu/sp_2008class.c b/arch/mips/math-emu/sp_2008class.c new file mode 100644 index 00000000000000..ff62606a1465d4 --- /dev/null +++ b/arch/mips/math-emu/sp_2008class.c @@ -0,0 +1,55 @@ +/* + * IEEE754 floating point arithmetic + * single precision: CLASS.f + * FPR[fd] = class(FPR[fs]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754sp.h" + +int ieee754sp_2008class(union ieee754sp x) +{ + COMPXSP; + + EXPLODEXSP; + + /* + * 10 bit mask as follows: + * + * bit0 = SNAN + * bit1 = QNAN + * bit2 = -INF + * bit3 = -NORM + * bit4 = -DNORM + * bit5 = -ZERO + * bit6 = INF + * bit7 = NORM + * bit8 = DNORM + * bit9 = ZERO + */ + + switch(xc) { + case IEEE754_CLASS_SNAN: + return 0x01; + case IEEE754_CLASS_QNAN: + return 0x02; + case IEEE754_CLASS_INF: + return 0x04 << (xs ? 0 : 4); + case IEEE754_CLASS_NORM: + return 0x08 << (xs ? 0 : 4); + case IEEE754_CLASS_DNORM: + return 0x10 << (xs ? 0 : 4); + case IEEE754_CLASS_ZERO: + return 0x20 << (xs ? 0 : 4); + default: + pr_err("Unknown class: %d\n", xc); + return 0; + } +} diff --git a/arch/mips/math-emu/sp_fmax.c b/arch/mips/math-emu/sp_fmax.c new file mode 100644 index 00000000000000..4d000844e48e3d --- /dev/null +++ b/arch/mips/math-emu/sp_fmax.c @@ -0,0 +1,213 @@ +/* + * IEEE754 floating point arithmetic + * single precision: MAX{,A}.f + * MAX : Scalar Floating-Point Maximum + * MAXA: Scalar Floating-Point argument with Maximum Absolute Value + * + * MAX.S : FPR[fd] = maxNum(FPR[fs],FPR[ft]) + * MAXA.S: FPR[fd] = maxNumMag(FPR[fs],FPR[ft]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754sp.h" + +union ieee754sp ieee754sp_fmax(union ieee754sp x, union ieee754sp y) +{ + COMPXSP; + COMPYSP; + + EXPLODEXSP; + EXPLODEYSP; + + FLUSHXSP; + FLUSHYSP; + + ieee754_clearcx(); + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754sp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754sp_nanxcpt(x); + + /* numbers are preferred to NaNs */ + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return x; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return y; + + /* + * Infinity and zero handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + return xs ? y : x; + + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + return ys ? x : y; + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + if (xs == ys) + return x; + return ieee754sp_zero(1); + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + SPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + SPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + SPDNORMX; + } + + /* Finally get to do some computation */ + + assert(xm & SP_HIDDEN_BIT); + assert(ym & SP_HIDDEN_BIT); + + /* Compare signs */ + if (xs > ys) + return y; + else if (xs < ys) + return x; + + /* Compare exponent */ + if (xe > ye) + return x; + else if (xe < ye) + return y; + + /* Compare mantissa */ + if (xm <= ym) + return y; + return x; +} + +union ieee754sp ieee754sp_fmaxa(union ieee754sp x, union ieee754sp y) +{ + COMPXSP; + COMPYSP; + + EXPLODEXSP; + EXPLODEYSP; + + FLUSHXSP; + FLUSHYSP; + + ieee754_clearcx(); + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754sp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754sp_nanxcpt(x); + + /* numbers are preferred to NaNs */ + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return x; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return y; + + /* + * Infinity and zero handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + return x; + + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + return y; + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + if (xs == ys) + return x; + return ieee754sp_zero(1); + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + SPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + SPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + SPDNORMX; + } + + /* Finally get to do some computation */ + + assert(xm & SP_HIDDEN_BIT); + assert(ym & SP_HIDDEN_BIT); + + /* Compare exponent */ + if (xe > ye) + return x; + else if (xe < ye) + return y; + + /* Compare mantissa */ + if (xm <= ym) + return y; + return x; +} diff --git a/arch/mips/math-emu/sp_fmin.c b/arch/mips/math-emu/sp_fmin.c new file mode 100644 index 00000000000000..4eb1bb9e9dec7b --- /dev/null +++ b/arch/mips/math-emu/sp_fmin.c @@ -0,0 +1,213 @@ +/* + * IEEE754 floating point arithmetic + * single precision: MIN{,A}.f + * MIN : Scalar Floating-Point Minimum + * MINA: Scalar Floating-Point argument with Minimum Absolute Value + * + * MIN.S : FPR[fd] = minNum(FPR[fs],FPR[ft]) + * MINA.S: FPR[fd] = maxNumMag(FPR[fs],FPR[ft]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754sp.h" + +union ieee754sp ieee754sp_fmin(union ieee754sp x, union ieee754sp y) +{ + COMPXSP; + COMPYSP; + + EXPLODEXSP; + EXPLODEYSP; + + FLUSHXSP; + FLUSHYSP; + + ieee754_clearcx(); + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754sp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754sp_nanxcpt(x); + + /* numbers are preferred to NaNs */ + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return x; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return y; + + /* + * Infinity and zero handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + return xs ? x : y; + + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + return ys ? y : x; + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + if (xs == ys) + return x; + return ieee754sp_zero(1); + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + SPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + SPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + SPDNORMX; + } + + /* Finally get to do some computation */ + + assert(xm & SP_HIDDEN_BIT); + assert(ym & SP_HIDDEN_BIT); + + /* Compare signs */ + if (xs > ys) + return x; + else if (xs < ys) + return y; + + /* Compare exponent */ + if (xe > ye) + return y; + else if (xe < ye) + return x; + + /* Compare mantissa */ + if (xm <= ym) + return x; + return y; +} + +union ieee754sp ieee754sp_fmina(union ieee754sp x, union ieee754sp y) +{ + COMPXSP; + COMPYSP; + + EXPLODEXSP; + EXPLODEYSP; + + FLUSHXSP; + FLUSHYSP; + + ieee754_clearcx(); + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754sp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754sp_nanxcpt(x); + + /* numbers are preferred to NaNs */ + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return x; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return y; + + /* + * Infinity and zero handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + return x; + + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + return y; + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + if (xs == ys) + return x; + return ieee754sp_zero(1); + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + SPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + SPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + SPDNORMX; + } + + /* Finally get to do some computation */ + + assert(xm & SP_HIDDEN_BIT); + assert(ym & SP_HIDDEN_BIT); + + /* Compare exponent */ + if (xe > ye) + return y; + else if (xe < ye) + return x; + + /* Compare mantissa */ + if (xm <= ym) + return x; + return y; +} diff --git a/arch/mips/math-emu/sp_maddf.c b/arch/mips/math-emu/sp_maddf.c new file mode 100644 index 00000000000000..dd1dd83e34eb0b --- /dev/null +++ b/arch/mips/math-emu/sp_maddf.c @@ -0,0 +1,255 @@ +/* + * IEEE754 floating point arithmetic + * single precision: MADDF.f (Fused Multiply Add) + * MADDF.fmt: FPR[fd] = FPR[fd] + (FPR[fs] x FPR[ft]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754sp.h" + +union ieee754sp ieee754sp_maddf(union ieee754sp z, union ieee754sp x, + union ieee754sp y) +{ + int re; + int rs; + unsigned rm; + unsigned short lxm; + unsigned short hxm; + unsigned short lym; + unsigned short hym; + unsigned lrm; + unsigned hrm; + unsigned t; + unsigned at; + int s; + + COMPXSP; + COMPYSP; + u32 zm; int ze; int zs __maybe_unused; int zc; + + EXPLODEXSP; + EXPLODEYSP; + EXPLODESP(z, zc, zs, ze, zm) + + FLUSHXSP; + FLUSHYSP; + FLUSHSP(z, zc, zs, ze, zm); + + ieee754_clearcx(); + + switch (zc) { + case IEEE754_CLASS_SNAN: + ieee754_setcx(IEEE754_INVALID_OPERATION); + return ieee754sp_nanxcpt(z); + case IEEE754_CLASS_DNORM: + SPDNORMx(zm, ze); + /* QNAN is handled separately below */ + } + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754sp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754sp_nanxcpt(x); + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return y; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return x; + + /* + * Infinity handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + if (zc == IEEE754_CLASS_QNAN) + return z; + ieee754_setcx(IEEE754_INVALID_OPERATION); + return ieee754sp_indef(); + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + if (zc == IEEE754_CLASS_QNAN) + return z; + return ieee754sp_inf(xs ^ ys); + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + if (zc == IEEE754_CLASS_INF) + return ieee754sp_inf(zs); + /* Multiplication is 0 so just return z */ + return z; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + SPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754sp_inf(zs); + SPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754sp_inf(zs); + SPDNORMX; + break; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_NORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754sp_inf(zs); + /* fall through to real computations */ + } + + /* Finally get to do some computation */ + + /* + * Do the multiplication bit first + * + * rm = xm * ym, re = xe + ye basically + * + * At this point xm and ym should have been normalized. + */ + + /* rm = xm * ym, re = xe+ye basically */ + assert(xm & SP_HIDDEN_BIT); + assert(ym & SP_HIDDEN_BIT); + + re = xe + ye; + rs = xs ^ ys; + + /* shunt to top of word */ + xm <<= 32 - (SP_FBITS + 1); + ym <<= 32 - (SP_FBITS + 1); + + /* + * Multiply 32 bits xm, ym to give high 32 bits rm with stickness. + */ + lxm = xm & 0xffff; + hxm = xm >> 16; + lym = ym & 0xffff; + hym = ym >> 16; + + lrm = lxm * lym; /* 16 * 16 => 32 */ + hrm = hxm * hym; /* 16 * 16 => 32 */ + + t = lxm * hym; /* 16 * 16 => 32 */ + at = lrm + (t << 16); + hrm += at < lrm; + lrm = at; + hrm = hrm + (t >> 16); + + t = hxm * lym; /* 16 * 16 => 32 */ + at = lrm + (t << 16); + hrm += at < lrm; + lrm = at; + hrm = hrm + (t >> 16); + + rm = hrm | (lrm != 0); + + /* + * Sticky shift down to normal rounding precision. + */ + if ((int) rm < 0) { + rm = (rm >> (32 - (SP_FBITS + 1 + 3))) | + ((rm << (SP_FBITS + 1 + 3)) != 0); + re++; + } else { + rm = (rm >> (32 - (SP_FBITS + 1 + 3 + 1))) | + ((rm << (SP_FBITS + 1 + 3 + 1)) != 0); + } + assert(rm & (SP_HIDDEN_BIT << 3)); + + /* And now the addition */ + + assert(zm & SP_HIDDEN_BIT); + + /* + * Provide guard,round and stick bit space. + */ + zm <<= 3; + + if (ze > re) { + /* + * Have to shift y fraction right to align. + */ + s = ze - re; + SPXSRSYn(s); + } else if (re > ze) { + /* + * Have to shift x fraction right to align. + */ + s = re - ze; + SPXSRSYn(s); + } + assert(ze == re); + assert(ze <= SP_EMAX); + + if (zs == rs) { + /* + * Generate 28 bit result of adding two 27 bit numbers + * leaving result in zm, zs and ze. + */ + zm = zm + rm; + + if (zm >> (SP_FBITS + 1 + 3)) { /* carry out */ + SPXSRSX1(); + } + } else { + if (zm >= rm) { + zm = zm - rm; + } else { + zm = rm - zm; + zs = rs; + } + if (zm == 0) + return ieee754sp_zero(ieee754_csr.rm == FPU_CSR_RD); + + /* + * Normalize in extended single precision + */ + while ((zm >> (SP_MBITS + 3)) == 0) { + zm <<= 1; + ze--; + } + + } + return ieee754sp_format(zs, ze, zm); +} diff --git a/arch/mips/math-emu/sp_msubf.c b/arch/mips/math-emu/sp_msubf.c new file mode 100644 index 00000000000000..81c38b980d69a9 --- /dev/null +++ b/arch/mips/math-emu/sp_msubf.c @@ -0,0 +1,258 @@ +/* + * IEEE754 floating point arithmetic + * single precision: MSUB.f (Fused Multiply Subtract) + * MSUBF.fmt: FPR[fd] = FPR[fd] - (FPR[fs] x FPR[ft]) + * + * MIPS floating point support + * Copyright (C) 2015 Imagination Technologies, Ltd. + * Author: Markos Chandras + * + * This program is free software; you can distribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; version 2 of the License. + */ + +#include "ieee754sp.h" + +union ieee754sp ieee754sp_msubf(union ieee754sp z, union ieee754sp x, + union ieee754sp y) +{ + int re; + int rs; + unsigned rm; + unsigned short lxm; + unsigned short hxm; + unsigned short lym; + unsigned short hym; + unsigned lrm; + unsigned hrm; + unsigned t; + unsigned at; + int s; + + COMPXSP; + COMPYSP; + u32 zm; int ze; int zs __maybe_unused; int zc; + + EXPLODEXSP; + EXPLODEYSP; + EXPLODESP(z, zc, zs, ze, zm) + + FLUSHXSP; + FLUSHYSP; + FLUSHSP(z, zc, zs, ze, zm); + + ieee754_clearcx(); + + switch (zc) { + case IEEE754_CLASS_SNAN: + ieee754_setcx(IEEE754_INVALID_OPERATION); + return ieee754sp_nanxcpt(z); + case IEEE754_CLASS_DNORM: + SPDNORMx(zm, ze); + /* QNAN is handled separately below */ + } + + switch (CLPAIR(xc, yc)) { + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN): + return ieee754sp_nanxcpt(y); + + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF): + return ieee754sp_nanxcpt(x); + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN): + return y; + + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF): + return x; + + /* + * Infinity handling + */ + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF): + if (zc == IEEE754_CLASS_QNAN) + return z; + ieee754_setcx(IEEE754_INVALID_OPERATION); + return ieee754sp_indef(); + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF): + if (zc == IEEE754_CLASS_QNAN) + return z; + return ieee754sp_inf(xs ^ ys); + + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM): + case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM): + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO): + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO): + if (zc == IEEE754_CLASS_INF) + return ieee754sp_inf(zs); + /* Multiplication is 0 so just return z */ + return z; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM): + SPDNORMX; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754sp_inf(zs); + SPDNORMY; + break; + + case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754sp_inf(zs); + SPDNORMX; + break; + + case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_NORM): + if (zc == IEEE754_CLASS_QNAN) + return z; + else if (zc == IEEE754_CLASS_INF) + return ieee754sp_inf(zs); + /* fall through to real compuation */ + } + + /* Finally get to do some computation */ + + /* + * Do the multiplication bit first + * + * rm = xm * ym, re = xe + ye basically + * + * At this point xm and ym should have been normalized. + */ + + /* rm = xm * ym, re = xe+ye basically */ + assert(xm & SP_HIDDEN_BIT); + assert(ym & SP_HIDDEN_BIT); + + re = xe + ye; + rs = xs ^ ys; + + /* shunt to top of word */ + xm <<= 32 - (SP_FBITS + 1); + ym <<= 32 - (SP_FBITS + 1); + + /* + * Multiply 32 bits xm, ym to give high 32 bits rm with stickness. + */ + lxm = xm & 0xffff; + hxm = xm >> 16; + lym = ym & 0xffff; + hym = ym >> 16; + + lrm = lxm * lym; /* 16 * 16 => 32 */ + hrm = hxm * hym; /* 16 * 16 => 32 */ + + t = lxm * hym; /* 16 * 16 => 32 */ + at = lrm + (t << 16); + hrm += at < lrm; + lrm = at; + hrm = hrm + (t >> 16); + + t = hxm * lym; /* 16 * 16 => 32 */ + at = lrm + (t << 16); + hrm += at < lrm; + lrm = at; + hrm = hrm + (t >> 16); + + rm = hrm | (lrm != 0); + + /* + * Sticky shift down to normal rounding precision. + */ + if ((int) rm < 0) { + rm = (rm >> (32 - (SP_FBITS + 1 + 3))) | + ((rm << (SP_FBITS + 1 + 3)) != 0); + re++; + } else { + rm = (rm >> (32 - (SP_FBITS + 1 + 3 + 1))) | + ((rm << (SP_FBITS + 1 + 3 + 1)) != 0); + } + assert(rm & (SP_HIDDEN_BIT << 3)); + + /* And now the subtraction */ + + /* Flip sign of r and handle as add */ + rs ^= 1; + + assert(zm & SP_HIDDEN_BIT); + + /* + * Provide guard,round and stick bit space. + */ + zm <<= 3; + + if (ze > re) { + /* + * Have to shift y fraction right to align. + */ + s = ze - re; + SPXSRSYn(s); + } else if (re > ze) { + /* + * Have to shift x fraction right to align. + */ + s = re - ze; + SPXSRSYn(s); + } + assert(ze == re); + assert(ze <= SP_EMAX); + + if (zs == rs) { + /* + * Generate 28 bit result of adding two 27 bit numbers + * leaving result in zm, zs and ze. + */ + zm = zm + rm; + + if (zm >> (SP_FBITS + 1 + 3)) { /* carry out */ + SPXSRSX1(); /* shift preserving sticky */ + } + } else { + if (zm >= rm) { + zm = zm - rm; + } else { + zm = rm - zm; + zs = rs; + } + if (zm == 0) + return ieee754sp_zero(ieee754_csr.rm == FPU_CSR_RD); + + /* + * Normalize in extended single precision + */ + while ((zm >> (SP_MBITS + 3)) == 0) { + zm <<= 1; + ze--; + } + + } + return ieee754sp_format(zs, ze, zm); +} diff --git a/arch/mips/mm/c-r4k.c b/arch/mips/mm/c-r4k.c index fbea4432f3f23f..5d3a25e1cfaea6 100644 --- a/arch/mips/mm/c-r4k.c +++ b/arch/mips/mm/c-r4k.c @@ -1276,6 +1276,7 @@ static void probe_pcache(void) case CPU_PROAPTIV: case CPU_M5150: case CPU_QEMU_GENERIC: + case CPU_I6400: if (!(read_c0_config7() & MIPS_CONF7_IAR) && (c->icache.waysize > PAGE_SIZE)) c->icache.flags |= MIPS_CACHE_ALIASES; diff --git a/arch/mips/mm/dma-default.c b/arch/mips/mm/dma-default.c index eeaf0245c3b16f..8f23cf08f4baa6 100644 --- a/arch/mips/mm/dma-default.c +++ b/arch/mips/mm/dma-default.c @@ -194,6 +194,40 @@ static void mips_dma_free_coherent(struct device *dev, size_t size, void *vaddr, __free_pages(page, get_order(size)); } +static int mips_dma_mmap(struct device *dev, struct vm_area_struct *vma, + void *cpu_addr, dma_addr_t dma_addr, size_t size, + struct dma_attrs *attrs) +{ + unsigned long user_count = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT; + unsigned long count = PAGE_ALIGN(size) >> PAGE_SHIFT; + unsigned long addr = (unsigned long)cpu_addr; + unsigned long off = vma->vm_pgoff; + unsigned long pfn; + int ret = -ENXIO; + + if (!plat_device_is_coherent(dev) && !hw_coherentio) + addr = CAC_ADDR(addr); + + pfn = page_to_pfn(virt_to_page((void *)addr)); + + if (dma_get_attr(DMA_ATTR_WRITE_COMBINE, attrs)) + vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot); + else + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + + if (dma_mmap_from_coherent(dev, vma, cpu_addr, size, &ret)) + return ret; + + if (off < count && user_count <= (count - off)) { + ret = remap_pfn_range(vma, vma->vm_start, + pfn + off, + user_count << PAGE_SHIFT, + vma->vm_page_prot); + } + + return ret; +} + static inline void __dma_sync_virtual(void *addr, size_t size, enum dma_data_direction direction) { @@ -380,6 +414,7 @@ EXPORT_SYMBOL(dma_cache_sync); static struct dma_map_ops mips_default_dma_map_ops = { .alloc = mips_dma_alloc_coherent, .free = mips_dma_free_coherent, + .mmap = mips_dma_mmap, .map_page = mips_dma_map_page, .unmap_page = mips_dma_unmap_page, .map_sg = mips_dma_map_sg, diff --git a/arch/mips/mm/fault.c b/arch/mips/mm/fault.c index 852a41c6da4507..4b88fa031891d4 100644 --- a/arch/mips/mm/fault.c +++ b/arch/mips/mm/fault.c @@ -57,12 +57,10 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, unsigned long write, #ifdef CONFIG_KPROBES /* - * This is to notify the fault handler of the kprobes. The - * exception code is redundant as it is also carried in REGS, - * but we pass it anyhow. + * This is to notify the fault handler of the kprobes. */ if (notify_die(DIE_PAGE_FAULT, "page fault", regs, -1, - (regs->cp0_cause >> 2) & 0x1f, SIGSEGV) == NOTIFY_STOP) + current->thread.trap_nr, SIGSEGV) == NOTIFY_STOP) return; #endif @@ -224,6 +222,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, unsigned long write, print_vma_addr(" ", regs->regs[31]); pr_info("\n"); } + current->thread.trap_nr = (regs->cp0_cause >> 2) & 0x1f; info.si_signo = SIGSEGV; info.si_errno = 0; /* info.si_code has been set above */ @@ -282,6 +281,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, unsigned long write, field, (unsigned long) regs->cp0_epc, field, (unsigned long) regs->regs[31]); #endif + current->thread.trap_nr = (regs->cp0_cause >> 2) & 0x1f; tsk->thread.cp0_badvaddr = address; info.si_signo = SIGBUS; info.si_errno = 0; diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c index 198a3147dd7d08..66d0f49c5bec4b 100644 --- a/arch/mips/mm/init.c +++ b/arch/mips/mm/init.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -333,9 +334,40 @@ static inline void mem_init_free_highmem(void) #endif } -unsigned __weak platform_maar_init(unsigned num_maars) +unsigned __weak platform_maar_init(unsigned num_pairs) { - return 0; + struct maar_config cfg[BOOT_MEM_MAP_MAX]; + unsigned i, num_configured, num_cfg = 0; + phys_addr_t skip; + + for (i = 0; i < boot_mem_map.nr_map; i++) { + switch (boot_mem_map.map[i].type) { + case BOOT_MEM_RAM: + case BOOT_MEM_INIT_RAM: + break; + default: + continue; + } + + skip = 0x10000 - (boot_mem_map.map[i].addr & 0xffff); + + cfg[num_cfg].lower = boot_mem_map.map[i].addr; + cfg[num_cfg].lower += skip; + + cfg[num_cfg].upper = cfg[num_cfg].lower; + cfg[num_cfg].upper += boot_mem_map.map[i].size - 1; + cfg[num_cfg].upper -= skip; + + cfg[num_cfg].attrs = MIPS_MAAR_S; + num_cfg++; + } + + num_configured = maar_config(cfg, num_cfg, num_pairs); + if (num_configured < num_cfg) + pr_warn("Not enough MAAR pairs (%u) for all bootmem regions (%u)\n", + num_pairs, num_cfg); + + return num_configured; } static void maar_init(void) diff --git a/arch/mips/mm/sc-mips.c b/arch/mips/mm/sc-mips.c index 4ceafd13870cd6..53ea8391f9bbf9 100644 --- a/arch/mips/mm/sc-mips.c +++ b/arch/mips/mm/sc-mips.c @@ -14,6 +14,7 @@ #include #include #include +#include /* * MIPS32/MIPS64 L2 cache handling @@ -94,6 +95,38 @@ static inline int mips_sc_is_activated(struct cpuinfo_mips *c) return 1; } +static int __init mips_sc_probe_cm3(void) +{ + struct cpuinfo_mips *c = ¤t_cpu_data; + unsigned long cfg = read_gcr_l2_config(); + unsigned long sets, line_sz, assoc; + + if (cfg & CM_GCR_L2_CONFIG_BYPASS_MSK) + return 0; + + sets = cfg & CM_GCR_L2_CONFIG_SET_SIZE_MSK; + sets >>= CM_GCR_L2_CONFIG_SET_SIZE_SHF; + c->scache.sets = 64 << sets; + + line_sz = cfg & CM_GCR_L2_CONFIG_LINE_SIZE_MSK; + line_sz >>= CM_GCR_L2_CONFIG_LINE_SIZE_SHF; + c->scache.linesz = 2 << line_sz; + + assoc = cfg & CM_GCR_L2_CONFIG_ASSOC_MSK; + assoc >>= CM_GCR_L2_CONFIG_ASSOC_SHF; + c->scache.ways = assoc + 1; + c->scache.waysize = c->scache.sets * c->scache.linesz; + c->scache.waybit = __ffs(c->scache.waysize); + + c->scache.flags &= ~MIPS_CACHE_NOT_PRESENT; + + return 1; +} + +void __weak platform_early_l2_init(void) +{ +} + static inline int __init mips_sc_probe(void) { struct cpuinfo_mips *c = ¤t_cpu_data; @@ -103,6 +136,15 @@ static inline int __init mips_sc_probe(void) /* Mark as not present until probe completed */ c->scache.flags |= MIPS_CACHE_NOT_PRESENT; + /* + * Do we need some platform specific probing before + * we configure L2? + */ + platform_early_l2_init(); + + if (mips_cm_revision() >= CM_REV_CM3) + return mips_sc_probe_cm3(); + /* Ignore anything but MIPSxx processors */ if (!(c->isa_level & (MIPS_CPU_ISA_M32R1 | MIPS_CPU_ISA_M32R2 | MIPS_CPU_ISA_M32R6 | MIPS_CPU_ISA_M64R1 | diff --git a/arch/mips/mm/tlb-r3k.c b/arch/mips/mm/tlb-r3k.c index 2b75b8f880ed1f..b4f366f7c0f57a 100644 --- a/arch/mips/mm/tlb-r3k.c +++ b/arch/mips/mm/tlb-r3k.c @@ -36,7 +36,7 @@ extern void build_tlb_refill_handler(void); "nop\n\t" \ ".set pop\n\t") -static int r3k_have_wired_reg; /* Should be in cpu_data? */ +int r3k_have_wired_reg; /* Should be in cpu_data? */ /* TLB operations. */ static void local_flush_tlb_from(int entry) diff --git a/arch/mips/mti-malta/malta-init.c b/arch/mips/mti-malta/malta-init.c index cec3e187c48f4d..53c24784a2f7bd 100644 --- a/arch/mips/mti-malta/malta-init.c +++ b/arch/mips/mti-malta/malta-init.c @@ -303,3 +303,10 @@ void __init prom_init(void) if (!register_vsmp_smp_ops()) return; } + +void platform_early_l2_init(void) +{ + /* L2 configuration lives in the CM3 */ + if (mips_cm_revision() >= CM_REV_CM3) + mips_cm_probe(); +} diff --git a/arch/mips/mti-malta/malta-int.c b/arch/mips/mti-malta/malta-int.c index fa8f591f371361..c6a6c7afddab41 100644 --- a/arch/mips/mti-malta/malta-int.c +++ b/arch/mips/mti-malta/malta-int.c @@ -382,122 +382,12 @@ void malta_be_init(void) /* Could change CM error mask register. */ } - -static char *tr[8] = { - "mem", "gcr", "gic", "mmio", - "0x04", "0x05", "0x06", "0x07" -}; - -static char *mcmd[32] = { - [0x00] = "0x00", - [0x01] = "Legacy Write", - [0x02] = "Legacy Read", - [0x03] = "0x03", - [0x04] = "0x04", - [0x05] = "0x05", - [0x06] = "0x06", - [0x07] = "0x07", - [0x08] = "Coherent Read Own", - [0x09] = "Coherent Read Share", - [0x0a] = "Coherent Read Discard", - [0x0b] = "Coherent Ready Share Always", - [0x0c] = "Coherent Upgrade", - [0x0d] = "Coherent Writeback", - [0x0e] = "0x0e", - [0x0f] = "0x0f", - [0x10] = "Coherent Copyback", - [0x11] = "Coherent Copyback Invalidate", - [0x12] = "Coherent Invalidate", - [0x13] = "Coherent Write Invalidate", - [0x14] = "Coherent Completion Sync", - [0x15] = "0x15", - [0x16] = "0x16", - [0x17] = "0x17", - [0x18] = "0x18", - [0x19] = "0x19", - [0x1a] = "0x1a", - [0x1b] = "0x1b", - [0x1c] = "0x1c", - [0x1d] = "0x1d", - [0x1e] = "0x1e", - [0x1f] = "0x1f" -}; - -static char *core[8] = { - "Invalid/OK", "Invalid/Data", - "Shared/OK", "Shared/Data", - "Modified/OK", "Modified/Data", - "Exclusive/OK", "Exclusive/Data" -}; - -static char *causes[32] = { - "None", "GC_WR_ERR", "GC_RD_ERR", "COH_WR_ERR", - "COH_RD_ERR", "MMIO_WR_ERR", "MMIO_RD_ERR", "0x07", - "0x08", "0x09", "0x0a", "0x0b", - "0x0c", "0x0d", "0x0e", "0x0f", - "0x10", "0x11", "0x12", "0x13", - "0x14", "0x15", "0x16", "INTVN_WR_ERR", - "INTVN_RD_ERR", "0x19", "0x1a", "0x1b", - "0x1c", "0x1d", "0x1e", "0x1f" -}; - int malta_be_handler(struct pt_regs *regs, int is_fixup) { /* This duplicates the handling in do_be which seems wrong */ int retval = is_fixup ? MIPS_BE_FIXUP : MIPS_BE_FATAL; - if (mips_cm_present()) { - unsigned long cm_error = read_gcr_error_cause(); - unsigned long cm_addr = read_gcr_error_addr(); - unsigned long cm_other = read_gcr_error_mult(); - unsigned long cause, ocause; - char buf[256]; - - cause = cm_error & CM_GCR_ERROR_CAUSE_ERRTYPE_MSK; - if (cause != 0) { - cause >>= CM_GCR_ERROR_CAUSE_ERRTYPE_SHF; - if (cause < 16) { - unsigned long cca_bits = (cm_error >> 15) & 7; - unsigned long tr_bits = (cm_error >> 12) & 7; - unsigned long cmd_bits = (cm_error >> 7) & 0x1f; - unsigned long stag_bits = (cm_error >> 3) & 15; - unsigned long sport_bits = (cm_error >> 0) & 7; - - snprintf(buf, sizeof(buf), - "CCA=%lu TR=%s MCmd=%s STag=%lu " - "SPort=%lu\n", - cca_bits, tr[tr_bits], mcmd[cmd_bits], - stag_bits, sport_bits); - } else { - /* glob state & sresp together */ - unsigned long c3_bits = (cm_error >> 18) & 7; - unsigned long c2_bits = (cm_error >> 15) & 7; - unsigned long c1_bits = (cm_error >> 12) & 7; - unsigned long c0_bits = (cm_error >> 9) & 7; - unsigned long sc_bit = (cm_error >> 8) & 1; - unsigned long cmd_bits = (cm_error >> 3) & 0x1f; - unsigned long sport_bits = (cm_error >> 0) & 7; - snprintf(buf, sizeof(buf), - "C3=%s C2=%s C1=%s C0=%s SC=%s " - "MCmd=%s SPort=%lu\n", - core[c3_bits], core[c2_bits], - core[c1_bits], core[c0_bits], - sc_bit ? "True" : "False", - mcmd[cmd_bits], sport_bits); - } - - ocause = (cm_other & CM_GCR_ERROR_MULT_ERR2ND_MSK) >> - CM_GCR_ERROR_MULT_ERR2ND_SHF; - - pr_err("CM_ERROR=%08lx %s <%s>\n", cm_error, - causes[cause], buf); - pr_err("CM_ADDR =%08lx\n", cm_addr); - pr_err("CM_OTHER=%08lx %s\n", cm_other, causes[ocause]); - - /* reprime cause register */ - write_gcr_error_cause(0); - } - } + mips_cm_error_report(); return retval; } diff --git a/arch/mips/mti-malta/malta-memory.c b/arch/mips/mti-malta/malta-memory.c index b769657be4d460..dadeb837918282 100644 --- a/arch/mips/mti-malta/malta-memory.c +++ b/arch/mips/mti-malta/malta-memory.c @@ -179,31 +179,6 @@ void __init prom_free_prom_memory(void) } } -unsigned platform_maar_init(unsigned num_pairs) -{ - phys_addr_t mem_end = (physical_memsize & ~0xffffull) - 1; - struct maar_config cfg[] = { - /* DRAM preceding I/O */ - { 0x00000000, 0x0fffffff, MIPS_MAAR_S }, - - /* DRAM following I/O */ - { 0x20000000, mem_end, MIPS_MAAR_S }, - - /* DRAM alias in upper half of physical */ - { 0x80000000, 0x80000000 + mem_end, MIPS_MAAR_S }, - }; - unsigned i, num_cfg = ARRAY_SIZE(cfg); - - /* If DRAM fits before I/O, drop the region following it */ - if (physical_memsize <= 0x10000000) { - num_cfg--; - for (i = 1; i < num_cfg; i++) - cfg[i] = cfg[i + 1]; - } - - return maar_config(cfg, num_cfg, num_pairs); -} - phys_addr_t mips_cdmm_phys_base(void) { /* This address is "typically unused" */ diff --git a/arch/mips/netlogic/common/irq.c b/arch/mips/netlogic/common/irq.c index 5f5d18b0e94d8a..3660dc67d544fb 100644 --- a/arch/mips/netlogic/common/irq.c +++ b/arch/mips/netlogic/common/irq.c @@ -87,7 +87,7 @@ struct nlm_pic_irq { static void xlp_pic_enable(struct irq_data *d) { unsigned long flags; - struct nlm_pic_irq *pd = irq_data_get_irq_handler_data(d); + struct nlm_pic_irq *pd = irq_data_get_irq_chip_data(d); BUG_ON(!pd); spin_lock_irqsave(&pd->node->piclock, flags); @@ -97,7 +97,7 @@ static void xlp_pic_enable(struct irq_data *d) static void xlp_pic_disable(struct irq_data *d) { - struct nlm_pic_irq *pd = irq_data_get_irq_handler_data(d); + struct nlm_pic_irq *pd = irq_data_get_irq_chip_data(d); unsigned long flags; BUG_ON(!pd); @@ -108,7 +108,7 @@ static void xlp_pic_disable(struct irq_data *d) static void xlp_pic_mask_ack(struct irq_data *d) { - struct nlm_pic_irq *pd = irq_data_get_irq_handler_data(d); + struct nlm_pic_irq *pd = irq_data_get_irq_chip_data(d); clear_c0_eimr(pd->picirq); ack_c0_eirr(pd->picirq); @@ -116,7 +116,7 @@ static void xlp_pic_mask_ack(struct irq_data *d) static void xlp_pic_unmask(struct irq_data *d) { - struct nlm_pic_irq *pd = irq_data_get_irq_handler_data(d); + struct nlm_pic_irq *pd = irq_data_get_irq_chip_data(d); BUG_ON(!pd); @@ -193,7 +193,7 @@ void nlm_setup_pic_irq(int node, int picirq, int irq, int irt) pic_data->picirq = picirq; pic_data->node = nlm_get_node(node); irq_set_chip_and_handler(xirq, &xlp_pic, handle_level_irq); - irq_set_handler_data(xirq, pic_data); + irq_set_chip_data(xirq, pic_data); } void nlm_set_pic_extra_ack(int node, int irq, void (*xack)(struct irq_data *)) @@ -202,7 +202,7 @@ void nlm_set_pic_extra_ack(int node, int irq, void (*xack)(struct irq_data *)) int xirq; xirq = nlm_irq_to_xirq(node, irq); - pic_data = irq_get_handler_data(xirq); + pic_data = irq_get_chip_data(xirq); if (WARN_ON(!pic_data)) return; pic_data->extra_ack = xack; diff --git a/arch/mips/netlogic/common/smp.c b/arch/mips/netlogic/common/smp.c index f5fff228b347b6..0136b4f9c9cde0 100644 --- a/arch/mips/netlogic/common/smp.c +++ b/arch/mips/netlogic/common/smp.c @@ -82,8 +82,9 @@ void nlm_send_ipi_mask(const struct cpumask *mask, unsigned int action) } /* IRQ_IPI_SMP_FUNCTION Handler */ -void nlm_smp_function_ipi_handler(unsigned int irq, struct irq_desc *desc) +void nlm_smp_function_ipi_handler(unsigned int __irq, struct irq_desc *desc) { + unsigned int irq = irq_desc_get_irq(desc); clear_c0_eimr(irq); ack_c0_eirr(irq); generic_smp_call_function_interrupt(); @@ -91,8 +92,9 @@ void nlm_smp_function_ipi_handler(unsigned int irq, struct irq_desc *desc) } /* IRQ_IPI_SMP_RESCHEDULE handler */ -void nlm_smp_resched_ipi_handler(unsigned int irq, struct irq_desc *desc) +void nlm_smp_resched_ipi_handler(unsigned int __irq, struct irq_desc *desc) { + unsigned int irq = irq_desc_get_irq(desc); clear_c0_eimr(irq); ack_c0_eirr(irq); scheduler_ipi(); diff --git a/arch/mips/netlogic/xlp/ahci-init-xlp2.c b/arch/mips/netlogic/xlp/ahci-init-xlp2.c index 7b066a44e6798e..c11b9c7dc7c8ea 100644 --- a/arch/mips/netlogic/xlp/ahci-init-xlp2.c +++ b/arch/mips/netlogic/xlp/ahci-init-xlp2.c @@ -152,7 +152,7 @@ static const u8 sata_phy_config1[] = { 0xC9, 0xC9, 0x07, 0x07, 0x18, 0x18, 0x01, 0x01, 0x22, 0x00 }; -/* SATA PHY config for register block 2 0x0x8065 .. 0x0x80A4 */ +/* SATA PHY config for register block 2 0x8065 .. 0x80A4 */ static const u8 sata_phy_config2[] = { 0xAA, 0x00, 0x4C, 0xC9, 0xC9, 0x07, 0x07, 0x18, 0x18, 0x05, 0x0C, 0x10, 0x00, 0x10, 0x00, 0xFF, diff --git a/arch/mips/netlogic/xlp/nlm_hal.c b/arch/mips/netlogic/xlp/nlm_hal.c index a8f4144a0297fc..80ec929747c37b 100644 --- a/arch/mips/netlogic/xlp/nlm_hal.c +++ b/arch/mips/netlogic/xlp/nlm_hal.c @@ -91,6 +91,8 @@ static int xlp9xx_irq_to_irt(int irq) return 134; case PIC_SATA_IRQ: return 143; + case PIC_NAND_IRQ: + return 151; case PIC_SPI_IRQ: return 152; case PIC_MMC_IRQ: diff --git a/arch/mips/oprofile/common.c b/arch/mips/oprofile/common.c index 81f58958cf081b..3c9ec3ddca845d 100644 --- a/arch/mips/oprofile/common.c +++ b/arch/mips/oprofile/common.c @@ -91,6 +91,7 @@ int __init oprofile_arch_init(struct oprofile_operations *ops) case CPU_INTERAPTIV: case CPU_PROAPTIV: case CPU_P5600: + case CPU_I6400: case CPU_M5150: case CPU_LOONGSON1: case CPU_SB1: diff --git a/arch/mips/oprofile/op_model_mipsxx.c b/arch/mips/oprofile/op_model_mipsxx.c index 6a6e2cc55b8926..8f988a61b7a850 100644 --- a/arch/mips/oprofile/op_model_mipsxx.c +++ b/arch/mips/oprofile/op_model_mipsxx.c @@ -392,6 +392,10 @@ static int __init mipsxx_init(void) op_model_mipsxx_ops.cpu_type = "mips/P5600"; break; + case CPU_I6400: + op_model_mipsxx_ops.cpu_type = "mips/I6400"; + break; + case CPU_M5150: op_model_mipsxx_ops.cpu_type = "mips/M5150"; break; diff --git a/arch/mips/pci/msi-xlp.c b/arch/mips/pci/msi-xlp.c index 3407495fcbe2fd..bb14335f804b4a 100644 --- a/arch/mips/pci/msi-xlp.c +++ b/arch/mips/pci/msi-xlp.c @@ -131,7 +131,7 @@ struct xlp_msi_data { */ static void xlp_msi_enable(struct irq_data *d) { - struct xlp_msi_data *md = irq_data_get_irq_handler_data(d); + struct xlp_msi_data *md = irq_data_get_irq_chip_data(d); unsigned long flags; int vec; @@ -148,7 +148,7 @@ static void xlp_msi_enable(struct irq_data *d) static void xlp_msi_disable(struct irq_data *d) { - struct xlp_msi_data *md = irq_data_get_irq_handler_data(d); + struct xlp_msi_data *md = irq_data_get_irq_chip_data(d); unsigned long flags; int vec; @@ -165,7 +165,7 @@ static void xlp_msi_disable(struct irq_data *d) static void xlp_msi_mask_ack(struct irq_data *d) { - struct xlp_msi_data *md = irq_data_get_irq_handler_data(d); + struct xlp_msi_data *md = irq_data_get_irq_chip_data(d); int link, vec; link = nlm_irq_msilink(d->irq); @@ -211,7 +211,7 @@ static void xlp_msix_mask_ack(struct irq_data *d) msixvec = nlm_irq_msixvec(d->irq); link = nlm_irq_msixlink(msixvec); pci_msi_mask_irq(d); - md = irq_data_get_irq_handler_data(d); + md = irq_data_get_irq_chip_data(d); /* Ack MSI on bridge */ if (cpu_is_xlp9xx()) { @@ -302,7 +302,7 @@ static int xlp_setup_msi(uint64_t lnkbase, int node, int link, /* Get MSI data for the link */ lirq = PIC_PCIE_LINK_MSI_IRQ(link); xirq = nlm_irq_to_xirq(node, nlm_link_msiirq(link, 0)); - md = irq_get_handler_data(xirq); + md = irq_get_chip_data(xirq); msiaddr = MSI_LINK_ADDR(node, link); spin_lock_irqsave(&md->msi_lock, flags); @@ -409,7 +409,7 @@ static int xlp_setup_msix(uint64_t lnkbase, int node, int link, /* Get MSI data for the link */ lirq = PIC_PCIE_MSIX_IRQ(link); xirq = nlm_irq_to_xirq(node, nlm_link_msixirq(link, 0)); - md = irq_get_handler_data(xirq); + md = irq_get_chip_data(xirq); msixaddr = MSIX_LINK_ADDR(node, link); spin_lock_irqsave(&md->msi_lock, flags); @@ -485,7 +485,7 @@ void __init xlp_init_node_msi_irqs(int node, int link) irq = nlm_irq_to_xirq(node, nlm_link_msiirq(link, 0)); for (i = irq; i < irq + XLP_MSIVEC_PER_LINK; i++) { irq_set_chip_and_handler(i, &xlp_msi_chip, handle_level_irq); - irq_set_handler_data(i, md); + irq_set_chip_data(i, md); } for (i = 0; i < XLP_MSIXVEC_PER_LINK ; i++) { @@ -508,7 +508,7 @@ void __init xlp_init_node_msi_irqs(int node, int link) /* Initialize MSI-X extended irq space for the link */ irq = nlm_irq_to_xirq(node, nlm_link_msixirq(link, i)); irq_set_chip_and_handler(irq, &xlp_msix_chip, handle_level_irq); - irq_set_handler_data(irq, md); + irq_set_chip_data(irq, md); } } @@ -520,7 +520,7 @@ void nlm_dispatch_msi(int node, int lirq) link = lirq - PIC_PCIE_LINK_MSI_IRQ_BASE; irqbase = nlm_irq_to_xirq(node, nlm_link_msiirq(link, 0)); - md = irq_get_handler_data(irqbase); + md = irq_get_chip_data(irqbase); if (cpu_is_xlp9xx()) status = nlm_read_reg(md->lnkbase, PCIE_9XX_MSI_STATUS) & md->msi_enabled_mask; @@ -550,7 +550,7 @@ void nlm_dispatch_msix(int node, int lirq) link = lirq - PIC_PCIE_MSIX_IRQ_BASE; irqbase = nlm_irq_to_xirq(node, nlm_link_msixirq(link, 0)); - md = irq_get_handler_data(irqbase); + md = irq_get_chip_data(irqbase); if (cpu_is_xlp9xx()) status = nlm_read_reg(md->lnkbase, PCIE_9XX_MSIX_STATUSX(link)); else diff --git a/arch/mips/pci/ops-emma2rh.c b/arch/mips/pci/ops-emma2rh.c index 710aef5c070e83..2dc97c45685e8b 100644 --- a/arch/mips/pci/ops-emma2rh.c +++ b/arch/mips/pci/ops-emma2rh.c @@ -25,7 +25,6 @@ #include #include -#include #include @@ -40,10 +39,9 @@ static int check_args(struct pci_bus *bus, u32 devfn, u32 * bus_num) { /* check if the bus is top-level */ - if (bus->parent != NULL) { + if (bus->parent != NULL) *bus_num = bus->number; - db_assert(bus_num != NULL); - } else + else *bus_num = 0; if (*bus_num == 0) { diff --git a/arch/mips/pci/pci-ar71xx.c b/arch/mips/pci/pci-ar71xx.c index 283157f8dc6479..ad35a5e6a56c5b 100644 --- a/arch/mips/pci/pci-ar71xx.c +++ b/arch/mips/pci/pci-ar71xx.c @@ -312,8 +312,8 @@ static void ar71xx_pci_irq_init(struct ar71xx_pci_controller *apc) irq_set_chip_data(i, apc); } - irq_set_handler_data(apc->irq, apc); - irq_set_chained_handler(apc->irq, ar71xx_pci_irq_handler); + irq_set_chained_handler_and_data(apc->irq, ar71xx_pci_irq_handler, + apc); } static void ar71xx_pci_reset(void) diff --git a/arch/mips/pci/pci-ar724x.c b/arch/mips/pci/pci-ar724x.c index 0af362b5af9266..907d11dd921b3d 100644 --- a/arch/mips/pci/pci-ar724x.c +++ b/arch/mips/pci/pci-ar724x.c @@ -321,8 +321,8 @@ static void ar724x_pci_irq_init(struct ar724x_pci_controller *apc, irq_set_chip_data(i, apc); } - irq_set_handler_data(apc->irq, apc); - irq_set_chained_handler(apc->irq, ar724x_pci_irq_handler); + irq_set_chained_handler_and_data(apc->irq, ar724x_pci_irq_handler, + apc); } static int ar724x_pci_probe(struct platform_device *pdev) diff --git a/arch/mips/pci/pci-lantiq.c b/arch/mips/pci/pci-lantiq.c index c5347d99cf3a2b..6a15dbd085aa8e 100644 --- a/arch/mips/pci/pci-lantiq.c +++ b/arch/mips/pci/pci-lantiq.c @@ -20,7 +20,6 @@ #include #include -#include #include #include diff --git a/arch/mips/pci/pci-rt3883.c b/arch/mips/pci/pci-rt3883.c index 80fafe646e7411..53c8efaf15723c 100644 --- a/arch/mips/pci/pci-rt3883.c +++ b/arch/mips/pci/pci-rt3883.c @@ -129,7 +129,7 @@ static void rt3883_pci_write_cfg32(struct rt3883_pci_controller *rpc, rt3883_pci_w32(rpc, val, RT3883_PCI_REG_CFGDATA); } -static void rt3883_pci_irq_handler(unsigned int irq, struct irq_desc *desc) +static void rt3883_pci_irq_handler(unsigned int __irq, struct irq_desc *desc) { struct rt3883_pci_controller *rpc; u32 pending; @@ -145,7 +145,7 @@ static void rt3883_pci_irq_handler(unsigned int irq, struct irq_desc *desc) } while (pending) { - unsigned bit = __ffs(pending); + unsigned irq, bit = __ffs(pending); irq = irq_find_mapping(rpc->irq_domain, bit); generic_handle_irq(irq); @@ -225,8 +225,7 @@ static int rt3883_pci_irq_init(struct device *dev, return -ENODEV; } - irq_set_handler_data(irq, rpc); - irq_set_chained_handler(irq, rt3883_pci_irq_handler); + irq_set_chained_handler_and_data(irq, rt3883_pci_irq_handler, rpc); return 0; } diff --git a/arch/mips/pistachio/Kconfig b/arch/mips/pistachio/Kconfig new file mode 100644 index 00000000000000..97731ea94322a8 --- /dev/null +++ b/arch/mips/pistachio/Kconfig @@ -0,0 +1,13 @@ +config PISTACHIO_GPTIMER_CLKSRC + bool "Enable General Purpose Timer based clocksource" + depends on MACH_PISTACHIO + select CLKSRC_PISTACHIO + select MIPS_EXTERNAL_TIMER + help + This option enables a clocksource driver based on a Pistachio + SoC General Purpose external timer. + + If you want to enable the CPUFreq, you need to enable + this option. + + If you don't want to enable CPUFreq, you can leave this disabled. diff --git a/arch/mips/pmcs-msp71xx/msp_irq_cic.c b/arch/mips/pmcs-msp71xx/msp_irq_cic.c index 1207ec4dfb7702..8b9cf64630406e 100644 --- a/arch/mips/pmcs-msp71xx/msp_irq_cic.c +++ b/arch/mips/pmcs-msp71xx/msp_irq_cic.c @@ -88,7 +88,8 @@ static void unmask_cic_irq(struct irq_data *d) * Make sure we have IRQ affinity. It may have changed while * we were processing the IRQ. */ - if (!cpumask_test_cpu(smp_processor_id(), d->affinity)) + if (!cpumask_test_cpu(smp_processor_id(), + irq_data_get_affinity_mask(d))) return; #endif diff --git a/arch/mips/ralink/cevt-rt3352.c b/arch/mips/ralink/cevt-rt3352.c index 24bf057a36136b..a8e70a9f274bdf 100644 --- a/arch/mips/ralink/cevt-rt3352.c +++ b/arch/mips/ralink/cevt-rt3352.c @@ -36,8 +36,8 @@ struct systick_device { int freq_scale; }; -static void systick_set_clock_mode(enum clock_event_mode mode, - struct clock_event_device *evt); +static int systick_set_oneshot(struct clock_event_device *evt); +static int systick_shutdown(struct clock_event_device *evt); static int systick_next_event(unsigned long delta, struct clock_event_device *evt) @@ -73,11 +73,12 @@ static struct systick_device systick = { * cevt-r4k uses 300, make sure systick * gets used if available */ - .rating = 310, - .features = CLOCK_EVT_FEAT_ONESHOT, - .set_next_event = systick_next_event, - .set_mode = systick_set_clock_mode, - .event_handler = systick_event_handler, + .rating = 310, + .features = CLOCK_EVT_FEAT_ONESHOT, + .set_next_event = systick_next_event, + .set_state_shutdown = systick_shutdown, + .set_state_oneshot = systick_set_oneshot, + .event_handler = systick_event_handler, }, }; @@ -87,33 +88,33 @@ static struct irqaction systick_irqaction = { .dev_id = &systick.dev, }; -static void systick_set_clock_mode(enum clock_event_mode mode, - struct clock_event_device *evt) +static int systick_shutdown(struct clock_event_device *evt) { struct systick_device *sdev; sdev = container_of(evt, struct systick_device, dev); - switch (mode) { - case CLOCK_EVT_MODE_ONESHOT: - if (!sdev->irq_requested) - setup_irq(systick.dev.irq, &systick_irqaction); - sdev->irq_requested = 1; - iowrite32(CFG_EXT_STK_EN | CFG_CNT_EN, - systick.membase + SYSTICK_CONFIG); - break; - - case CLOCK_EVT_MODE_SHUTDOWN: - if (sdev->irq_requested) - free_irq(systick.dev.irq, &systick_irqaction); - sdev->irq_requested = 0; - iowrite32(0, systick.membase + SYSTICK_CONFIG); - break; - - default: - pr_err("%s: Unhandeled mips clock_mode\n", systick.dev.name); - break; - } + if (sdev->irq_requested) + free_irq(systick.dev.irq, &systick_irqaction); + sdev->irq_requested = 0; + iowrite32(0, systick.membase + SYSTICK_CONFIG); + + return 0; +} + +static int systick_set_oneshot(struct clock_event_device *evt) +{ + struct systick_device *sdev; + + sdev = container_of(evt, struct systick_device, dev); + + if (!sdev->irq_requested) + setup_irq(systick.dev.irq, &systick_irqaction); + sdev->irq_requested = 1; + iowrite32(CFG_EXT_STK_EN | CFG_CNT_EN, + systick.membase + SYSTICK_CONFIG); + + return 0; } static void __init ralink_systick_init(struct device_node *np) diff --git a/arch/mips/rb532/devices.c b/arch/mips/rb532/devices.c index e31e8cdcb296bf..9bd7a2de0765fc 100644 --- a/arch/mips/rb532/devices.c +++ b/arch/mips/rb532/devices.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include diff --git a/arch/mips/rb532/gpio.c b/arch/mips/rb532/gpio.c index 5aa3df8530826d..650d5d39f34d90 100644 --- a/arch/mips/rb532/gpio.c +++ b/arch/mips/rb532/gpio.c @@ -140,6 +140,11 @@ static int rb532_gpio_direction_output(struct gpio_chip *chip, return 0; } +static int rb532_gpio_to_irq(struct gpio_chip *chip, unsigned gpio) +{ + return 8 + 4 * 32 + gpio; +} + static struct rb532_gpio_chip rb532_gpio_chip[] = { [0] = { .chip = { @@ -148,6 +153,7 @@ static struct rb532_gpio_chip rb532_gpio_chip[] = { .direction_output = rb532_gpio_direction_output, .get = rb532_gpio_get, .set = rb532_gpio_set, + .to_irq = rb532_gpio_to_irq, .base = 0, .ngpio = 32, }, diff --git a/arch/mips/sgi-ip27/ip27-timer.c b/arch/mips/sgi-ip27/ip27-timer.c index a6d10f607f348c..42d6cb9f956e35 100644 --- a/arch/mips/sgi-ip27/ip27-timer.c +++ b/arch/mips/sgi-ip27/ip27-timer.c @@ -64,12 +64,6 @@ static int rt_next_event(unsigned long delta, struct clock_event_device *evt) return LOCAL_HUB_L(PI_RT_COUNT) >= cnt ? -ETIME : 0; } -static void rt_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) -{ - /* Nothing to do ... */ -} - unsigned int rt_timer_irq; static DEFINE_PER_CPU(struct clock_event_device, hub_rt_clockevent); @@ -124,7 +118,6 @@ void hub_rt_clock_event_init(void) cd->irq = irq; cd->cpumask = cpumask_of(cpu); cd->set_next_event = rt_next_event; - cd->set_mode = rt_set_mode; clockevents_register_device(cd); } diff --git a/arch/mips/sibyte/common/bus_watcher.c b/arch/mips/sibyte/common/bus_watcher.c index 41a1d22422112c..a4e55999ecb4c7 100644 --- a/arch/mips/sibyte/common/bus_watcher.c +++ b/arch/mips/sibyte/common/bus_watcher.c @@ -250,4 +250,4 @@ int __init sibyte_bus_watcher(void) return 0; } -__initcall(sibyte_bus_watcher); +device_initcall(sibyte_bus_watcher); diff --git a/arch/mips/sni/time.c b/arch/mips/sni/time.c index cf8ec568b9dfad..fb4b3520cdc615 100644 --- a/arch/mips/sni/time.c +++ b/arch/mips/sni/time.c @@ -14,44 +14,33 @@ #define SNI_COUNTER2_DIV 64 #define SNI_COUNTER0_DIV ((SNI_CLOCK_TICK_RATE / SNI_COUNTER2_DIV) / HZ) -static void a20r_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) +static int a20r_set_periodic(struct clock_event_device *evt) { - switch (mode) { - case CLOCK_EVT_MODE_PERIODIC: - *(volatile u8 *)(A20R_PT_CLOCK_BASE + 12) = 0x34; - wmb(); - *(volatile u8 *)(A20R_PT_CLOCK_BASE + 0) = SNI_COUNTER0_DIV; - wmb(); - *(volatile u8 *)(A20R_PT_CLOCK_BASE + 0) = SNI_COUNTER0_DIV >> 8; - wmb(); - - *(volatile u8 *)(A20R_PT_CLOCK_BASE + 12) = 0xb4; - wmb(); - *(volatile u8 *)(A20R_PT_CLOCK_BASE + 8) = SNI_COUNTER2_DIV; - wmb(); - *(volatile u8 *)(A20R_PT_CLOCK_BASE + 8) = SNI_COUNTER2_DIV >> 8; - wmb(); + *(volatile u8 *)(A20R_PT_CLOCK_BASE + 12) = 0x34; + wmb(); + *(volatile u8 *)(A20R_PT_CLOCK_BASE + 0) = SNI_COUNTER0_DIV; + wmb(); + *(volatile u8 *)(A20R_PT_CLOCK_BASE + 0) = SNI_COUNTER0_DIV >> 8; + wmb(); - break; - case CLOCK_EVT_MODE_ONESHOT: - case CLOCK_EVT_MODE_UNUSED: - case CLOCK_EVT_MODE_SHUTDOWN: - break; - case CLOCK_EVT_MODE_RESUME: - break; - } + *(volatile u8 *)(A20R_PT_CLOCK_BASE + 12) = 0xb4; + wmb(); + *(volatile u8 *)(A20R_PT_CLOCK_BASE + 8) = SNI_COUNTER2_DIV; + wmb(); + *(volatile u8 *)(A20R_PT_CLOCK_BASE + 8) = SNI_COUNTER2_DIV >> 8; + wmb(); + return 0; } static struct clock_event_device a20r_clockevent_device = { - .name = "a20r-timer", - .features = CLOCK_EVT_FEAT_PERIODIC, + .name = "a20r-timer", + .features = CLOCK_EVT_FEAT_PERIODIC, /* .mult, .shift, .max_delta_ns and .min_delta_ns left uninitialized */ - .rating = 300, - .irq = SNI_A20R_IRQ_TIMER, - .set_mode = a20r_set_mode, + .rating = 300, + .irq = SNI_A20R_IRQ_TIMER, + .set_state_periodic = a20r_set_periodic, }; static irqreturn_t a20r_interrupt(int irq, void *dev_id) diff --git a/arch/mips/txx9/generic/setup.c b/arch/mips/txx9/generic/setup.c index 2791b8641df64c..9d9962ab7d25cc 100644 --- a/arch/mips/txx9/generic/setup.c +++ b/arch/mips/txx9/generic/setup.c @@ -117,22 +117,6 @@ void clk_put(struct clk *clk) } EXPORT_SYMBOL(clk_put); -/* GPIO support */ - -#ifdef CONFIG_GPIOLIB -int gpio_to_irq(unsigned gpio) -{ - return -EINVAL; -} -EXPORT_SYMBOL(gpio_to_irq); - -int irq_to_gpio(unsigned irq) -{ - return -EINVAL; -} -EXPORT_SYMBOL(irq_to_gpio); -#endif - #define BOARD_VEC(board) extern struct txx9_board_vec board; #include #undef BOARD_VEC diff --git a/arch/mn10300/include/asm/atomic.h b/arch/mn10300/include/asm/atomic.h index 5be655e83e709d..375e59140c9cef 100644 --- a/arch/mn10300/include/asm/atomic.h +++ b/arch/mn10300/include/asm/atomic.h @@ -89,6 +89,10 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP @@ -127,73 +131,6 @@ static inline void atomic_dec(atomic_t *v) #define atomic_xchg(ptr, v) (xchg(&(ptr)->counter, (v))) #define atomic_cmpxchg(v, old, new) (cmpxchg(&((v)->counter), (old), (new))) -/** - * atomic_clear_mask - Atomically clear bits in memory - * @mask: Mask of the bits to be cleared - * @v: pointer to word in memory - * - * Atomically clears the bits set in mask from the memory word specified. - */ -static inline void atomic_clear_mask(unsigned long mask, unsigned long *addr) -{ -#ifdef CONFIG_SMP - int status; - - asm volatile( - "1: mov %3,(_AAR,%2) \n" - " mov (_ADR,%2),%0 \n" - " and %4,%0 \n" - " mov %0,(_ADR,%2) \n" - " mov (_ADR,%2),%0 \n" /* flush */ - " mov (_ASR,%2),%0 \n" - " or %0,%0 \n" - " bne 1b \n" - : "=&r"(status), "=m"(*addr) - : "a"(ATOMIC_OPS_BASE_ADDR), "r"(addr), "r"(~mask) - : "memory", "cc"); -#else - unsigned long flags; - - mask = ~mask; - flags = arch_local_cli_save(); - *addr &= mask; - arch_local_irq_restore(flags); -#endif -} - -/** - * atomic_set_mask - Atomically set bits in memory - * @mask: Mask of the bits to be set - * @v: pointer to word in memory - * - * Atomically sets the bits set in mask from the memory word specified. - */ -static inline void atomic_set_mask(unsigned long mask, unsigned long *addr) -{ -#ifdef CONFIG_SMP - int status; - - asm volatile( - "1: mov %3,(_AAR,%2) \n" - " mov (_ADR,%2),%0 \n" - " or %4,%0 \n" - " mov %0,(_ADR,%2) \n" - " mov (_ADR,%2),%0 \n" /* flush */ - " mov (_ASR,%2),%0 \n" - " or %0,%0 \n" - " bne 1b \n" - : "=&r"(status), "=m"(*addr) - : "a"(ATOMIC_OPS_BASE_ADDR), "r"(addr), "r"(mask) - : "memory", "cc"); -#else - unsigned long flags; - - flags = arch_local_cli_save(); - *addr |= mask; - arch_local_irq_restore(flags); -#endif -} - #endif /* __KERNEL__ */ #endif /* CONFIG_SMP */ #endif /* _ASM_ATOMIC_H */ diff --git a/arch/mn10300/mm/tlb-smp.c b/arch/mn10300/mm/tlb-smp.c index e5d0ef722bfa86..9a39ea9031d4f7 100644 --- a/arch/mn10300/mm/tlb-smp.c +++ b/arch/mn10300/mm/tlb-smp.c @@ -119,7 +119,7 @@ static void flush_tlb_others(cpumask_t cpumask, struct mm_struct *mm, flush_mm = mm; flush_va = va; #if NR_CPUS <= BITS_PER_LONG - atomic_set_mask(cpumask.bits[0], &flush_cpumask.bits[0]); + atomic_or(cpumask.bits[0], (atomic_t *)&flush_cpumask.bits[0]); #else #error Not supported. #endif diff --git a/arch/parisc/configs/c8000_defconfig b/arch/parisc/configs/c8000_defconfig index 269c23d23fcb6f..1a8f6f95689e64 100644 --- a/arch/parisc/configs/c8000_defconfig +++ b/arch/parisc/configs/c8000_defconfig @@ -242,7 +242,6 @@ CONFIG_LOCKUP_DETECTOR=y CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=y CONFIG_PANIC_ON_OOPS=y CONFIG_DEBUG_RT_MUTEXES=y -CONFIG_RT_MUTEX_TESTER=y CONFIG_PROVE_RCU_DELAY=y CONFIG_DEBUG_BLOCK_EXT_DEVT=y CONFIG_LATENCYTOP=y diff --git a/arch/parisc/configs/generic-32bit_defconfig b/arch/parisc/configs/generic-32bit_defconfig index 33b148f825baa1..0ffb08ff51252a 100644 --- a/arch/parisc/configs/generic-32bit_defconfig +++ b/arch/parisc/configs/generic-32bit_defconfig @@ -295,7 +295,6 @@ CONFIG_DEBUG_SHIRQ=y CONFIG_DETECT_HUNG_TASK=y CONFIG_TIMER_STATS=y CONFIG_DEBUG_RT_MUTEXES=y -CONFIG_RT_MUTEX_TESTER=y CONFIG_DEBUG_SPINLOCK=y CONFIG_DEBUG_MUTEXES=y CONFIG_RCU_CPU_STALL_INFO=y diff --git a/arch/parisc/include/asm/atomic.h b/arch/parisc/include/asm/atomic.h index 226f8ca993f693..2536965d00eae4 100644 --- a/arch/parisc/include/asm/atomic.h +++ b/arch/parisc/include/asm/atomic.h @@ -126,6 +126,10 @@ static __inline__ int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add, +=) ATOMIC_OPS(sub, -=) +ATOMIC_OP(and, &=) +ATOMIC_OP(or, |=) +ATOMIC_OP(xor, ^=) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP @@ -185,6 +189,9 @@ static __inline__ s64 atomic64_##op##_return(s64 i, atomic64_t *v) \ ATOMIC64_OPS(add, +=) ATOMIC64_OPS(sub, -=) +ATOMIC64_OP(and, &=) +ATOMIC64_OP(or, |=) +ATOMIC64_OP(xor, ^=) #undef ATOMIC64_OPS #undef ATOMIC64_OP_RETURN diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 5ef27113b898c3..b447918b9e2c8b 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -82,6 +82,9 @@ config GENERIC_HWEIGHT bool default y +config ARCH_HAS_DMA_SET_COHERENT_MASK + bool + config PPC bool default y @@ -155,6 +158,8 @@ config PPC select HAVE_PERF_EVENTS_NMI if PPC64 select EDAC_SUPPORT select EDAC_ATOMIC_SCRUB + select ARCH_HAS_DMA_SET_COHERENT_MASK + select HAVE_ARCH_SECCOMP_FILTER config GENERIC_CSUM def_bool CPU_LITTLE_ENDIAN @@ -514,11 +519,6 @@ config NODES_SPAN_OTHER_NODES def_bool y depends on NEED_MULTIPLE_NODES -config PPC_HAS_HASH_64K - bool - depends on PPC64 - default n - config STDBINUTILS bool "Using standard binutils settings" depends on 44x @@ -560,16 +560,16 @@ config PPC_4K_PAGES bool "4k page size" config PPC_16K_PAGES - bool "16k page size" if 44x || PPC_8xx + bool "16k page size" + depends on 44x || PPC_8xx config PPC_64K_PAGES - bool "64k page size" if 44x || PPC_STD_MMU_64 || PPC_BOOK3E_64 - depends on !PPC_FSL_BOOK3E - select PPC_HAS_HASH_64K if PPC_STD_MMU_64 + bool "64k page size" + depends on !PPC_FSL_BOOK3E && (44x || PPC_STD_MMU_64 || PPC_BOOK3E_64) config PPC_256K_PAGES - bool "256k page size" if 44x - depends on !STDBINUTILS + bool "256k page size" + depends on 44x && !STDBINUTILS help Make the page size 256k. diff --git a/arch/powerpc/Makefile b/arch/powerpc/Makefile index 05f464eb69527e..4ca54fdd8768c2 100644 --- a/arch/powerpc/Makefile +++ b/arch/powerpc/Makefile @@ -288,6 +288,26 @@ PHONY += pseries_le_defconfig pseries_le_defconfig: $(call merge_into_defconfig,pseries_defconfig,le) +PHONY += mpc85xx_defconfig +mpc85xx_defconfig: + $(call merge_into_defconfig,mpc85xx_basic_defconfig,\ + 85xx-32bit 85xx-hw fsl-emb-nonhw) + +PHONY += mpc85xx_smp_defconfig +mpc85xx_smp_defconfig: + $(call merge_into_defconfig,mpc85xx_basic_defconfig,\ + 85xx-32bit 85xx-smp 85xx-hw fsl-emb-nonhw) + +PHONY += corenet32_smp_defconfig +corenet32_smp_defconfig: + $(call merge_into_defconfig,corenet_basic_defconfig,\ + 85xx-32bit 85xx-smp 85xx-hw fsl-emb-nonhw) + +PHONY += corenet64_smp_defconfig +corenet64_smp_defconfig: + $(call merge_into_defconfig,corenet_basic_defconfig,\ + 85xx-64bit 85xx-smp altivec 85xx-hw fsl-emb-nonhw) + define archhelp @echo '* zImage - Build default images selected by kernel config' @echo ' zImage.* - Compressed kernel image (arch/$(ARCH)/boot/zImage.*)' diff --git a/arch/powerpc/boot/dts/fsl/p1022si-post.dtsi b/arch/powerpc/boot/dts/fsl/p1022si-post.dtsi index ebf20223454986..426bf4103b9ee6 100644 --- a/arch/powerpc/boot/dts/fsl/p1022si-post.dtsi +++ b/arch/powerpc/boot/dts/fsl/p1022si-post.dtsi @@ -175,7 +175,7 @@ /include/ "pq3-gpio-0.dtsi" - display@10000 { + display: display@10000 { compatible = "fsl,diu", "fsl,p1022-diu"; reg = <0x10000 1000>; interrupts = <64 2 0 0>; diff --git a/arch/powerpc/boot/dts/fsl/p1022si-pre.dtsi b/arch/powerpc/boot/dts/fsl/p1022si-pre.dtsi index 1956dea040cca3..de76ae8992c6eb 100644 --- a/arch/powerpc/boot/dts/fsl/p1022si-pre.dtsi +++ b/arch/powerpc/boot/dts/fsl/p1022si-pre.dtsi @@ -50,6 +50,8 @@ pci0 = &pci0; pci1 = &pci1; pci2 = &pci2; + vga = &display; + display = &display; }; cpus { diff --git a/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi b/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi index 9e9f7e201d43a2..9770d027849371 100644 --- a/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi +++ b/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi @@ -484,6 +484,11 @@ reg = <0xea000 0x4000>; }; + scfg: global-utilities@fc000 { + compatible = "fsl,t1040-scfg"; + reg = <0xfc000 0x1000>; + }; + /include/ "elo3-dma-0.dtsi" /include/ "elo3-dma-1.dtsi" /include/ "qoriq-espi-0.dtsi" diff --git a/arch/powerpc/boot/dts/t1023rdb.dts b/arch/powerpc/boot/dts/t1023rdb.dts index 06b090aba0662a..d3fa8294cd4906 100644 --- a/arch/powerpc/boot/dts/t1023rdb.dts +++ b/arch/powerpc/boot/dts/t1023rdb.dts @@ -60,7 +60,7 @@ #address-cells = <1>; #size-cells = <1>; compatible = "fsl,ifc-nand"; - reg = <0x2 0x0 0x10000>; + reg = <0x1 0x0 0x10000>; }; }; @@ -99,6 +99,17 @@ }; i2c@118100 { + current-sensor@40 { + compatible = "ti,ina220"; + reg = <0x40>; + shunt-resistor = <1000>; + }; + + current-sensor@41 { + compatible = "ti,ina220"; + reg = <0x41>; + shunt-resistor = <1000>; + }; }; }; diff --git a/arch/powerpc/boot/dts/t1024rdb.dts b/arch/powerpc/boot/dts/t1024rdb.dts index 733e723ffed621..bf05e324fda2e2 100644 --- a/arch/powerpc/boot/dts/t1024rdb.dts +++ b/arch/powerpc/boot/dts/t1024rdb.dts @@ -114,6 +114,12 @@ reg = <0x4c>; }; + current-sensor@40 { + compatible = "ti,ina220"; + reg = <0x40>; + shunt-resistor = <1000>; + }; + eeprom@50 { compatible = "atmel,24c256"; reg = <0x50>; diff --git a/arch/powerpc/boot/dts/t1040d4rdb.dts b/arch/powerpc/boot/dts/t1040d4rdb.dts new file mode 100644 index 00000000000000..2d1315a1670e82 --- /dev/null +++ b/arch/powerpc/boot/dts/t1040d4rdb.dts @@ -0,0 +1,46 @@ +/* + * T1040D4RDB Device Tree Source + * + * Copyright 2015 Freescale Semiconductor Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * * Neither the name of Freescale Semiconductor nor the + * names of its contributors may be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * + * ALTERNATIVELY, this software may be distributed under the terms of the + * GNU General Public License ("GPL") as published by the Free Software + * Foundation, either version 2 of that License or (at your option) any + * later version. + * + * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY + * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE + * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY + * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +/include/ "fsl/t104xsi-pre.dtsi" +/include/ "t104xd4rdb.dtsi" + +/ { + model = "fsl,T1040D4RDB"; + compatible = "fsl,T1040D4RDB"; + #address-cells = <2>; + #size-cells = <2>; + interrupt-parent = <&mpic>; +}; + +/include/ "fsl/t1040si-post.dtsi" diff --git a/arch/powerpc/boot/dts/t1042d4rdb.dts b/arch/powerpc/boot/dts/t1042d4rdb.dts new file mode 100644 index 00000000000000..846f8c87e85a24 --- /dev/null +++ b/arch/powerpc/boot/dts/t1042d4rdb.dts @@ -0,0 +1,53 @@ +/* + * T1042D4RDB Device Tree Source + * + * Copyright 2015 Freescale Semiconductor Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * * Neither the name of Freescale Semiconductor nor the + * names of its contributors may be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * + * ALTERNATIVELY, this software may be distributed under the terms of the + * GNU General Public License ("GPL") as published by the Free Software + * Foundation, either version 2 of that License or (at your option) any + * later version. + * + * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY + * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE + * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY + * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +/include/ "fsl/t104xsi-pre.dtsi" +/include/ "t104xd4rdb.dtsi" + +/ { + model = "fsl,T1042D4RDB"; + compatible = "fsl,T1042D4RDB"; + #address-cells = <2>; + #size-cells = <2>; + interrupt-parent = <&mpic>; + + ifc: localbus@ffe124000 { + cpld@3,0 { + compatible = "fsl,t1040d4rdb-cpld", + "fsl,deepsleep-cpld"; + }; + }; +}; + +/include/ "fsl/t1040si-post.dtsi" diff --git a/arch/powerpc/boot/dts/t104xd4rdb.dtsi b/arch/powerpc/boot/dts/t104xd4rdb.dtsi new file mode 100644 index 00000000000000..491367bd388311 --- /dev/null +++ b/arch/powerpc/boot/dts/t104xd4rdb.dtsi @@ -0,0 +1,205 @@ +/* + * T1040D4RDB/T1042D4RDB Device Tree Source + * + * Copyright 2015 Freescale Semiconductor Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * * Neither the name of Freescale Semiconductor nor the + * names of its contributors may be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * + * ALTERNATIVELY, this software may be distributed under the terms of the + * GNU General Public License ("GPL") as published by the Free Software + * Foundation, either version 2 of that License or (at your option) any + * later version. + * + * THIS SOFTWARE IS PROVIDED BY Freescale Semiconductor "AS IS" AND ANY + * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE + * DISCLAIMED. IN NO EVENT SHALL Freescale Semiconductor BE LIABLE FOR ANY + * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +/ { + reserved-memory { + #address-cells = <2>; + #size-cells = <2>; + ranges; + + bman_fbpr: bman-fbpr { + size = <0 0x1000000>; + alignment = <0 0x1000000>; + }; + qman_fqd: qman-fqd { + size = <0 0x400000>; + alignment = <0 0x400000>; + }; + qman_pfdr: qman-pfdr { + size = <0 0x2000000>; + alignment = <0 0x2000000>; + }; + }; + + ifc: localbus@ffe124000 { + reg = <0xf 0xfe124000 0 0x2000>; + ranges = <0 0 0xf 0xe8000000 0x08000000 + 2 0 0xf 0xff800000 0x00010000 + 3 0 0xf 0xffdf0000 0x00008000>; + + nor@0,0 { + #address-cells = <1>; + #size-cells = <1>; + compatible = "cfi-flash"; + reg = <0x0 0x0 0x8000000>; + bank-width = <2>; + device-width = <1>; + }; + + nand@2,0 { + #address-cells = <1>; + #size-cells = <1>; + compatible = "fsl,ifc-nand"; + reg = <0x2 0x0 0x10000>; + }; + + cpld@3,0 { + compatible = "fsl,t1040d4rdb-cpld"; + reg = <3 0 0x300>; + }; + }; + + memory { + device_type = "memory"; + }; + + dcsr: dcsr@f00000000 { + ranges = <0x00000000 0xf 0x00000000 0x01072000>; + }; + + bportals: bman-portals@ff4000000 { + ranges = <0x0 0xf 0xf4000000 0x2000000>; + }; + + qportals: qman-portals@ff6000000 { + ranges = <0x0 0xf 0xf6000000 0x2000000>; + }; + + soc: soc@ffe000000 { + ranges = <0x00000000 0xf 0xfe000000 0x1000000>; + reg = <0xf 0xfe000000 0 0x00001000>; + + spi@110000 { + flash@0 { + #address-cells = <1>; + #size-cells = <1>; + compatible = "micron,n25q512ax3"; + reg = <0>; + /* input clock */ + spi-max-frequency = <10000000>; + }; + }; + i2c@118000 { + hwmon@4c { + compatible = "adi,adt7461"; + reg = <0x4c>; + }; + + rtc@68 { + compatible = "dallas,ds1337"; + reg = <0x68>; + interrupts = <0x2 0x1 0 0>; + }; + }; + + i2c@118100 { + mux@77 { + /* + * Child nodes of mux depend on which i2c + * devices are connected via the mini PCI + * connector slot1, the mini PCI connector + * slot2, the HDMI connector, and the PEX + * slot. Systems with such devices attached + * should provide a wrapper .dts file that + * includes this one, and adds those nodes + */ + compatible = "nxp,pca9546"; + reg = <0x77>; + #address-cells = <1>; + #size-cells = <0>; + }; + }; + + }; + + pci0: pcie@ffe240000 { + reg = <0xf 0xfe240000 0 0x10000>; + ranges = <0x02000000 0 0xe0000000 0xc 0x0 0x0 0x10000000 + 0x01000000 0 0x0 0xf 0xf8000000 0x0 0x00010000>; + pcie@0 { + ranges = <0x02000000 0 0xe0000000 + 0x02000000 0 0xe0000000 + 0 0x10000000 + + 0x01000000 0 0x00000000 + 0x01000000 0 0x00000000 + 0 0x00010000>; + }; + }; + + pci1: pcie@ffe250000 { + reg = <0xf 0xfe250000 0 0x10000>; + ranges = <0x02000000 0 0xe0000000 0xc 0x10000000 0 0x10000000 + 0x01000000 0 0 0xf 0xf8010000 0 0x00010000>; + pcie@0 { + ranges = <0x02000000 0 0xe0000000 + 0x02000000 0 0xe0000000 + 0 0x10000000 + + 0x01000000 0 0x00000000 + 0x01000000 0 0x00000000 + 0 0x00010000>; + }; + }; + + pci2: pcie@ffe260000 { + reg = <0xf 0xfe260000 0 0x10000>; + ranges = <0x02000000 0 0xe0000000 0xc 0x20000000 0 0x10000000 + 0x01000000 0 0x00000000 0xf 0xf8020000 0 0x00010000>; + pcie@0 { + ranges = <0x02000000 0 0xe0000000 + 0x02000000 0 0xe0000000 + 0 0x10000000 + + 0x01000000 0 0x00000000 + 0x01000000 0 0x00000000 + 0 0x00010000>; + }; + }; + + pci3: pcie@ffe270000 { + reg = <0xf 0xfe270000 0 0x10000>; + ranges = <0x02000000 0 0xe0000000 0xc 0x30000000 0 0x10000000 + 0x01000000 0 0x00000000 0xf 0xf8030000 0 0x00010000>; + pcie@0 { + ranges = <0x02000000 0 0xe0000000 + 0x02000000 0 0xe0000000 + 0 0x10000000 + + 0x01000000 0 0x00000000 + 0x01000000 0 0x00000000 + 0 0x00010000>; + }; + }; +}; diff --git a/arch/powerpc/configs/85xx-32bit.config b/arch/powerpc/configs/85xx-32bit.config new file mode 100644 index 00000000000000..6b8894d727a26a --- /dev/null +++ b/arch/powerpc/configs/85xx-32bit.config @@ -0,0 +1,5 @@ +CONFIG_HIGHMEM=y +CONFIG_KEXEC=y +CONFIG_PPC_85xx=y +CONFIG_PROC_KCORE=y +CONFIG_PHYS_64BIT=y diff --git a/arch/powerpc/configs/85xx-64bit.config b/arch/powerpc/configs/85xx-64bit.config new file mode 100644 index 00000000000000..4aba81222885b8 --- /dev/null +++ b/arch/powerpc/configs/85xx-64bit.config @@ -0,0 +1,4 @@ +CONFIG_MATH_EMULATION=y +CONFIG_MATH_EMULATION_HW_UNIMPLEMENTED=y +CONFIG_PPC64=y +CONFIG_PPC_BOOK3E_64=y diff --git a/arch/powerpc/configs/85xx-hw.config b/arch/powerpc/configs/85xx-hw.config new file mode 100644 index 00000000000000..528ff0e714e685 --- /dev/null +++ b/arch/powerpc/configs/85xx-hw.config @@ -0,0 +1,142 @@ +CONFIG_AQUANTIA_PHY=y +CONFIG_AT803X_PHY=y +CONFIG_ATA=y +CONFIG_BLK_DEV_SD=y +CONFIG_BLK_DEV_SR_VENDOR=y +CONFIG_BLK_DEV_SR=y +CONFIG_BROADCOM_PHY=y +CONFIG_C293_PCIE=y +CONFIG_CHR_DEV_SG=y +CONFIG_CHR_DEV_ST=y +CONFIG_CICADA_PHY=y +CONFIG_CLK_QORIQ=y +CONFIG_CRYPTO_DEV_FSL_CAAM=y +CONFIG_CRYPTO_DEV_TALITOS=y +CONFIG_DAVICOM_PHY=y +CONFIG_DMADEVICES=y +CONFIG_E1000E=y +CONFIG_E1000=y +CONFIG_EDAC_MM_EDAC=y +CONFIG_EDAC_MPC85XX=y +CONFIG_EDAC=y +CONFIG_EEPROM_AT24=y +CONFIG_EEPROM_LEGACY=y +CONFIG_FB_FSL_DIU=y +CONFIG_FS_ENET=y +CONFIG_FSL_CORENET_CF=y +CONFIG_FSL_DMA=y +CONFIG_FSL_HV_MANAGER=y +CONFIG_FSL_PQ_MDIO=y +CONFIG_FSL_RIO=y +CONFIG_FSL_XGMAC_MDIO=y +CONFIG_GIANFAR=y +CONFIG_GPIO_MPC8XXX=y +CONFIG_HID_A4TECH=y +CONFIG_HID_APPLE=y +CONFIG_HID_BELKIN=y +CONFIG_HID_CHERRY=y +CONFIG_HID_CHICONY=y +CONFIG_HID_CYPRESS=y +CONFIG_HID_EZKEY=y +CONFIG_HID_GYRATION=y +CONFIG_HID_LOGITECH=y +CONFIG_HID_MICROSOFT=y +CONFIG_HID_MONTEREY=y +CONFIG_HID_PANTHERLORD=y +CONFIG_HID_PETALYNX=y +CONFIG_HID_SAMSUNG=y +CONFIG_HID_SUNPLUS=y +CONFIG_I2C_CHARDEV=y +CONFIG_I2C_CPM=m +CONFIG_I2C_MPC=y +CONFIG_I2C_MUX_PCA954x=y +CONFIG_I2C_MUX=y +CONFIG_I2C=y +CONFIG_IGB=y +CONFIG_INPUT_FF_MEMLESS=m +# CONFIG_INPUT_KEYBOARD is not set +# CONFIG_INPUT_MOUSEDEV is not set +# CONFIG_INPUT_MOUSE is not set +CONFIG_MARVELL_PHY=y +CONFIG_MDIO_BUS_MUX_GPIO=y +CONFIG_MDIO_BUS_MUX_MMIOREG=y +CONFIG_MMC_SDHCI_OF_ESDHC=y +CONFIG_MMC_SDHCI_PLTFM=y +CONFIG_MMC_SDHCI=y +CONFIG_MMC=y +CONFIG_MTD_BLOCK=y +CONFIG_MTD_CFI_AMDSTD=y +CONFIG_MTD_CFI_INTELEXT=y +CONFIG_MTD_CFI=y +CONFIG_MTD_CMDLINE_PARTS=y +CONFIG_MTD_M25P80=y +CONFIG_MTD_NAND_FSL_ELBC=y +CONFIG_MTD_NAND_FSL_IFC=y +CONFIG_MTD_NAND=y +CONFIG_MTD_PHYSMAP_OF=y +CONFIG_MTD_PHYSMAP=y +CONFIG_MTD_PLATRAM=y +CONFIG_MTD_SPI_NOR=y +CONFIG_NETDEVICES=y +CONFIG_NVRAM=y +CONFIG_PATA_ALI=y +CONFIG_PATA_SIL680=y +CONFIG_PATA_VIA=y +# CONFIG_PCIEASPM is not set +CONFIG_PCIEPORTBUS=y +CONFIG_PCI_MSI=y +CONFIG_PCI=y +CONFIG_PPC_EPAPR_HV_BYTECHAN=y +# CONFIG_PPC_OF_BOOT_TRAMPOLINE is not set +CONFIG_QE_GPIO=y +CONFIG_QUICC_ENGINE=y +CONFIG_RAPIDIO=y +CONFIG_RTC_CLASS=y +CONFIG_RTC_DRV_CMOS=y +CONFIG_RTC_DRV_DS1307=y +CONFIG_RTC_DRV_DS1374=y +CONFIG_RTC_DRV_DS3232=y +CONFIG_SATA_AHCI=y +CONFIG_SATA_FSL=y +CONFIG_SATA_SIL24=y +CONFIG_SATA_SIL=y +CONFIG_SCSI_LOGGING=y +CONFIG_SCSI_SYM53C8XX_2=y +CONFIG_SENSORS_INA2XX=y +CONFIG_SENSORS_LM90=y +CONFIG_SERIAL_8250_CONSOLE=y +CONFIG_SERIAL_8250_DETECT_IRQ=y +CONFIG_SERIAL_8250_MANY_PORTS=y +CONFIG_SERIAL_8250_NR_UARTS=6 +CONFIG_SERIAL_8250_RSA=y +CONFIG_SERIAL_8250_RUNTIME_UARTS=6 +CONFIG_SERIAL_8250=y +CONFIG_SERIAL_QE=m +CONFIG_SERIO_LIBPS2=y +# CONFIG_SND_DRIVERS is not set +CONFIG_SND_INTEL8X0=y +CONFIG_SND_POWERPC_SOC=y +# CONFIG_SND_PPC is not set +CONFIG_SND_SOC=y +# CONFIG_SND_SUPPORT_OLD_API is not set +# CONFIG_SND_USB is not set +CONFIG_SND=y +CONFIG_SOUND=y +CONFIG_SPI_FSL_ESPI=y +CONFIG_SPI_FSL_SPI=y +CONFIG_SPI_GPIO=y +CONFIG_SPI=y +CONFIG_TERANETICS_PHY=y +CONFIG_UCC_GETH=y +CONFIG_USB_EHCI_FSL=y +CONFIG_USB_EHCI_HCD=y +CONFIG_USB_HID=m +CONFIG_USB_MON=y +CONFIG_USB_OHCI_HCD_PPC_OF_BE=y +CONFIG_USB_OHCI_HCD_PPC_OF_LE=y +CONFIG_USB_OHCI_HCD=y +CONFIG_USB_STORAGE=y +CONFIG_USB=y +# CONFIG_VGA_CONSOLE is not set +CONFIG_VIRT_DRIVERS=y +CONFIG_VITESSE_PHY=y diff --git a/arch/powerpc/configs/85xx-smp.config b/arch/powerpc/configs/85xx-smp.config new file mode 100644 index 00000000000000..3b4d1e54636dea --- /dev/null +++ b/arch/powerpc/configs/85xx-smp.config @@ -0,0 +1,2 @@ +CONFIG_NR_CPUS=24 +CONFIG_SMP=y diff --git a/arch/powerpc/configs/altivec.config b/arch/powerpc/configs/altivec.config new file mode 100644 index 00000000000000..58a697cb5a620a --- /dev/null +++ b/arch/powerpc/configs/altivec.config @@ -0,0 +1 @@ +CONFIG_ALTIVEC=y diff --git a/arch/powerpc/configs/corenet32_smp_defconfig b/arch/powerpc/configs/corenet32_smp_defconfig deleted file mode 100644 index 37659937bd12d9..00000000000000 --- a/arch/powerpc/configs/corenet32_smp_defconfig +++ /dev/null @@ -1,185 +0,0 @@ -CONFIG_PPC_85xx=y -CONFIG_SMP=y -CONFIG_NR_CPUS=8 -CONFIG_SYSVIPC=y -CONFIG_POSIX_MQUEUE=y -CONFIG_AUDIT=y -CONFIG_NO_HZ=y -CONFIG_HIGH_RES_TIMERS=y -CONFIG_BSD_PROCESS_ACCT=y -CONFIG_IKCONFIG=y -CONFIG_IKCONFIG_PROC=y -CONFIG_LOG_BUF_SHIFT=14 -CONFIG_BLK_DEV_INITRD=y -CONFIG_KALLSYMS_ALL=y -CONFIG_EMBEDDED=y -CONFIG_PERF_EVENTS=y -CONFIG_SLAB=y -CONFIG_MODULES=y -CONFIG_MODULE_UNLOAD=y -CONFIG_MODULE_FORCE_UNLOAD=y -CONFIG_MODVERSIONS=y -# CONFIG_BLK_DEV_BSG is not set -CONFIG_PARTITION_ADVANCED=y -CONFIG_MAC_PARTITION=y -CONFIG_CORENET_GENERIC=y -CONFIG_HIGHMEM=y -# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set -CONFIG_BINFMT_MISC=m -CONFIG_KEXEC=y -CONFIG_FORCE_MAX_ZONEORDER=13 -CONFIG_PCI=y -CONFIG_PCIEPORTBUS=y -# CONFIG_PCIEASPM is not set -CONFIG_PCI_MSI=y -CONFIG_RAPIDIO=y -CONFIG_FSL_RIO=y -CONFIG_NET=y -CONFIG_PACKET=y -CONFIG_UNIX=y -CONFIG_XFRM_USER=y -CONFIG_XFRM_SUB_POLICY=y -CONFIG_XFRM_STATISTICS=y -CONFIG_NET_KEY=y -CONFIG_NET_KEY_MIGRATE=y -CONFIG_INET=y -CONFIG_IP_MULTICAST=y -CONFIG_IP_ADVANCED_ROUTER=y -CONFIG_IP_MULTIPLE_TABLES=y -CONFIG_IP_ROUTE_MULTIPATH=y -CONFIG_IP_ROUTE_VERBOSE=y -CONFIG_IP_PNP=y -CONFIG_IP_PNP_DHCP=y -CONFIG_IP_PNP_BOOTP=y -CONFIG_IP_PNP_RARP=y -CONFIG_NET_IPIP=y -CONFIG_IP_MROUTE=y -CONFIG_IP_PIMSM_V1=y -CONFIG_IP_PIMSM_V2=y -CONFIG_INET_AH=y -CONFIG_INET_ESP=y -CONFIG_INET_IPCOMP=y -# CONFIG_INET_LRO is not set -CONFIG_IPV6=y -CONFIG_IP_SCTP=m -CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" -CONFIG_DEVTMPFS=y -CONFIG_DEVTMPFS_MOUNT=y -CONFIG_MTD=y -CONFIG_MTD_CMDLINE_PARTS=y -CONFIG_MTD_BLOCK=y -CONFIG_MTD_CFI=y -CONFIG_MTD_CFI_INTELEXT=y -CONFIG_MTD_CFI_AMDSTD=y -CONFIG_MTD_PHYSMAP_OF=y -CONFIG_MTD_NAND=y -CONFIG_MTD_NAND_FSL_ELBC=y -CONFIG_MTD_NAND_FSL_IFC=y -CONFIG_MTD_SPI_NOR=y -CONFIG_BLK_DEV_LOOP=y -CONFIG_BLK_DEV_RAM=y -CONFIG_BLK_DEV_RAM_SIZE=131072 -CONFIG_BLK_DEV_SD=y -CONFIG_CHR_DEV_ST=y -CONFIG_BLK_DEV_SR=y -CONFIG_CHR_DEV_SG=y -CONFIG_SCSI_LOGGING=y -CONFIG_SCSI_SYM53C8XX_2=y -CONFIG_ATA=y -CONFIG_SATA_AHCI=y -CONFIG_SATA_FSL=y -CONFIG_SATA_SIL24=y -CONFIG_SATA_SIL=y -CONFIG_PATA_SIL680=y -CONFIG_NETDEVICES=y -CONFIG_FSL_PQ_MDIO=y -CONFIG_FSL_XGMAC_MDIO=y -CONFIG_E1000=y -CONFIG_E1000E=y -CONFIG_AT803X_PHY=y -CONFIG_VITESSE_PHY=y -CONFIG_FIXED_PHY=y -CONFIG_MDIO_BUS_MUX_GPIO=y -CONFIG_MDIO_BUS_MUX_MMIOREG=y -# CONFIG_INPUT_MOUSEDEV is not set -# CONFIG_INPUT_KEYBOARD is not set -# CONFIG_INPUT_MOUSE is not set -CONFIG_SERIO_LIBPS2=y -# CONFIG_LEGACY_PTYS is not set -CONFIG_PPC_EPAPR_HV_BYTECHAN=y -CONFIG_SERIAL_8250=y -CONFIG_SERIAL_8250_CONSOLE=y -CONFIG_SERIAL_8250_MANY_PORTS=y -CONFIG_SERIAL_8250_DETECT_IRQ=y -CONFIG_SERIAL_8250_RSA=y -CONFIG_NVRAM=y -CONFIG_I2C=y -CONFIG_I2C_CHARDEV=y -CONFIG_I2C_MPC=y -CONFIG_I2C_MUX=y -CONFIG_I2C_MUX_PCA954x=y -CONFIG_SPI=y -CONFIG_SPI_GPIO=y -CONFIG_SPI_FSL_SPI=y -CONFIG_SPI_FSL_ESPI=y -CONFIG_SENSORS_LM90=y -CONFIG_SENSORS_INA2XX=y -CONFIG_USB_HID=m -CONFIG_USB=y -CONFIG_USB_MON=y -CONFIG_USB_EHCI_HCD=y -CONFIG_USB_EHCI_FSL=y -CONFIG_USB_OHCI_HCD=y -CONFIG_USB_OHCI_HCD_PPC_OF_BE=y -CONFIG_USB_OHCI_HCD_PPC_OF_LE=y -CONFIG_USB_STORAGE=y -CONFIG_MMC=y -CONFIG_MMC_SDHCI=y -CONFIG_EDAC=y -CONFIG_EDAC_MM_EDAC=y -CONFIG_EDAC_MPC85XX=y -CONFIG_RTC_CLASS=y -CONFIG_RTC_DRV_DS1307=y -CONFIG_RTC_DRV_DS1374=y -CONFIG_RTC_DRV_DS3232=y -CONFIG_UIO=y -CONFIG_VIRT_DRIVERS=y -CONFIG_FSL_HV_MANAGER=y -CONFIG_STAGING=y -CONFIG_FSL_CORENET_CF=y -CONFIG_CLK_QORIQ=y -CONFIG_EXT2_FS=y -CONFIG_EXT3_FS=y -# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set -CONFIG_ISO9660_FS=m -CONFIG_JOLIET=y -CONFIG_ZISOFS=y -CONFIG_UDF_FS=m -CONFIG_MSDOS_FS=m -CONFIG_VFAT_FS=y -CONFIG_NTFS_FS=y -CONFIG_PROC_KCORE=y -CONFIG_TMPFS=y -CONFIG_HUGETLBFS=y -CONFIG_JFFS2_FS=y -CONFIG_CRAMFS=y -CONFIG_NFS_FS=y -CONFIG_NFS_V4=y -CONFIG_ROOT_NFS=y -CONFIG_NFSD=m -CONFIG_NLS_CODEPAGE_437=y -CONFIG_NLS_CODEPAGE_850=y -CONFIG_NLS_ISO8859_1=y -CONFIG_NLS_UTF8=m -CONFIG_DEBUG_INFO=y -CONFIG_MAGIC_SYSRQ=y -CONFIG_DEBUG_SHIRQ=y -CONFIG_DETECT_HUNG_TASK=y -CONFIG_RCU_TRACE=y -CONFIG_CRYPTO_NULL=y -CONFIG_CRYPTO_PCBC=m -CONFIG_CRYPTO_MD4=y -CONFIG_CRYPTO_SHA256=y -CONFIG_CRYPTO_SHA512=y -# CONFIG_CRYPTO_ANSI_CPRNG is not set -CONFIG_CRYPTO_DEV_FSL_CAAM=y diff --git a/arch/powerpc/configs/corenet_basic_defconfig b/arch/powerpc/configs/corenet_basic_defconfig new file mode 100644 index 00000000000000..b568d465e59ee2 --- /dev/null +++ b/arch/powerpc/configs/corenet_basic_defconfig @@ -0,0 +1 @@ +CONFIG_CORENET_GENERIC=y diff --git a/arch/powerpc/configs/corenet64_smp_defconfig b/arch/powerpc/configs/fsl-emb-nonhw.config similarity index 50% rename from arch/powerpc/configs/corenet64_smp_defconfig rename to arch/powerpc/configs/fsl-emb-nonhw.config index 33cd1df818add3..41e4d359524d18 100644 --- a/arch/powerpc/configs/corenet64_smp_defconfig +++ b/arch/powerpc/configs/fsl-emb-nonhw.config @@ -1,176 +1,126 @@ -CONFIG_PPC64=y -CONFIG_PPC_BOOK3E_64=y -CONFIG_ALTIVEC=y -CONFIG_SMP=y -CONFIG_NR_CPUS=24 -CONFIG_SYSVIPC=y -CONFIG_FHANDLE=y -CONFIG_IRQ_DOMAIN_DEBUG=y -CONFIG_NO_HZ=y -CONFIG_HIGH_RES_TIMERS=y +CONFIG_ADFS_FS=m +CONFIG_AFFS_FS=m +CONFIG_AUDIT=y +CONFIG_BEFS_FS=m +CONFIG_BFS_FS=m +CONFIG_BINFMT_MISC=m +# CONFIG_BLK_DEV_BSG is not set +CONFIG_BLK_DEV_INITRD=y +CONFIG_BLK_DEV_LOOP=y +CONFIG_BLK_DEV_NBD=y +CONFIG_BLK_DEV_RAM_SIZE=131072 +CONFIG_BLK_DEV_RAM=y CONFIG_BSD_PROCESS_ACCT=y -CONFIG_IKCONFIG=y -CONFIG_IKCONFIG_PROC=y -CONFIG_LOG_BUF_SHIFT=14 -CONFIG_CGROUPS=y -CONFIG_CPUSETS=y CONFIG_CGROUP_CPUACCT=y CONFIG_CGROUP_SCHED=y -CONFIG_BLK_DEV_INITRD=y +CONFIG_CGROUPS=y +# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set +CONFIG_CRC_T10DIF=y +CONFIG_CPUSETS=y +CONFIG_CRAMFS=y +CONFIG_CRYPTO_MD4=y +CONFIG_CRYPTO_NULL=y +CONFIG_CRYPTO_PCBC=m +CONFIG_CRYPTO_SHA256=y +CONFIG_CRYPTO_SHA512=y +CONFIG_DEBUG_FS=y +CONFIG_DEBUG_INFO=y +CONFIG_DEBUG_SHIRQ=y +CONFIG_DETECT_HUNG_TASK=y +CONFIG_DEVTMPFS_MOUNT=y +CONFIG_DEVTMPFS=y +CONFIG_DUMMY=y +CONFIG_EFS_FS=m CONFIG_EXPERT=y -CONFIG_KALLSYMS_ALL=y -CONFIG_MODULES=y -CONFIG_MODULE_UNLOAD=y -CONFIG_MODULE_FORCE_UNLOAD=y -CONFIG_MODVERSIONS=y -# CONFIG_BLK_DEV_BSG is not set -CONFIG_PARTITION_ADVANCED=y -CONFIG_MAC_PARTITION=y -CONFIG_CORENET_GENERIC=y -# CONFIG_PPC_OF_BOOT_TRAMPOLINE is not set -CONFIG_BINFMT_MISC=m -CONFIG_MATH_EMULATION=y -CONFIG_MATH_EMULATION_HW_UNIMPLEMENTED=y -CONFIG_PCIEPORTBUS=y -CONFIG_PCI_MSI=y -CONFIG_RAPIDIO=y -CONFIG_FSL_RIO=y -CONFIG_NET=y -CONFIG_PACKET=y -CONFIG_UNIX=y -CONFIG_XFRM_USER=y -CONFIG_NET_KEY=y +CONFIG_EXT2_FS=y +# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set +CONFIG_EXT3_FS=y +CONFIG_FB=y +CONFIG_FHANDLE=y +CONFIG_FIXED_PHY=y +CONFIG_FONT_8x16=y +CONFIG_FONT_8x8=y +CONFIG_FONTS=y +CONFIG_FORCE_MAX_ZONEORDER=13 +CONFIG_FRAMEBUFFER_CONSOLE=y +CONFIG_FRAME_WARN=1024 +CONFIG_FTL=y +CONFIG_HFS_FS=m +CONFIG_HFSPLUS_FS=m +CONFIG_HIGH_RES_TIMERS=y +CONFIG_HPFS_FS=m +CONFIG_HUGETLBFS=y +CONFIG_IKCONFIG_PROC=y +CONFIG_IKCONFIG=y +CONFIG_INET_AH=y +CONFIG_INET_ESP=y +CONFIG_INET_IPCOMP=y +# CONFIG_INET_LRO is not set +# CONFIG_INET_XFRM_MODE_BEET is not set CONFIG_INET=y -CONFIG_IP_MULTICAST=y CONFIG_IP_ADVANCED_ROUTER=y -CONFIG_IP_MULTIPLE_TABLES=y -CONFIG_IP_ROUTE_MULTIPATH=y -CONFIG_IP_ROUTE_VERBOSE=y -CONFIG_IP_PNP=y -CONFIG_IP_PNP_DHCP=y -CONFIG_IP_PNP_BOOTP=y -CONFIG_IP_PNP_RARP=y -CONFIG_NET_IPIP=y CONFIG_IP_MROUTE=y +CONFIG_IP_MULTICAST=y +CONFIG_IP_MULTIPLE_TABLES=y CONFIG_IP_PIMSM_V1=y CONFIG_IP_PIMSM_V2=y -CONFIG_INET_ESP=y -# CONFIG_INET_XFRM_MODE_BEET is not set -# CONFIG_INET_LRO is not set -CONFIG_IPV6=y +CONFIG_IP_PNP_BOOTP=y +CONFIG_IP_PNP_DHCP=y +CONFIG_IP_PNP_RARP=y +CONFIG_IP_PNP=y +CONFIG_IP_ROUTE_MULTIPATH=y +CONFIG_IP_ROUTE_VERBOSE=y CONFIG_IP_SCTP=m -CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" -CONFIG_DEVTMPFS=y -CONFIG_DEVTMPFS_MOUNT=y -CONFIG_MTD=y -CONFIG_MTD_CMDLINE_PARTS=y -CONFIG_MTD_BLOCK=y -CONFIG_FTL=y -CONFIG_MTD_CFI=y -CONFIG_MTD_CFI_INTELEXT=y -CONFIG_MTD_CFI_AMDSTD=y -CONFIG_MTD_PHYSMAP_OF=y -CONFIG_MTD_NAND=y -CONFIG_MTD_NAND_FSL_ELBC=y -CONFIG_MTD_NAND_FSL_IFC=y -CONFIG_MTD_SPI_NOR=y -CONFIG_MTD_UBI=y -CONFIG_BLK_DEV_LOOP=y -CONFIG_BLK_DEV_RAM=y -CONFIG_BLK_DEV_RAM_SIZE=131072 -CONFIG_EEPROM_LEGACY=y -CONFIG_BLK_DEV_SD=y -CONFIG_BLK_DEV_SR=y -CONFIG_BLK_DEV_SR_VENDOR=y -CONFIG_CHR_DEV_SG=y -CONFIG_ATA=y -CONFIG_SATA_FSL=y -CONFIG_SATA_SIL24=y -CONFIG_NETDEVICES=y -CONFIG_DUMMY=y -CONFIG_FSL_PQ_MDIO=y -CONFIG_FSL_XGMAC_MDIO=y -CONFIG_E1000E=y -CONFIG_VITESSE_PHY=y -CONFIG_FIXED_PHY=y -CONFIG_MDIO_BUS_MUX_GPIO=y -CONFIG_MDIO_BUS_MUX_MMIOREG=y -CONFIG_INPUT_FF_MEMLESS=m -# CONFIG_INPUT_MOUSEDEV is not set -# CONFIG_INPUT_KEYBOARD is not set -# CONFIG_INPUT_MOUSE is not set -CONFIG_SERIO_LIBPS2=y -CONFIG_PPC_EPAPR_HV_BYTECHAN=y -CONFIG_SERIAL_8250=y -CONFIG_SERIAL_8250_CONSOLE=y -CONFIG_SERIAL_8250_MANY_PORTS=y -CONFIG_SERIAL_8250_DETECT_IRQ=y -CONFIG_SERIAL_8250_RSA=y -CONFIG_I2C=y -CONFIG_I2C_CHARDEV=y -CONFIG_I2C_MPC=y -CONFIG_I2C_MUX=y -CONFIG_I2C_MUX_PCA954x=y -CONFIG_SPI=y -CONFIG_SPI_GPIO=y -CONFIG_SPI_FSL_SPI=y -CONFIG_SPI_FSL_ESPI=y -CONFIG_SENSORS_LM90=y -CONFIG_SENSORS_INA2XX=y -CONFIG_USB_HID=m -CONFIG_USB=y -CONFIG_USB_MON=y -CONFIG_USB_EHCI_HCD=y -CONFIG_USB_EHCI_FSL=y -CONFIG_USB_STORAGE=y -CONFIG_MMC=y -CONFIG_MMC_SDHCI=y -CONFIG_EDAC=y -CONFIG_EDAC_MM_EDAC=y -CONFIG_RTC_CLASS=y -CONFIG_RTC_DRV_DS1307=y -CONFIG_RTC_DRV_DS1374=y -CONFIG_RTC_DRV_DS3232=y -CONFIG_DMADEVICES=y -CONFIG_FSL_DMA=y -CONFIG_VIRT_DRIVERS=y -CONFIG_FSL_HV_MANAGER=y -CONFIG_CLK_QORIQ=y -CONFIG_FSL_CORENET_CF=y -CONFIG_EXT2_FS=y -CONFIG_EXT3_FS=y +CONFIG_IPV6=y +CONFIG_IRQ_DOMAIN_DEBUG=y CONFIG_ISO9660_FS=m +CONFIG_JFFS2_FS_DEBUG=1 +CONFIG_JFFS2_FS=y CONFIG_JOLIET=y -CONFIG_ZISOFS=y -CONFIG_UDF_FS=m +CONFIG_KALLSYMS_ALL=y +# CONFIG_LEGACY_PTYS is not set +CONFIG_LOG_BUF_SHIFT=14 +CONFIG_MAC_PARTITION=y +CONFIG_MAGIC_SYSRQ=y +CONFIG_MODULE_FORCE_UNLOAD=y +CONFIG_MODULES=y +CONFIG_MODULE_UNLOAD=y +CONFIG_MODVERSIONS=y CONFIG_MSDOS_FS=m -CONFIG_VFAT_FS=y -CONFIG_NTFS_FS=y -CONFIG_PROC_KCORE=y -CONFIG_TMPFS=y -CONFIG_HUGETLBFS=y -CONFIG_JFFS2_FS=y -CONFIG_JFFS2_FS_DEBUG=1 -CONFIG_UBIFS_FS=y +CONFIG_MTD_UBI=y +CONFIG_MTD=y +CONFIG_NET_IPIP=y +CONFIG_NET_KEY_MIGRATE=y +CONFIG_NET_KEY=y +CONFIG_NET=y +CONFIG_NFSD=y CONFIG_NFS_FS=y CONFIG_NFS_V4=y -CONFIG_ROOT_NFS=y -CONFIG_NFSD=m CONFIG_NLS_CODEPAGE_437=y CONFIG_NLS_CODEPAGE_850=y CONFIG_NLS_ISO8859_1=y CONFIG_NLS_UTF8=m -CONFIG_CRC_T10DIF=y -CONFIG_DEBUG_INFO=y -CONFIG_FRAME_WARN=1024 -CONFIG_DEBUG_FS=y -CONFIG_MAGIC_SYSRQ=y -CONFIG_DEBUG_SHIRQ=y -CONFIG_DETECT_HUNG_TASK=y -CONFIG_CRYPTO_NULL=y -CONFIG_CRYPTO_PCBC=m -CONFIG_CRYPTO_MD4=y -CONFIG_CRYPTO_SHA256=y -CONFIG_CRYPTO_SHA512=y -# CONFIG_CRYPTO_ANSI_CPRNG is not set -CONFIG_CRYPTO_DEV_FSL_CAAM=y +CONFIG_NO_HZ=y +CONFIG_NTFS_FS=y +CONFIG_PACKET=y +CONFIG_PARTITION_ADVANCED=y +CONFIG_PERF_EVENTS=y +CONFIG_POSIX_MQUEUE=y +CONFIG_QNX4FS_FS=m +CONFIG_RCU_TRACE=y +CONFIG_ROOT_NFS=y +CONFIG_SYSV_FS=m +CONFIG_SYSVIPC=y +CONFIG_TMPFS=y +CONFIG_UBIFS_FS=y +CONFIG_UDF_FS=m +CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" +CONFIG_UFS_FS=m +CONFIG_UIO=y +CONFIG_UNIX=y +CONFIG_VFAT_FS=y +CONFIG_VXFS_FS=m +CONFIG_XFRM_STATISTICS=y +CONFIG_XFRM_SUB_POLICY=y +CONFIG_XFRM_USER=y +CONFIG_ZISOFS=y diff --git a/arch/powerpc/configs/mpc85xx_basic_defconfig b/arch/powerpc/configs/mpc85xx_basic_defconfig new file mode 100644 index 00000000000000..850bd195d0e816 --- /dev/null +++ b/arch/powerpc/configs/mpc85xx_basic_defconfig @@ -0,0 +1,23 @@ +CONFIG_MATH_EMULATION=y +CONFIG_MPC8536_DS=y +CONFIG_MPC8540_ADS=y +CONFIG_MPC8560_ADS=y +CONFIG_MPC85xx_CDS=y +CONFIG_MPC85xx_DS=y +CONFIG_MPC85xx_MDS=y +CONFIG_MPC85xx_RDB=y +CONFIG_KSI8560=y +CONFIG_MVME2500=y +CONFIG_P1010_RDB=y +CONFIG_P1022_DS=y +CONFIG_P1022_RDK=y +CONFIG_P1023_RDB=y +CONFIG_SBC8548=y +CONFIG_SOCRATES=y +CONFIG_STX_GP3=y +CONFIG_TQM8540=y +CONFIG_TQM8541=y +CONFIG_TQM8548=y +CONFIG_TQM8555=y +CONFIG_TQM8560=y +CONFIG_XES_MPC85xx=y diff --git a/arch/powerpc/configs/mpc85xx_defconfig b/arch/powerpc/configs/mpc85xx_defconfig deleted file mode 100644 index 6ecf7bdbc2f985..00000000000000 --- a/arch/powerpc/configs/mpc85xx_defconfig +++ /dev/null @@ -1,252 +0,0 @@ -CONFIG_PPC_85xx=y -CONFIG_PHYS_64BIT=y -CONFIG_SYSVIPC=y -CONFIG_POSIX_MQUEUE=y -CONFIG_AUDIT=y -CONFIG_IRQ_DOMAIN_DEBUG=y -CONFIG_NO_HZ=y -CONFIG_HIGH_RES_TIMERS=y -CONFIG_BSD_PROCESS_ACCT=y -CONFIG_IKCONFIG=y -CONFIG_IKCONFIG_PROC=y -CONFIG_LOG_BUF_SHIFT=14 -CONFIG_BLK_DEV_INITRD=y -CONFIG_EXPERT=y -CONFIG_KALLSYMS_ALL=y -CONFIG_MODULES=y -CONFIG_MODULE_UNLOAD=y -CONFIG_MODULE_FORCE_UNLOAD=y -CONFIG_MODVERSIONS=y -# CONFIG_BLK_DEV_BSG is not set -CONFIG_PARTITION_ADVANCED=y -CONFIG_MAC_PARTITION=y -CONFIG_C293_PCIE=y -CONFIG_MPC8540_ADS=y -CONFIG_MPC8560_ADS=y -CONFIG_MPC85xx_CDS=y -CONFIG_MPC85xx_MDS=y -CONFIG_MPC8536_DS=y -CONFIG_MPC85xx_DS=y -CONFIG_MPC85xx_RDB=y -CONFIG_P1010_RDB=y -CONFIG_P1022_DS=y -CONFIG_P1022_RDK=y -CONFIG_P1023_RDB=y -CONFIG_SOCRATES=y -CONFIG_KSI8560=y -CONFIG_XES_MPC85xx=y -CONFIG_STX_GP3=y -CONFIG_TQM8540=y -CONFIG_TQM8541=y -CONFIG_TQM8548=y -CONFIG_TQM8555=y -CONFIG_TQM8560=y -CONFIG_SBC8548=y -CONFIG_MVME2500=y -CONFIG_QUICC_ENGINE=y -CONFIG_QE_GPIO=y -CONFIG_HIGHMEM=y -CONFIG_BINFMT_MISC=m -CONFIG_MATH_EMULATION=y -CONFIG_FORCE_MAX_ZONEORDER=12 -CONFIG_PCI=y -CONFIG_PCIEPORTBUS=y -# CONFIG_PCIEASPM is not set -CONFIG_PCI_MSI=y -CONFIG_RAPIDIO=y -CONFIG_NET=y -CONFIG_PACKET=y -CONFIG_UNIX=y -CONFIG_XFRM_USER=y -CONFIG_NET_KEY=y -CONFIG_INET=y -CONFIG_IP_MULTICAST=y -CONFIG_IP_ADVANCED_ROUTER=y -CONFIG_IP_MULTIPLE_TABLES=y -CONFIG_IP_ROUTE_MULTIPATH=y -CONFIG_IP_ROUTE_VERBOSE=y -CONFIG_IP_PNP=y -CONFIG_IP_PNP_DHCP=y -CONFIG_IP_PNP_BOOTP=y -CONFIG_IP_PNP_RARP=y -CONFIG_NET_IPIP=y -CONFIG_IP_MROUTE=y -CONFIG_IP_PIMSM_V1=y -CONFIG_IP_PIMSM_V2=y -CONFIG_INET_ESP=y -# CONFIG_INET_XFRM_MODE_BEET is not set -# CONFIG_INET_LRO is not set -CONFIG_IPV6=y -CONFIG_IP_SCTP=m -CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" -CONFIG_DEVTMPFS=y -CONFIG_DEVTMPFS_MOUNT=y -CONFIG_MTD=y -CONFIG_MTD_CMDLINE_PARTS=y -CONFIG_MTD_BLOCK=y -CONFIG_FTL=y -CONFIG_MTD_CFI=y -CONFIG_MTD_CFI_INTELEXT=y -CONFIG_MTD_CFI_AMDSTD=y -CONFIG_MTD_PHYSMAP=y -CONFIG_MTD_PHYSMAP_OF=y -CONFIG_MTD_PLATRAM=y -CONFIG_MTD_M25P80=y -CONFIG_MTD_NAND=y -CONFIG_MTD_NAND_FSL_ELBC=y -CONFIG_MTD_NAND_FSL_IFC=y -CONFIG_MTD_SPI_NOR=y -CONFIG_MTD_UBI=y -CONFIG_BLK_DEV_LOOP=y -CONFIG_BLK_DEV_NBD=y -CONFIG_BLK_DEV_RAM=y -CONFIG_BLK_DEV_RAM_SIZE=131072 -CONFIG_EEPROM_AT24=y -CONFIG_EEPROM_LEGACY=y -CONFIG_BLK_DEV_SD=y -CONFIG_CHR_DEV_ST=y -CONFIG_BLK_DEV_SR=y -CONFIG_CHR_DEV_SG=y -CONFIG_SCSI_LOGGING=y -CONFIG_ATA=y -CONFIG_SATA_AHCI=y -CONFIG_SATA_FSL=y -CONFIG_SATA_SIL24=y -CONFIG_PATA_ALI=y -CONFIG_PATA_VIA=y -CONFIG_NETDEVICES=y -CONFIG_DUMMY=y -CONFIG_FS_ENET=y -CONFIG_UCC_GETH=y -CONFIG_GIANFAR=y -CONFIG_E1000=y -CONFIG_E1000E=y -CONFIG_IGB=y -CONFIG_AT803X_PHY=y -CONFIG_MARVELL_PHY=y -CONFIG_DAVICOM_PHY=y -CONFIG_CICADA_PHY=y -CONFIG_VITESSE_PHY=y -CONFIG_BROADCOM_PHY=y -CONFIG_FIXED_PHY=y -CONFIG_INPUT_FF_MEMLESS=m -# CONFIG_INPUT_MOUSEDEV is not set -# CONFIG_INPUT_KEYBOARD is not set -# CONFIG_INPUT_MOUSE is not set -CONFIG_SERIO_LIBPS2=y -CONFIG_SERIAL_8250=y -CONFIG_SERIAL_8250_CONSOLE=y -CONFIG_SERIAL_8250_NR_UARTS=6 -CONFIG_SERIAL_8250_RUNTIME_UARTS=6 -CONFIG_SERIAL_8250_MANY_PORTS=y -CONFIG_SERIAL_8250_DETECT_IRQ=y -CONFIG_SERIAL_8250_RSA=y -CONFIG_SERIAL_QE=m -CONFIG_NVRAM=y -CONFIG_I2C_CHARDEV=y -CONFIG_I2C_CPM=m -CONFIG_I2C_MPC=y -CONFIG_SPI=y -CONFIG_SPI_FSL_SPI=y -CONFIG_SPI_FSL_ESPI=y -CONFIG_GPIO_MPC8XXX=y -CONFIG_SENSORS_LM90=y -CONFIG_FB=y -CONFIG_FB_FSL_DIU=y -# CONFIG_VGA_CONSOLE is not set -CONFIG_FRAMEBUFFER_CONSOLE=y -CONFIG_SOUND=y -CONFIG_SND=y -# CONFIG_SND_SUPPORT_OLD_API is not set -# CONFIG_SND_DRIVERS is not set -CONFIG_SND_INTEL8X0=y -# CONFIG_SND_PPC is not set -# CONFIG_SND_USB is not set -CONFIG_SND_SOC=y -CONFIG_SND_POWERPC_SOC=y -CONFIG_HID_A4TECH=y -CONFIG_HID_APPLE=y -CONFIG_HID_BELKIN=y -CONFIG_HID_CHERRY=y -CONFIG_HID_CHICONY=y -CONFIG_HID_CYPRESS=y -CONFIG_HID_EZKEY=y -CONFIG_HID_GYRATION=y -CONFIG_HID_LOGITECH=y -CONFIG_HID_MICROSOFT=y -CONFIG_HID_MONTEREY=y -CONFIG_HID_PANTHERLORD=y -CONFIG_HID_PETALYNX=y -CONFIG_HID_SAMSUNG=y -CONFIG_HID_SUNPLUS=y -CONFIG_USB=y -CONFIG_USB_MON=y -CONFIG_USB_EHCI_HCD=y -CONFIG_USB_EHCI_FSL=y -CONFIG_USB_OHCI_HCD=y -CONFIG_USB_OHCI_HCD_PPC_OF_BE=y -CONFIG_USB_OHCI_HCD_PPC_OF_LE=y -CONFIG_USB_STORAGE=y -CONFIG_MMC=y -CONFIG_MMC_SDHCI=y -CONFIG_MMC_SDHCI_PLTFM=y -CONFIG_MMC_SDHCI_OF_ESDHC=y -CONFIG_EDAC=y -CONFIG_EDAC_MM_EDAC=y -CONFIG_EDAC_MPC85XX=y -CONFIG_RTC_CLASS=y -CONFIG_RTC_DRV_DS1307=y -CONFIG_RTC_DRV_DS1374=y -CONFIG_RTC_DRV_DS3232=y -CONFIG_RTC_DRV_CMOS=y -CONFIG_DMADEVICES=y -CONFIG_FSL_DMA=y -CONFIG_EXT2_FS=y -CONFIG_EXT3_FS=y -# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set -CONFIG_ISO9660_FS=m -CONFIG_JOLIET=y -CONFIG_ZISOFS=y -CONFIG_UDF_FS=m -CONFIG_MSDOS_FS=m -CONFIG_VFAT_FS=y -CONFIG_NTFS_FS=y -CONFIG_PROC_KCORE=y -CONFIG_TMPFS=y -CONFIG_HUGETLBFS=y -CONFIG_ADFS_FS=m -CONFIG_AFFS_FS=m -CONFIG_HFS_FS=m -CONFIG_HFSPLUS_FS=m -CONFIG_BEFS_FS=m -CONFIG_BFS_FS=m -CONFIG_EFS_FS=m -CONFIG_JFFS2_FS=y -CONFIG_JFFS2_FS_DEBUG=1 -CONFIG_UBIFS_FS=y -CONFIG_CRAMFS=y -CONFIG_VXFS_FS=m -CONFIG_HPFS_FS=m -CONFIG_QNX4FS_FS=m -CONFIG_SYSV_FS=m -CONFIG_UFS_FS=m -CONFIG_NFS_FS=y -CONFIG_NFS_V4=y -CONFIG_ROOT_NFS=y -CONFIG_NFSD=y -CONFIG_NLS_CODEPAGE_437=y -CONFIG_NLS_CODEPAGE_850=y -CONFIG_NLS_ISO8859_1=y -CONFIG_CRC_T10DIF=y -CONFIG_FONTS=y -CONFIG_FONT_8x8=y -CONFIG_FONT_8x16=y -CONFIG_DEBUG_INFO=y -CONFIG_DEBUG_FS=y -CONFIG_DETECT_HUNG_TASK=y -CONFIG_CRYPTO_PCBC=m -CONFIG_CRYPTO_SHA256=y -CONFIG_CRYPTO_SHA512=y -# CONFIG_CRYPTO_ANSI_CPRNG is not set -CONFIG_CRYPTO_DEV_FSL_CAAM=y -CONFIG_CRYPTO_DEV_TALITOS=y diff --git a/arch/powerpc/configs/mpc85xx_smp_defconfig b/arch/powerpc/configs/mpc85xx_smp_defconfig deleted file mode 100644 index b6c7111ea9138c..00000000000000 --- a/arch/powerpc/configs/mpc85xx_smp_defconfig +++ /dev/null @@ -1,244 +0,0 @@ -CONFIG_PPC_85xx=y -CONFIG_PHYS_64BIT=y -CONFIG_SMP=y -CONFIG_NR_CPUS=8 -CONFIG_SYSVIPC=y -CONFIG_POSIX_MQUEUE=y -CONFIG_AUDIT=y -CONFIG_IRQ_DOMAIN_DEBUG=y -CONFIG_NO_HZ=y -CONFIG_HIGH_RES_TIMERS=y -CONFIG_BSD_PROCESS_ACCT=y -CONFIG_IKCONFIG=y -CONFIG_IKCONFIG_PROC=y -CONFIG_LOG_BUF_SHIFT=14 -CONFIG_BLK_DEV_INITRD=y -CONFIG_EXPERT=y -CONFIG_KALLSYMS_ALL=y -CONFIG_MODULES=y -CONFIG_MODULE_UNLOAD=y -CONFIG_MODULE_FORCE_UNLOAD=y -CONFIG_MODVERSIONS=y -# CONFIG_BLK_DEV_BSG is not set -CONFIG_PARTITION_ADVANCED=y -CONFIG_MAC_PARTITION=y -CONFIG_C293_PCIE=y -CONFIG_MPC8540_ADS=y -CONFIG_MPC8560_ADS=y -CONFIG_MPC85xx_CDS=y -CONFIG_MPC85xx_MDS=y -CONFIG_MPC8536_DS=y -CONFIG_MPC85xx_DS=y -CONFIG_MPC85xx_RDB=y -CONFIG_P1010_RDB=y -CONFIG_P1022_DS=y -CONFIG_P1022_RDK=y -CONFIG_P1023_RDB=y -CONFIG_SOCRATES=y -CONFIG_KSI8560=y -CONFIG_XES_MPC85xx=y -CONFIG_STX_GP3=y -CONFIG_TQM8540=y -CONFIG_TQM8541=y -CONFIG_TQM8548=y -CONFIG_TQM8555=y -CONFIG_TQM8560=y -CONFIG_SBC8548=y -CONFIG_QUICC_ENGINE=y -CONFIG_QE_GPIO=y -CONFIG_HIGHMEM=y -CONFIG_BINFMT_MISC=m -CONFIG_MATH_EMULATION=y -CONFIG_FORCE_MAX_ZONEORDER=12 -CONFIG_PCI=y -CONFIG_PCI_MSI=y -CONFIG_RAPIDIO=y -CONFIG_NET=y -CONFIG_PACKET=y -CONFIG_UNIX=y -CONFIG_XFRM_USER=y -CONFIG_NET_KEY=y -CONFIG_INET=y -CONFIG_IP_MULTICAST=y -CONFIG_IP_ADVANCED_ROUTER=y -CONFIG_IP_MULTIPLE_TABLES=y -CONFIG_IP_ROUTE_MULTIPATH=y -CONFIG_IP_ROUTE_VERBOSE=y -CONFIG_IP_PNP=y -CONFIG_IP_PNP_DHCP=y -CONFIG_IP_PNP_BOOTP=y -CONFIG_IP_PNP_RARP=y -CONFIG_NET_IPIP=y -CONFIG_IP_MROUTE=y -CONFIG_IP_PIMSM_V1=y -CONFIG_IP_PIMSM_V2=y -CONFIG_INET_ESP=y -# CONFIG_INET_XFRM_MODE_BEET is not set -# CONFIG_INET_LRO is not set -CONFIG_IPV6=y -CONFIG_IP_SCTP=m -CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug" -CONFIG_DEVTMPFS=y -CONFIG_DEVTMPFS_MOUNT=y -CONFIG_MTD=y -CONFIG_MTD_CMDLINE_PARTS=y -CONFIG_MTD_BLOCK=y -CONFIG_FTL=y -CONFIG_MTD_CFI=y -CONFIG_MTD_CFI_INTELEXT=y -CONFIG_MTD_CFI_AMDSTD=y -CONFIG_MTD_PHYSMAP_OF=y -CONFIG_MTD_NAND=y -CONFIG_MTD_NAND_FSL_ELBC=y -CONFIG_MTD_NAND_FSL_IFC=y -CONFIG_MTD_SPI_NOR=y -CONFIG_MTD_UBI=y -CONFIG_BLK_DEV_LOOP=y -CONFIG_BLK_DEV_NBD=y -CONFIG_BLK_DEV_RAM=y -CONFIG_BLK_DEV_RAM_SIZE=131072 -CONFIG_EEPROM_AT24=y -CONFIG_EEPROM_LEGACY=y -CONFIG_BLK_DEV_SD=y -CONFIG_CHR_DEV_ST=y -CONFIG_BLK_DEV_SR=y -CONFIG_CHR_DEV_SG=y -CONFIG_SCSI_LOGGING=y -CONFIG_ATA=y -CONFIG_SATA_AHCI=y -CONFIG_SATA_FSL=y -CONFIG_SATA_SIL24=y -CONFIG_PATA_ALI=y -CONFIG_NETDEVICES=y -CONFIG_DUMMY=y -CONFIG_FS_ENET=y -CONFIG_UCC_GETH=y -CONFIG_GIANFAR=y -CONFIG_E1000E=y -CONFIG_AT803X_PHY=y -CONFIG_MARVELL_PHY=y -CONFIG_DAVICOM_PHY=y -CONFIG_CICADA_PHY=y -CONFIG_VITESSE_PHY=y -CONFIG_FIXED_PHY=y -CONFIG_INPUT_FF_MEMLESS=m -# CONFIG_INPUT_MOUSEDEV is not set -# CONFIG_INPUT_KEYBOARD is not set -# CONFIG_INPUT_MOUSE is not set -CONFIG_SERIO_LIBPS2=y -CONFIG_SERIAL_8250=y -CONFIG_SERIAL_8250_CONSOLE=y -CONFIG_SERIAL_8250_NR_UARTS=2 -CONFIG_SERIAL_8250_RUNTIME_UARTS=2 -CONFIG_SERIAL_8250_MANY_PORTS=y -CONFIG_SERIAL_8250_DETECT_IRQ=y -CONFIG_SERIAL_8250_RSA=y -CONFIG_SERIAL_QE=m -CONFIG_NVRAM=y -CONFIG_I2C=y -CONFIG_I2C_CHARDEV=y -CONFIG_I2C_CPM=m -CONFIG_I2C_MPC=y -CONFIG_SPI=y -CONFIG_SPI_FSL_SPI=y -CONFIG_SPI_FSL_ESPI=y -CONFIG_GPIO_MPC8XXX=y -CONFIG_SENSORS_LM90=y -CONFIG_FB=y -CONFIG_FB_FSL_DIU=y -# CONFIG_VGA_CONSOLE is not set -CONFIG_FRAMEBUFFER_CONSOLE=y -CONFIG_SOUND=y -CONFIG_SND=y -# CONFIG_SND_SUPPORT_OLD_API is not set -# CONFIG_SND_DRIVERS is not set -CONFIG_SND_INTEL8X0=y -# CONFIG_SND_PPC is not set -# CONFIG_SND_USB is not set -CONFIG_SND_SOC=y -CONFIG_SND_POWERPC_SOC=y -CONFIG_HID_A4TECH=y -CONFIG_HID_APPLE=y -CONFIG_HID_BELKIN=y -CONFIG_HID_CHERRY=y -CONFIG_HID_CHICONY=y -CONFIG_HID_CYPRESS=y -CONFIG_HID_EZKEY=y -CONFIG_HID_GYRATION=y -CONFIG_HID_LOGITECH=y -CONFIG_HID_MICROSOFT=y -CONFIG_HID_MONTEREY=y -CONFIG_HID_PANTHERLORD=y -CONFIG_HID_PETALYNX=y -CONFIG_HID_SAMSUNG=y -CONFIG_HID_SUNPLUS=y -CONFIG_USB=y -CONFIG_USB_MON=y -CONFIG_USB_EHCI_HCD=y -CONFIG_USB_EHCI_FSL=y -CONFIG_USB_OHCI_HCD=y -CONFIG_USB_OHCI_HCD_PPC_OF_BE=y -CONFIG_USB_OHCI_HCD_PPC_OF_LE=y -CONFIG_USB_STORAGE=y -CONFIG_MMC=y -CONFIG_MMC_SDHCI=y -CONFIG_MMC_SDHCI_PLTFM=y -CONFIG_MMC_SDHCI_OF_ESDHC=y -CONFIG_EDAC=y -CONFIG_EDAC_MM_EDAC=y -CONFIG_RTC_CLASS=y -CONFIG_RTC_DRV_DS1307=y -CONFIG_RTC_DRV_DS1374=y -CONFIG_RTC_DRV_DS3232=y -CONFIG_RTC_DRV_CMOS=y -CONFIG_DMADEVICES=y -CONFIG_FSL_DMA=y -CONFIG_EXT2_FS=y -CONFIG_EXT3_FS=y -# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set -CONFIG_ISO9660_FS=m -CONFIG_JOLIET=y -CONFIG_ZISOFS=y -CONFIG_UDF_FS=m -CONFIG_MSDOS_FS=m -CONFIG_VFAT_FS=y -CONFIG_NTFS_FS=y -CONFIG_PROC_KCORE=y -CONFIG_TMPFS=y -CONFIG_HUGETLBFS=y -CONFIG_ADFS_FS=m -CONFIG_AFFS_FS=m -CONFIG_HFS_FS=m -CONFIG_HFSPLUS_FS=m -CONFIG_BEFS_FS=m -CONFIG_BFS_FS=m -CONFIG_EFS_FS=m -CONFIG_JFFS2_FS=y -CONFIG_JFFS2_FS_DEBUG=1 -CONFIG_UBIFS_FS=y -CONFIG_CRAMFS=y -CONFIG_VXFS_FS=m -CONFIG_HPFS_FS=m -CONFIG_QNX4FS_FS=m -CONFIG_SYSV_FS=m -CONFIG_UFS_FS=m -CONFIG_NFS_FS=y -CONFIG_NFS_V4=y -CONFIG_ROOT_NFS=y -CONFIG_NFSD=y -CONFIG_NLS_CODEPAGE_437=y -CONFIG_NLS_CODEPAGE_850=y -CONFIG_NLS_ISO8859_1=y -CONFIG_CRC_T10DIF=y -CONFIG_FONTS=y -CONFIG_FONT_8x8=y -CONFIG_FONT_8x16=y -CONFIG_DEBUG_INFO=y -CONFIG_DEBUG_FS=y -CONFIG_DETECT_HUNG_TASK=y -CONFIG_CRYPTO_PCBC=m -CONFIG_CRYPTO_SHA256=y -CONFIG_CRYPTO_SHA512=y -# CONFIG_CRYPTO_ANSI_CPRNG is not set -CONFIG_CRYPTO_DEV_FSL_CAAM=y -CONFIG_CRYPTO_DEV_TALITOS=y diff --git a/arch/powerpc/configs/ppc64_defconfig b/arch/powerpc/configs/ppc64_defconfig index a97efc2146fdfe..6bc0ee4b1070a8 100644 --- a/arch/powerpc/configs/ppc64_defconfig +++ b/arch/powerpc/configs/ppc64_defconfig @@ -355,3 +355,6 @@ CONFIG_CRYPTO_DEV_NX_ENCRYPT=m CONFIG_VIRTUALIZATION=y CONFIG_KVM_BOOK3S_64=m CONFIG_KVM_BOOK3S_64_HV=m +CONFIG_NEW_LEDS=y +CONFIG_LEDS_CLASS=m +CONFIG_LEDS_POWERNV=m diff --git a/arch/powerpc/configs/pseries_defconfig b/arch/powerpc/configs/pseries_defconfig index 0d9efcedaf3473..7991f37e5fe2a1 100644 --- a/arch/powerpc/configs/pseries_defconfig +++ b/arch/powerpc/configs/pseries_defconfig @@ -190,7 +190,8 @@ CONFIG_HVC_RTAS=y CONFIG_HVCS=m CONFIG_VIRTIO_CONSOLE=m CONFIG_IBM_BSR=m -CONFIG_GEN_RTC=y +CONFIG_RTC_CLASS=y +CONFIG_RTC_DRV_GENERIC=y CONFIG_RAW_DRIVER=y CONFIG_MAX_RAW_DEVS=1024 CONFIG_FB=y @@ -319,3 +320,6 @@ CONFIG_CRYPTO_DEV_NX_ENCRYPT=m CONFIG_VIRTUALIZATION=y CONFIG_KVM_BOOK3S_64=m CONFIG_KVM_BOOK3S_64_HV=m +CONFIG_NEW_LEDS=y +CONFIG_LEDS_CLASS=m +CONFIG_LEDS_POWERNV=m diff --git a/arch/powerpc/include/asm/Kbuild b/arch/powerpc/include/asm/Kbuild index 050712e1ce4126..ab9f4e0ed4cfcf 100644 --- a/arch/powerpc/include/asm/Kbuild +++ b/arch/powerpc/include/asm/Kbuild @@ -6,5 +6,4 @@ generic-y += local64.h generic-y += mcs_spinlock.h generic-y += preempt.h generic-y += rwsem.h -generic-y += trace_clock.h generic-y += vtime.h diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h index 0cc6eedc478032..85e88f7a59c0a0 100644 --- a/arch/powerpc/include/asm/archrandom.h +++ b/arch/powerpc/include/asm/archrandom.h @@ -7,13 +7,22 @@ static inline int arch_get_random_long(unsigned long *v) { - if (ppc_md.get_random_long) - return ppc_md.get_random_long(v); - return 0; } static inline int arch_get_random_int(unsigned int *v) +{ + return 0; +} + +static inline int arch_get_random_seed_long(unsigned long *v) +{ + if (ppc_md.get_random_seed) + return ppc_md.get_random_seed(v); + + return 0; +} +static inline int arch_get_random_seed_int(unsigned int *v) { unsigned long val; int rc; @@ -26,23 +35,14 @@ static inline int arch_get_random_int(unsigned int *v) } static inline int arch_has_random(void) -{ - return !!ppc_md.get_random_long; -} - -static inline int arch_get_random_seed_long(unsigned long *v) -{ - return 0; -} -static inline int arch_get_random_seed_int(unsigned int *v) { return 0; } + static inline int arch_has_random_seed(void) { - return 0; + return !!ppc_md.get_random_seed; } - #endif /* CONFIG_ARCH_RANDOM */ #ifdef CONFIG_PPC_POWERNV diff --git a/arch/powerpc/include/asm/atomic.h b/arch/powerpc/include/asm/atomic.h index 512d2782b043dd..55f106ed12bf2e 100644 --- a/arch/powerpc/include/asm/atomic.h +++ b/arch/powerpc/include/asm/atomic.h @@ -67,6 +67,10 @@ static __inline__ int atomic_##op##_return(int a, atomic_t *v) \ ATOMIC_OPS(add, add) ATOMIC_OPS(sub, subf) +ATOMIC_OP(and, and) +ATOMIC_OP(or, or) +ATOMIC_OP(xor, xor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP @@ -304,6 +308,9 @@ static __inline__ long atomic64_##op##_return(long a, atomic64_t *v) \ ATOMIC64_OPS(add, add) ATOMIC64_OPS(sub, subf) +ATOMIC64_OP(and, and) +ATOMIC64_OP(or, or) +ATOMIC64_OP(xor, xor) #undef ATOMIC64_OPS #undef ATOMIC64_OP_RETURN diff --git a/arch/powerpc/include/asm/barrier.h b/arch/powerpc/include/asm/barrier.h index 51ccc7232042e9..0eca6efc0631d5 100644 --- a/arch/powerpc/include/asm/barrier.h +++ b/arch/powerpc/include/asm/barrier.h @@ -76,12 +76,12 @@ do { \ compiletime_assert_atomic_type(*p); \ smp_lwsync(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ smp_lwsync(); \ ___p1; \ diff --git a/arch/powerpc/include/asm/cacheflush.h b/arch/powerpc/include/asm/cacheflush.h index 30b35fff2deaba..6229e6b6037b97 100644 --- a/arch/powerpc/include/asm/cacheflush.h +++ b/arch/powerpc/include/asm/cacheflush.h @@ -40,7 +40,12 @@ extern void __flush_dcache_icache(void *page_va); extern void flush_dcache_icache_page(struct page *page); #if defined(CONFIG_PPC32) && !defined(CONFIG_BOOKE) extern void __flush_dcache_icache_phys(unsigned long physaddr); -#endif /* CONFIG_PPC32 && !CONFIG_BOOKE */ +#else +static inline void __flush_dcache_icache_phys(unsigned long physaddr) +{ + BUG(); +} +#endif extern void flush_dcache_range(unsigned long start, unsigned long stop); #ifdef CONFIG_PPC32 diff --git a/arch/powerpc/include/asm/checksum.h b/arch/powerpc/include/asm/checksum.h index 8251a3ba870f8c..e8d9ef4755a499 100644 --- a/arch/powerpc/include/asm/checksum.h +++ b/arch/powerpc/include/asm/checksum.h @@ -19,15 +19,6 @@ #else extern __sum16 ip_fast_csum(const void *iph, unsigned int ihl); -/* - * computes the checksum of the TCP/UDP pseudo-header - * returns a 16-bit checksum, already complemented - */ -extern __sum16 csum_tcpudp_magic(__be32 saddr, __be32 daddr, - unsigned short len, - unsigned short proto, - __wsum sum); - /* * computes the checksum of a memory block at buff, length len, * and adds in "sum" (32-bit) @@ -127,6 +118,34 @@ static inline __wsum csum_tcpudp_nofold(__be32 saddr, __be32 daddr, #endif } +/* + * computes the checksum of the TCP/UDP pseudo-header + * returns a 16-bit checksum, already complemented + */ +static inline __sum16 csum_tcpudp_magic(__be32 saddr, __be32 daddr, + unsigned short len, + unsigned short proto, + __wsum sum) +{ + return csum_fold(csum_tcpudp_nofold(saddr, daddr, len, proto, sum)); +} + +#define HAVE_ARCH_CSUM_ADD +static inline __wsum csum_add(__wsum csum, __wsum addend) +{ +#ifdef __powerpc64__ + u64 res = (__force u64)csum; + + res += (__force u64)addend; + return (__force __wsum)((u32)res + (res >> 32)); +#else + asm("addc %0,%0,%1;" + "addze %0,%0;" + : "+r" (csum) : "r" (addend)); + return csum; +#endif +} + #endif #endif /* __KERNEL__ */ #endif diff --git a/arch/powerpc/include/asm/compat.h b/arch/powerpc/include/asm/compat.h index b142b8e0ed9e26..4f2df589ec1de2 100644 --- a/arch/powerpc/include/asm/compat.h +++ b/arch/powerpc/include/asm/compat.h @@ -174,6 +174,13 @@ typedef struct compat_siginfo { int _band; /* POLL_IN, POLL_OUT, POLL_MSG */ int _fd; } _sigpoll; + + /* SIGSYS */ + struct { + unsigned int _call_addr; /* calling insn */ + int _syscall; /* triggering system call number */ + unsigned int _arch; /* AUDIT_ARCH_* of syscall */ + } _sigsys; } _sifields; } compat_siginfo_t; diff --git a/arch/powerpc/include/asm/device.h b/arch/powerpc/include/asm/device.h index e9bdda88f1fbb1..406c2b1ff82d67 100644 --- a/arch/powerpc/include/asm/device.h +++ b/arch/powerpc/include/asm/device.h @@ -10,6 +10,7 @@ struct dma_map_ops; struct device_node; #ifdef CONFIG_PPC64 struct pci_dn; +struct iommu_table; #endif /* @@ -23,13 +24,15 @@ struct dev_archdata { struct dma_map_ops *dma_ops; /* - * When an iommu is in use, dma_data is used as a ptr to the base of the - * iommu_table. Otherwise, it is a simple numerical offset. + * These two used to be a union. However, with the hybrid ops we need + * both so here we store both a DMA offset for direct mappings and + * an iommu_table for remapped DMA. */ - union { - dma_addr_t dma_offset; - void *iommu_table_base; - } dma_data; + dma_addr_t dma_offset; + +#ifdef CONFIG_PPC64 + struct iommu_table *iommu_table_base; +#endif #ifdef CONFIG_IOMMU_API void *iommu_domain; diff --git a/arch/powerpc/include/asm/dma-mapping.h b/arch/powerpc/include/asm/dma-mapping.h index 9103687b043636..710f60e380e07d 100644 --- a/arch/powerpc/include/asm/dma-mapping.h +++ b/arch/powerpc/include/asm/dma-mapping.h @@ -21,12 +21,12 @@ #define DMA_ERROR_CODE (~(dma_addr_t)0x0) /* Some dma direct funcs must be visible for use in other dma_ops */ -extern void *dma_direct_alloc_coherent(struct device *dev, size_t size, - dma_addr_t *dma_handle, gfp_t flag, +extern void *__dma_direct_alloc_coherent(struct device *dev, size_t size, + dma_addr_t *dma_handle, gfp_t flag, + struct dma_attrs *attrs); +extern void __dma_direct_free_coherent(struct device *dev, size_t size, + void *vaddr, dma_addr_t dma_handle, struct dma_attrs *attrs); -extern void dma_direct_free_coherent(struct device *dev, size_t size, - void *vaddr, dma_addr_t dma_handle, - struct dma_attrs *attrs); extern int dma_direct_mmap_coherent(struct device *dev, struct vm_area_struct *vma, void *cpu_addr, dma_addr_t handle, @@ -106,7 +106,7 @@ static inline void set_dma_ops(struct device *dev, struct dma_map_ops *ops) static inline dma_addr_t get_dma_offset(struct device *dev) { if (dev) - return dev->archdata.dma_data.dma_offset; + return dev->archdata.dma_offset; return PCI_DRAM_OFFSET; } @@ -114,7 +114,7 @@ static inline dma_addr_t get_dma_offset(struct device *dev) static inline void set_dma_offset(struct device *dev, dma_addr_t off) { if (dev) - dev->archdata.dma_data.dma_offset = off; + dev->archdata.dma_offset = off; } /* this will be removed soon */ diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h index ca18cff909006b..7b87bab095641e 100644 --- a/arch/powerpc/include/asm/iommu.h +++ b/arch/powerpc/include/asm/iommu.h @@ -2,17 +2,17 @@ * Copyright (C) 2001 Mike Corrigan & Dave Engebretsen, IBM Corporation * Rewrite, cleanup: * Copyright (C) 2004 Olof Johansson , IBM Corporation - * + * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. - * + * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. - * + * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA @@ -131,16 +131,21 @@ int get_iommu_order(unsigned long size, struct iommu_table *tbl) struct scatterlist; -static inline void set_iommu_table_base(struct device *dev, void *base) +#ifdef CONFIG_PPC64 + +static inline void set_iommu_table_base(struct device *dev, + struct iommu_table *base) { - dev->archdata.dma_data.iommu_table_base = base; + dev->archdata.iommu_table_base = base; } static inline void *get_iommu_table_base(struct device *dev) { - return dev->archdata.dma_data.iommu_table_base; + return dev->archdata.iommu_table_base; } +extern int dma_iommu_dma_supported(struct device *dev, u64 mask); + /* Frees table for an individual device node */ extern void iommu_free_table(struct iommu_table *tbl, const char *node_name); @@ -225,6 +230,20 @@ static inline int __init tce_iommu_bus_notifier_init(void) } #endif /* !CONFIG_IOMMU_API */ +#else + +static inline void *get_iommu_table_base(struct device *dev) +{ + return NULL; +} + +static inline int dma_iommu_dma_supported(struct device *dev, u64 mask) +{ + return 0; +} + +#endif /* CONFIG_PPC64 */ + extern int ppc_iommu_map_sg(struct device *dev, struct iommu_table *tbl, struct scatterlist *sglist, int nelems, unsigned long mask, diff --git a/arch/powerpc/include/asm/jump_label.h b/arch/powerpc/include/asm/jump_label.h index efbf9a322a2368..47e155f15433c6 100644 --- a/arch/powerpc/include/asm/jump_label.h +++ b/arch/powerpc/include/asm/jump_label.h @@ -18,14 +18,29 @@ #define JUMP_ENTRY_TYPE stringify_in_c(FTR_ENTRY_LONG) #define JUMP_LABEL_NOP_SIZE 4 -static __always_inline bool arch_static_branch(struct static_key *key) +static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { asm_volatile_goto("1:\n\t" "nop\n\t" ".pushsection __jump_table, \"aw\"\n\t" JUMP_ENTRY_TYPE "1b, %l[l_yes], %c0\n\t" ".popsection \n\t" - : : "i" (key) : : l_yes); + : : "i" (&((char *)key)[branch]) : : l_yes); + + return false; +l_yes: + return true; +} + +static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch) +{ + asm_volatile_goto("1:\n\t" + "b %l[l_yes]\n\t" + ".pushsection __jump_table, \"aw\"\n\t" + JUMP_ENTRY_TYPE "1b, %l[l_yes], %c0\n\t" + ".popsection \n\t" + : : "i" (&((char *)key)[branch]) : : l_yes); + return false; l_yes: return true; diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h index 952579f5e79a93..cab6753f1be56e 100644 --- a/arch/powerpc/include/asm/machdep.h +++ b/arch/powerpc/include/asm/machdep.h @@ -249,7 +249,7 @@ struct machdep_calls { #endif #ifdef CONFIG_ARCH_RANDOM - int (*get_random_long)(unsigned long *v); + int (*get_random_seed)(unsigned long *v); #endif }; diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h index 64dc9f547fb6bf..8374afed9d0a95 100644 --- a/arch/powerpc/include/asm/opal-api.h +++ b/arch/powerpc/include/asm/opal-api.h @@ -154,7 +154,10 @@ #define OPAL_FLASH_WRITE 111 #define OPAL_FLASH_ERASE 112 #define OPAL_PRD_MSG 113 -#define OPAL_LAST 113 +#define OPAL_LEDS_GET_INDICATOR 114 +#define OPAL_LEDS_SET_INDICATOR 115 +#define OPAL_CEC_REBOOT2 116 +#define OPAL_LAST 116 /* Device tree flags */ @@ -340,6 +343,18 @@ enum OpalPciResetState { OPAL_ASSERT_RESET = 1 }; +enum OpalSlotLedType { + OPAL_SLOT_LED_TYPE_ID = 0, /* IDENTIFY LED */ + OPAL_SLOT_LED_TYPE_FAULT = 1, /* FAULT LED */ + OPAL_SLOT_LED_TYPE_ATTN = 2, /* System Attention LED */ + OPAL_SLOT_LED_TYPE_MAX = 3 +}; + +enum OpalSlotLedState { + OPAL_SLOT_LED_STATE_OFF = 0, /* LED is OFF */ + OPAL_SLOT_LED_STATE_ON = 1 /* LED is ON */ +}; + /* * Address cycle types for LPC accesses. These also correspond * to the content of the first cell of the "reg" property for @@ -438,6 +453,7 @@ struct OpalMemoryErrorData { /* HMI interrupt event */ enum OpalHMI_Version { OpalHMIEvt_V1 = 1, + OpalHMIEvt_V2 = 2, }; enum OpalHMI_Severity { @@ -468,6 +484,49 @@ enum OpalHMI_ErrType { OpalHMI_ERROR_CAPP_RECOVERY, }; +enum OpalHMI_XstopType { + CHECKSTOP_TYPE_UNKNOWN = 0, + CHECKSTOP_TYPE_CORE = 1, + CHECKSTOP_TYPE_NX = 2, +}; + +enum OpalHMI_CoreXstopReason { + CORE_CHECKSTOP_IFU_REGFILE = 0x00000001, + CORE_CHECKSTOP_IFU_LOGIC = 0x00000002, + CORE_CHECKSTOP_PC_DURING_RECOV = 0x00000004, + CORE_CHECKSTOP_ISU_REGFILE = 0x00000008, + CORE_CHECKSTOP_ISU_LOGIC = 0x00000010, + CORE_CHECKSTOP_FXU_LOGIC = 0x00000020, + CORE_CHECKSTOP_VSU_LOGIC = 0x00000040, + CORE_CHECKSTOP_PC_RECOV_IN_MAINT_MODE = 0x00000080, + CORE_CHECKSTOP_LSU_REGFILE = 0x00000100, + CORE_CHECKSTOP_PC_FWD_PROGRESS = 0x00000200, + CORE_CHECKSTOP_LSU_LOGIC = 0x00000400, + CORE_CHECKSTOP_PC_LOGIC = 0x00000800, + CORE_CHECKSTOP_PC_HYP_RESOURCE = 0x00001000, + CORE_CHECKSTOP_PC_HANG_RECOV_FAILED = 0x00002000, + CORE_CHECKSTOP_PC_AMBI_HANG_DETECTED = 0x00004000, + CORE_CHECKSTOP_PC_DEBUG_TRIG_ERR_INJ = 0x00008000, + CORE_CHECKSTOP_PC_SPRD_HYP_ERR_INJ = 0x00010000, +}; + +enum OpalHMI_NestAccelXstopReason { + NX_CHECKSTOP_SHM_INVAL_STATE_ERR = 0x00000001, + NX_CHECKSTOP_DMA_INVAL_STATE_ERR_1 = 0x00000002, + NX_CHECKSTOP_DMA_INVAL_STATE_ERR_2 = 0x00000004, + NX_CHECKSTOP_DMA_CH0_INVAL_STATE_ERR = 0x00000008, + NX_CHECKSTOP_DMA_CH1_INVAL_STATE_ERR = 0x00000010, + NX_CHECKSTOP_DMA_CH2_INVAL_STATE_ERR = 0x00000020, + NX_CHECKSTOP_DMA_CH3_INVAL_STATE_ERR = 0x00000040, + NX_CHECKSTOP_DMA_CH4_INVAL_STATE_ERR = 0x00000080, + NX_CHECKSTOP_DMA_CH5_INVAL_STATE_ERR = 0x00000100, + NX_CHECKSTOP_DMA_CH6_INVAL_STATE_ERR = 0x00000200, + NX_CHECKSTOP_DMA_CH7_INVAL_STATE_ERR = 0x00000400, + NX_CHECKSTOP_DMA_CRB_UE = 0x00000800, + NX_CHECKSTOP_DMA_CRB_SUE = 0x00001000, + NX_CHECKSTOP_PBI_ISN_UE = 0x00002000, +}; + struct OpalHMIEvent { uint8_t version; /* 0x00 */ uint8_t severity; /* 0x01 */ @@ -478,6 +537,23 @@ struct OpalHMIEvent { __be64 hmer; /* TFMR register. Valid only for TFAC and TFMR_PARITY error type. */ __be64 tfmr; + + /* version 2 and later */ + union { + /* + * checkstop info (Core/NX). + * Valid for OpalHMI_ERROR_MALFUNC_ALERT. + */ + struct { + uint8_t xstop_type; /* enum OpalHMI_XstopType */ + uint8_t reserved_1[3]; + __be32 xstop_reason; + union { + __be32 pir; /* for CHECKSTOP_TYPE_CORE */ + __be32 chip_id; /* for CHECKSTOP_TYPE_NX */ + } u; + } xstop_error; + } u; }; enum { @@ -768,6 +844,52 @@ struct opal_i2c_request { __be64 buffer_ra; /* Buffer real address */ }; +/* + * EPOW status sharing (OPAL and the host) + * + * The host will pass on OPAL, a buffer of length OPAL_SYSEPOW_MAX + * with individual elements being 16 bits wide to fetch the system + * wide EPOW status. Each element in the buffer will contain the + * EPOW status in it's bit representation for a particular EPOW sub + * class as defiend here. So multiple detailed EPOW status bits + * specific for any sub class can be represented in a single buffer + * element as it's bit representation. + */ + +/* System EPOW type */ +enum OpalSysEpow { + OPAL_SYSEPOW_POWER = 0, /* Power EPOW */ + OPAL_SYSEPOW_TEMP = 1, /* Temperature EPOW */ + OPAL_SYSEPOW_COOLING = 2, /* Cooling EPOW */ + OPAL_SYSEPOW_MAX = 3, /* Max EPOW categories */ +}; + +/* Power EPOW */ +enum OpalSysPower { + OPAL_SYSPOWER_UPS = 0x0001, /* System on UPS power */ + OPAL_SYSPOWER_CHNG = 0x0002, /* System power config change */ + OPAL_SYSPOWER_FAIL = 0x0004, /* System impending power failure */ + OPAL_SYSPOWER_INCL = 0x0008, /* System incomplete power */ +}; + +/* Temperature EPOW */ +enum OpalSysTemp { + OPAL_SYSTEMP_AMB = 0x0001, /* System over ambient temperature */ + OPAL_SYSTEMP_INT = 0x0002, /* System over internal temperature */ + OPAL_SYSTEMP_HMD = 0x0004, /* System over ambient humidity */ +}; + +/* Cooling EPOW */ +enum OpalSysCooling { + OPAL_SYSCOOL_INSF = 0x0001, /* System insufficient cooling */ +}; + +/* Argument to OPAL_CEC_REBOOT2() */ +enum { + OPAL_REBOOT_NORMAL = 0, + OPAL_REBOOT_PLATFORM_ERROR = 1, +}; + #endif /* __ASSEMBLY__ */ #endif /* __OPAL_API_H */ diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h index 958e941c0cda88..800115910e43a3 100644 --- a/arch/powerpc/include/asm/opal.h +++ b/arch/powerpc/include/asm/opal.h @@ -44,6 +44,7 @@ int64_t opal_tpo_write(uint64_t token, uint32_t year_mon_day, uint32_t hour_min); int64_t opal_cec_power_down(uint64_t request); int64_t opal_cec_reboot(void); +int64_t opal_cec_reboot2(uint32_t reboot_type, char *diag); int64_t opal_read_nvram(uint64_t buffer, uint64_t size, uint64_t offset); int64_t opal_write_nvram(uint64_t buffer, uint64_t size, uint64_t offset); int64_t opal_handle_interrupt(uint64_t isn, __be64 *outstanding_event_mask); @@ -141,7 +142,8 @@ int64_t opal_pci_fence_phb(uint64_t phb_id); int64_t opal_pci_reinit(uint64_t phb_id, uint64_t reinit_scope, uint64_t data); int64_t opal_pci_mask_pe_error(uint64_t phb_id, uint16_t pe_number, uint8_t error_type, uint8_t mask_action); int64_t opal_set_slot_led_status(uint64_t phb_id, uint64_t slot_id, uint8_t led_type, uint8_t led_action); -int64_t opal_get_epow_status(__be64 *status); +int64_t opal_get_epow_status(__be16 *epow_status, __be16 *num_epow_classes); +int64_t opal_get_dpo_status(__be64 *dpo_timeout); int64_t opal_set_system_attention_led(uint8_t led_action); int64_t opal_pci_next_error(uint64_t phb_id, __be64 *first_frozen_pe, __be16 *pci_error_type, __be16 *severity); @@ -195,6 +197,10 @@ int64_t opal_ipmi_recv(uint64_t interface, struct opal_ipmi_msg *msg, int64_t opal_i2c_request(uint64_t async_token, uint32_t bus_id, struct opal_i2c_request *oreq); int64_t opal_prd_msg(struct opal_prd_msg *msg); +int64_t opal_leds_get_ind(char *loc_code, __be64 *led_mask, + __be64 *led_value, __be64 *max_led_type); +int64_t opal_leds_set_ind(uint64_t token, char *loc_code, const u64 led_mask, + const u64 led_value, __be64 *max_led_type); int64_t opal_flash_read(uint64_t id, uint64_t offset, uint64_t buf, uint64_t size, uint64_t token); diff --git a/arch/powerpc/include/asm/pci-bridge.h b/arch/powerpc/include/asm/pci-bridge.h index 712add59044543..37fc53587bb45b 100644 --- a/arch/powerpc/include/asm/pci-bridge.h +++ b/arch/powerpc/include/asm/pci-bridge.h @@ -42,6 +42,7 @@ struct pci_controller_ops { #endif int (*dma_set_mask)(struct pci_dev *dev, u64 dma_mask); + u64 (*dma_get_required_mask)(struct pci_dev *dev); void (*shutdown)(struct pci_controller *); }; diff --git a/arch/powerpc/include/asm/pgtable-ppc64.h b/arch/powerpc/include/asm/pgtable-ppc64.h index 3bb7488bd24b19..fa1dfb7f7b48ed 100644 --- a/arch/powerpc/include/asm/pgtable-ppc64.h +++ b/arch/powerpc/include/asm/pgtable-ppc64.h @@ -134,11 +134,11 @@ #define pte_iterate_hashed_end() } while(0) -#ifdef CONFIG_PPC_HAS_HASH_64K -#define pte_pagesize_index(mm, addr, pte) get_slice_psize(mm, addr) -#else +/* + * We expect this to be called only for user addresses or kernel virtual + * addresses other than the linear mapping. + */ #define pte_pagesize_index(mm, addr, pte) MMU_PAGE_4K -#endif #endif /* __real_pte */ diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h index 11a38635dd658b..0717693c842897 100644 --- a/arch/powerpc/include/asm/pgtable.h +++ b/arch/powerpc/include/asm/pgtable.h @@ -169,6 +169,17 @@ static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr, * cases, and 32-bit non-hash with 32-bit PTEs. */ *ptep = pte; + +#ifdef CONFIG_PPC_BOOK3E_64 + /* + * With hardware tablewalk, a sync is needed to ensure that + * subsequent accesses see the PTE we just wrote. Unlike userspace + * mappings, we can't tolerate spurious faults, so make sure + * the new PTE will be seen the first time. + */ + if (is_kernel_addr(addr)) + mb(); +#endif #endif } diff --git a/arch/powerpc/include/asm/ppc-pci.h b/arch/powerpc/include/asm/ppc-pci.h index 4122a86d6858b1..ca0c5bff78490e 100644 --- a/arch/powerpc/include/asm/ppc-pci.h +++ b/arch/powerpc/include/asm/ppc-pci.h @@ -61,6 +61,7 @@ int rtas_write_config(struct pci_dn *, int where, int size, u32 val); int rtas_read_config(struct pci_dn *, int where, int size, u32 *val); void eeh_pe_state_mark(struct eeh_pe *pe, int state); void eeh_pe_state_clear(struct eeh_pe *pe, int state); +void eeh_pe_state_mark_with_cfg(struct eeh_pe *pe, int state); void eeh_pe_dev_mode_mark(struct eeh_pe *pe, int mode); void eeh_sysfs_add_device(struct pci_dev *pdev); diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h index 28ded5d9b57961..5afea361beaae9 100644 --- a/arch/powerpc/include/asm/processor.h +++ b/arch/powerpc/include/asm/processor.h @@ -264,7 +264,6 @@ struct thread_struct { u64 tm_tfhar; /* Transaction fail handler addr */ u64 tm_texasr; /* Transaction exception & summary */ u64 tm_tfiar; /* Transaction fail instr address reg */ - unsigned long tm_orig_msr; /* Thread's MSR on ctx switch */ struct pt_regs ckpt_regs; /* Checkpointed registers */ unsigned long tm_tar; diff --git a/arch/powerpc/include/asm/pte-common.h b/arch/powerpc/include/asm/pte-common.h index b7c8d079c121e8..71537a319fc828 100644 --- a/arch/powerpc/include/asm/pte-common.h +++ b/arch/powerpc/include/asm/pte-common.h @@ -109,7 +109,8 @@ extern unsigned long bad_call_to_PMD_PAGE_SIZE(void); * the processor might need it for DMA coherency. */ #define _PAGE_BASE_NC (_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_PSIZE) -#if defined(CONFIG_SMP) || defined(CONFIG_PPC_STD_MMU) +#if defined(CONFIG_SMP) || defined(CONFIG_PPC_STD_MMU) || \ + defined(CONFIG_PPC_E500MC) #define _PAGE_BASE (_PAGE_BASE_NC | _PAGE_COHERENT) #else #define _PAGE_BASE (_PAGE_BASE_NC) diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h index af56b5c6c81ab1..aa1cc5f015eeee 100644 --- a/arch/powerpc/include/asm/reg.h +++ b/arch/powerpc/include/asm/reg.h @@ -1193,8 +1193,7 @@ #ifdef CONFIG_PPC_BOOK3S_64 #define __mtmsrd(v, l) asm volatile("mtmsrd %0," __stringify(l) \ : : "r" (v) : "memory") -#define mtmsrd(v) __mtmsrd((v), 0) -#define mtmsr(v) mtmsrd(v) +#define mtmsr(v) __mtmsrd((v), 0) #else #define mtmsr(v) asm volatile("mtmsr %0" : \ : "r" ((unsigned long)(v)) \ @@ -1281,6 +1280,15 @@ struct pt_regs; extern void ppc_save_regs(struct pt_regs *regs); +static inline void update_power8_hid0(unsigned long hid0) +{ + /* + * The HID0 update on Power8 should at the very least be + * preceded by a a SYNC instruction followed by an ISYNC + * instruction + */ + asm volatile("sync; mtspr %0,%1; isync":: "i"(SPRN_HID0), "r"(hid0)); +} #endif /* __ASSEMBLY__ */ #endif /* __KERNEL__ */ #endif /* _ASM_POWERPC_REG_H */ diff --git a/arch/powerpc/include/asm/rtas.h b/arch/powerpc/include/asm/rtas.h index 7a4ede16b28364..b77ef369c0f0ea 100644 --- a/arch/powerpc/include/asm/rtas.h +++ b/arch/powerpc/include/asm/rtas.h @@ -343,6 +343,7 @@ extern void rtas_power_off(void); extern void rtas_halt(void); extern void rtas_os_term(char *str); extern int rtas_get_sensor(int sensor, int index, int *state); +extern int rtas_get_sensor_fast(int sensor, int index, int *state); extern int rtas_get_power_level(int powerdomain, int *level); extern int rtas_set_power_level(int powerdomain, int level, int *setlevel); extern bool rtas_indicator_present(int token, int *maxindex); diff --git a/arch/powerpc/include/asm/spu_csa.h b/arch/powerpc/include/asm/spu_csa.h index a40fd491250c2f..51f80b41cda338 100644 --- a/arch/powerpc/include/asm/spu_csa.h +++ b/arch/powerpc/include/asm/spu_csa.h @@ -241,12 +241,6 @@ struct spu_priv2_collapsed { */ struct spu_state { struct spu_lscsa *lscsa; -#ifdef CONFIG_SPU_FS_64K_LS - int use_big_pages; - /* One struct page per 64k page */ -#define SPU_LSCSA_NUM_BIG_PAGES (sizeof(struct spu_lscsa) / 0x10000) - struct page *lscsa_pages[SPU_LSCSA_NUM_BIG_PAGES]; -#endif struct spu_problem_collapsed prob; struct spu_priv1_collapsed priv1; struct spu_priv2_collapsed priv2; diff --git a/arch/powerpc/include/asm/syscall.h b/arch/powerpc/include/asm/syscall.h index ff21b7a2f0cc58..ab9f3f0a863741 100644 --- a/arch/powerpc/include/asm/syscall.h +++ b/arch/powerpc/include/asm/syscall.h @@ -22,10 +22,15 @@ extern const unsigned long sys_call_table[]; #endif /* CONFIG_FTRACE_SYSCALLS */ -static inline long syscall_get_nr(struct task_struct *task, - struct pt_regs *regs) +static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) { - return TRAP(regs) == 0xc00 ? regs->gpr[0] : -1L; + /* + * Note that we are returning an int here. That means 0xffffffff, ie. + * 32-bit negative 1, will be interpreted as -1 on a 64-bit kernel. + * This is important for seccomp so that compat tasks can set r0 = -1 + * to reject the syscall. + */ + return TRAP(regs) == 0xc00 ? regs->gpr[0] : -1; } static inline void syscall_rollback(struct task_struct *task, @@ -34,12 +39,6 @@ static inline void syscall_rollback(struct task_struct *task, regs->gpr[3] = regs->orig_gpr3; } -static inline long syscall_get_error(struct task_struct *task, - struct pt_regs *regs) -{ - return (regs->ccr & 0x10000000) ? -regs->gpr[3] : 0; -} - static inline long syscall_get_return_value(struct task_struct *task, struct pt_regs *regs) { @@ -50,9 +49,15 @@ static inline void syscall_set_return_value(struct task_struct *task, struct pt_regs *regs, int error, long val) { + /* + * In the general case it's not obvious that we must deal with CCR + * here, as the syscall exit path will also do that for us. However + * there are some places, eg. the signal code, which check ccr to + * decide if the value in r3 is actually an error. + */ if (error) { regs->ccr |= 0x10000000L; - regs->gpr[3] = -error; + regs->gpr[3] = error; } else { regs->ccr &= ~0x10000000L; regs->gpr[3] = val; @@ -64,19 +69,22 @@ static inline void syscall_get_arguments(struct task_struct *task, unsigned int i, unsigned int n, unsigned long *args) { + unsigned long val, mask = -1UL; + BUG_ON(i + n > 6); -#ifdef CONFIG_PPC64 - if (test_tsk_thread_flag(task, TIF_32BIT)) { - /* - * Zero-extend 32-bit argument values. The high bits are - * garbage ignored by the actual syscall dispatch. - */ - while (n-- > 0) - args[n] = (u32) regs->gpr[3 + i + n]; - return; - } + +#ifdef CONFIG_COMPAT + if (test_tsk_thread_flag(task, TIF_32BIT)) + mask = 0xffffffff; #endif - memcpy(args, ®s->gpr[3 + i], n * sizeof(args[0])); + while (n--) { + if (n == 0 && i == 0) + val = regs->orig_gpr3; + else + val = regs->gpr[3 + i + n]; + + args[n] = val & mask; + } } static inline void syscall_set_arguments(struct task_struct *task, @@ -86,6 +94,10 @@ static inline void syscall_set_arguments(struct task_struct *task, { BUG_ON(i + n > 6); memcpy(®s->gpr[3 + i], args, n * sizeof(args[0])); + + /* Also copy the first argument into orig_gpr3 */ + if (i == 0 && n > 0) + regs->orig_gpr3 = args[0]; } static inline int syscall_get_arch(void) diff --git a/arch/powerpc/include/asm/trace_clock.h b/arch/powerpc/include/asm/trace_clock.h new file mode 100644 index 00000000000000..cf1ee75ca0695a --- /dev/null +++ b/arch/powerpc/include/asm/trace_clock.h @@ -0,0 +1,19 @@ +/* + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2, as + * published by the Free Software Foundation. + * + * Copyright (C) 2015 Naveen N. Rao, IBM Corporation + */ + +#ifndef _ASM_PPC_TRACE_CLOCK_H +#define _ASM_PPC_TRACE_CLOCK_H + +#include +#include + +extern u64 notrace trace_clock_ppc_tb(void); + +#define ARCH_TRACE_CLOCKS { trace_clock_ppc_tb, "ppc-tb", 0 }, + +#endif /* _ASM_PPC_TRACE_CLOCK_H */ diff --git a/arch/powerpc/include/uapi/asm/Kbuild b/arch/powerpc/include/uapi/asm/Kbuild index f44a027818afe5..dab3717e3ea09d 100644 --- a/arch/powerpc/include/uapi/asm/Kbuild +++ b/arch/powerpc/include/uapi/asm/Kbuild @@ -6,6 +6,7 @@ header-y += bitsperlong.h header-y += bootx.h header-y += byteorder.h header-y += cputable.h +header-y += eeh.h header-y += elf.h header-y += epapr_hcalls.h header-y += errno.h diff --git a/arch/powerpc/include/uapi/asm/errno.h b/arch/powerpc/include/uapi/asm/errno.h index 8c145fd17d8678..e8b6b5f7de7c4b 100644 --- a/arch/powerpc/include/uapi/asm/errno.h +++ b/arch/powerpc/include/uapi/asm/errno.h @@ -6,6 +6,4 @@ #undef EDEADLOCK #define EDEADLOCK 58 /* File locking deadlock error */ -#define _LAST_ERRNO 516 - #endif /* _ASM_POWERPC_ERRNO_H */ diff --git a/arch/powerpc/include/uapi/asm/sigcontext.h b/arch/powerpc/include/uapi/asm/sigcontext.h index 9c1f24fd5d112a..3ad0c7f001a952 100644 --- a/arch/powerpc/include/uapi/asm/sigcontext.h +++ b/arch/powerpc/include/uapi/asm/sigcontext.h @@ -28,7 +28,7 @@ struct sigcontext { /* * To maintain compatibility with current implementations the sigcontext is * extended by appending a pointer (v_regs) to a quadword type (elf_vrreg_t) - * followed by an unstructured (vmx_reserve) field of 69 doublewords. This + * followed by an unstructured (vmx_reserve) field of 101 doublewords. This * allows the array of vector registers to be quadword aligned independent of * the alignment of the containing sigcontext or ucontext. It is the * responsibility of the code setting the sigcontext to set this pointer to @@ -80,7 +80,7 @@ struct sigcontext { * registers and vscr/vrsave. */ elf_vrreg_t __user *v_regs; - long vmx_reserve[ELF_NVRREG+ELF_NVRREG+32+1]; + long vmx_reserve[ELF_NVRREG + ELF_NVRREG + 1 + 32]; #endif }; diff --git a/arch/powerpc/kernel/Makefile b/arch/powerpc/kernel/Makefile index 12868b1c4e0562..ba336930d44879 100644 --- a/arch/powerpc/kernel/Makefile +++ b/arch/powerpc/kernel/Makefile @@ -118,6 +118,7 @@ obj-$(CONFIG_PPC_IO_WORKAROUNDS) += io-workarounds.o obj-$(CONFIG_DYNAMIC_FTRACE) += ftrace.o obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o obj-$(CONFIG_FTRACE_SYSCALLS) += ftrace.o +obj-$(CONFIG_TRACING) += trace_clock.o ifneq ($(CONFIG_PPC_INDIRECT_PIO),y) obj-y += iomap.o diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c index 98230579d99c74..810f433731dcac 100644 --- a/arch/powerpc/kernel/asm-offsets.c +++ b/arch/powerpc/kernel/asm-offsets.c @@ -213,7 +213,6 @@ int main(void) offsetof(struct tlb_core_data, esel_max)); DEFINE(TCD_ESEL_FIRST, offsetof(struct tlb_core_data, esel_first)); - DEFINE(TCD_LOCK, offsetof(struct tlb_core_data, lock)); #endif /* CONFIG_PPC_BOOK3E */ #ifdef CONFIG_PPC_STD_MMU_64 diff --git a/arch/powerpc/kernel/dma-iommu.c b/arch/powerpc/kernel/dma-iommu.c index 4c68bfe4108a98..41a7d9d49a5a3c 100644 --- a/arch/powerpc/kernel/dma-iommu.c +++ b/arch/powerpc/kernel/dma-iommu.c @@ -73,7 +73,7 @@ static void dma_iommu_unmap_sg(struct device *dev, struct scatterlist *sglist, } /* We support DMA to/from any memory page via the iommu */ -static int dma_iommu_dma_supported(struct device *dev, u64 mask) +int dma_iommu_dma_supported(struct device *dev, u64 mask) { struct iommu_table *tbl = get_iommu_table_base(dev); diff --git a/arch/powerpc/kernel/dma-swiotlb.c b/arch/powerpc/kernel/dma-swiotlb.c index 6e8d764ce47bc2..c6689f658b50fe 100644 --- a/arch/powerpc/kernel/dma-swiotlb.c +++ b/arch/powerpc/kernel/dma-swiotlb.c @@ -47,8 +47,8 @@ static u64 swiotlb_powerpc_get_required(struct device *dev) * for everything else. */ struct dma_map_ops swiotlb_dma_ops = { - .alloc = dma_direct_alloc_coherent, - .free = dma_direct_free_coherent, + .alloc = __dma_direct_alloc_coherent, + .free = __dma_direct_free_coherent, .mmap = dma_direct_mmap_coherent, .map_sg = swiotlb_map_sg_attrs, .unmap_sg = swiotlb_unmap_sg_attrs, diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c index 35e4dcc5dce362..59503ed98e5fcd 100644 --- a/arch/powerpc/kernel/dma.c +++ b/arch/powerpc/kernel/dma.c @@ -16,6 +16,7 @@ #include #include #include +#include /* * Generic direct DMA implementation @@ -39,9 +40,31 @@ static u64 __maybe_unused get_pfn_limit(struct device *dev) return pfn; } -void *dma_direct_alloc_coherent(struct device *dev, size_t size, - dma_addr_t *dma_handle, gfp_t flag, - struct dma_attrs *attrs) +static int dma_direct_dma_supported(struct device *dev, u64 mask) +{ +#ifdef CONFIG_PPC64 + u64 limit = get_dma_offset(dev) + (memblock_end_of_DRAM() - 1); + + /* Limit fits in the mask, we are good */ + if (mask >= limit) + return 1; + +#ifdef CONFIG_FSL_SOC + /* Freescale gets another chance via ZONE_DMA/ZONE_DMA32, however + * that will have to be refined if/when they support iommus + */ + return 1; +#endif + /* Sorry ... */ + return 0; +#else + return 1; +#endif +} + +void *__dma_direct_alloc_coherent(struct device *dev, size_t size, + dma_addr_t *dma_handle, gfp_t flag, + struct dma_attrs *attrs) { void *ret; #ifdef CONFIG_NOT_COHERENT_CACHE @@ -96,9 +119,9 @@ void *dma_direct_alloc_coherent(struct device *dev, size_t size, #endif } -void dma_direct_free_coherent(struct device *dev, size_t size, - void *vaddr, dma_addr_t dma_handle, - struct dma_attrs *attrs) +void __dma_direct_free_coherent(struct device *dev, size_t size, + void *vaddr, dma_addr_t dma_handle, + struct dma_attrs *attrs) { #ifdef CONFIG_NOT_COHERENT_CACHE __dma_free_coherent(size, vaddr); @@ -107,6 +130,51 @@ void dma_direct_free_coherent(struct device *dev, size_t size, #endif } +static void *dma_direct_alloc_coherent(struct device *dev, size_t size, + dma_addr_t *dma_handle, gfp_t flag, + struct dma_attrs *attrs) +{ + struct iommu_table *iommu; + + /* The coherent mask may be smaller than the real mask, check if + * we can really use the direct ops + */ + if (dma_direct_dma_supported(dev, dev->coherent_dma_mask)) + return __dma_direct_alloc_coherent(dev, size, dma_handle, + flag, attrs); + + /* Ok we can't ... do we have an iommu ? If not, fail */ + iommu = get_iommu_table_base(dev); + if (!iommu) + return NULL; + + /* Try to use the iommu */ + return iommu_alloc_coherent(dev, iommu, size, dma_handle, + dev->coherent_dma_mask, flag, + dev_to_node(dev)); +} + +static void dma_direct_free_coherent(struct device *dev, size_t size, + void *vaddr, dma_addr_t dma_handle, + struct dma_attrs *attrs) +{ + struct iommu_table *iommu; + + /* See comments in dma_direct_alloc_coherent() */ + if (dma_direct_dma_supported(dev, dev->coherent_dma_mask)) + return __dma_direct_free_coherent(dev, size, vaddr, dma_handle, + attrs); + /* Maybe we used an iommu ... */ + iommu = get_iommu_table_base(dev); + + /* If we hit that we should have never allocated in the first + * place so how come we are freeing ? + */ + if (WARN_ON(!iommu)) + return; + iommu_free_coherent(iommu, size, vaddr, dma_handle); +} + int dma_direct_mmap_coherent(struct device *dev, struct vm_area_struct *vma, void *cpu_addr, dma_addr_t handle, size_t size, struct dma_attrs *attrs) @@ -147,18 +215,6 @@ static void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sg, { } -static int dma_direct_dma_supported(struct device *dev, u64 mask) -{ -#ifdef CONFIG_PPC64 - /* Could be improved so platforms can set the limit in case - * they have limited DMA windows - */ - return mask >= get_dma_offset(dev) + (memblock_end_of_DRAM() - 1); -#else - return 1; -#endif -} - static u64 dma_direct_get_required_mask(struct device *dev) { u64 end, mask; @@ -230,6 +286,25 @@ struct dma_map_ops dma_direct_ops = { }; EXPORT_SYMBOL(dma_direct_ops); +int dma_set_coherent_mask(struct device *dev, u64 mask) +{ + if (!dma_supported(dev, mask)) { + /* + * We need to special case the direct DMA ops which can + * support a fallback for coherent allocations. There + * is no dma_op->set_coherent_mask() so we have to do + * things the hard way: + */ + if (get_dma_ops(dev) != &dma_direct_ops || + get_iommu_table_base(dev) == NULL || + !dma_iommu_dma_supported(dev, mask)) + return -EIO; + } + dev->coherent_dma_mask = mask; + return 0; +} +EXPORT_SYMBOL_GPL(dma_set_coherent_mask); + #define PREALLOC_DMA_DEBUG_ENTRIES (1 << 16) int __dma_set_mask(struct device *dev, u64 dma_mask) @@ -278,6 +353,13 @@ u64 dma_get_required_mask(struct device *dev) if (ppc_md.dma_get_required_mask) return ppc_md.dma_get_required_mask(dev); + if (dev_is_pci(dev)) { + struct pci_dev *pdev = to_pci_dev(dev); + struct pci_controller *phb = pci_bus_to_host(pdev->bus); + if (phb->controller_ops.dma_get_required_mask) + return phb->controller_ops.dma_get_required_mask(pdev); + } + return __dma_get_required_mask(dev); } EXPORT_SYMBOL_GPL(dma_get_required_mask); diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c index af9b597b10af65..e968533e3e0576 100644 --- a/arch/powerpc/kernel/eeh.c +++ b/arch/powerpc/kernel/eeh.c @@ -308,11 +308,26 @@ void eeh_slot_error_detail(struct eeh_pe *pe, int severity) if (!(pe->type & EEH_PE_PHB)) { if (eeh_has_flag(EEH_ENABLE_IO_FOR_LOG)) eeh_pci_enable(pe, EEH_OPT_THAW_MMIO); + + /* + * The config space of some PCI devices can't be accessed + * when their PEs are in frozen state. Otherwise, fenced + * PHB might be seen. Those PEs are identified with flag + * EEH_PE_CFG_RESTRICTED, indicating EEH_PE_CFG_BLOCKED + * is set automatically when the PE is put to EEH_PE_ISOLATED. + * + * Restoring BARs possibly triggers PCI config access in + * (OPAL) firmware and then causes fenced PHB. If the + * PCI config is blocked with flag EEH_PE_CFG_BLOCKED, it's + * pointless to restore BARs and dump config space. + */ eeh_ops->configure_bridge(pe); - eeh_pe_restore_bars(pe); + if (!(pe->state & EEH_PE_CFG_BLOCKED)) { + eeh_pe_restore_bars(pe); - pci_regs_buf[0] = 0; - eeh_pe_traverse(pe, eeh_dump_pe_log, &loglen); + pci_regs_buf[0] = 0; + eeh_pe_traverse(pe, eeh_dump_pe_log, &loglen); + } } eeh_ops->get_log(pe, severity, pci_regs_buf, loglen); @@ -750,14 +765,14 @@ int pcibios_set_pcie_reset_state(struct pci_dev *dev, enum pcie_reset_state stat eeh_pe_state_clear(pe, EEH_PE_ISOLATED); break; case pcie_hot_reset: - eeh_pe_state_mark(pe, EEH_PE_ISOLATED); + eeh_pe_state_mark_with_cfg(pe, EEH_PE_ISOLATED); eeh_ops->set_option(pe, EEH_OPT_FREEZE_PE); eeh_pe_dev_traverse(pe, eeh_disable_and_save_dev_state, dev); eeh_pe_state_mark(pe, EEH_PE_CFG_BLOCKED); eeh_ops->reset(pe, EEH_RESET_HOT); break; case pcie_warm_reset: - eeh_pe_state_mark(pe, EEH_PE_ISOLATED); + eeh_pe_state_mark_with_cfg(pe, EEH_PE_ISOLATED); eeh_ops->set_option(pe, EEH_OPT_FREEZE_PE); eeh_pe_dev_traverse(pe, eeh_disable_and_save_dev_state, dev); eeh_pe_state_mark(pe, EEH_PE_CFG_BLOCKED); @@ -1116,9 +1131,6 @@ void eeh_add_device_late(struct pci_dev *dev) return; } - if (eeh_has_flag(EEH_PROBE_MODE_DEV)) - eeh_ops->probe(pdn, NULL); - /* * The EEH cache might not be removed correctly because of * unbalanced kref to the device during unplug time, which @@ -1142,6 +1154,9 @@ void eeh_add_device_late(struct pci_dev *dev) dev->dev.archdata.edev = NULL; } + if (eeh_has_flag(EEH_PROBE_MODE_DEV)) + eeh_ops->probe(pdn, NULL); + edev->pdev = dev; dev->dev.archdata.edev = edev; diff --git a/arch/powerpc/kernel/eeh_pe.c b/arch/powerpc/kernel/eeh_pe.c index 35f0b62259bbd8..8654cb166c19b0 100644 --- a/arch/powerpc/kernel/eeh_pe.c +++ b/arch/powerpc/kernel/eeh_pe.c @@ -657,6 +657,28 @@ void eeh_pe_state_clear(struct eeh_pe *pe, int state) eeh_pe_traverse(pe, __eeh_pe_state_clear, &state); } +/** + * eeh_pe_state_mark_with_cfg - Mark PE state with unblocked config space + * @pe: PE + * @state: PE state to be set + * + * Set specified flag to PE and its child PEs. The PCI config space + * of some PEs is blocked automatically when EEH_PE_ISOLATED is set, + * which isn't needed in some situations. The function allows to set + * the specified flag to indicated PEs without blocking their PCI + * config space. + */ +void eeh_pe_state_mark_with_cfg(struct eeh_pe *pe, int state) +{ + eeh_pe_traverse(pe, __eeh_pe_state_mark, &state); + if (!(state & EEH_PE_ISOLATED)) + return; + + /* Clear EEH_PE_CFG_BLOCKED, which might be set just now */ + state = EEH_PE_CFG_BLOCKED; + eeh_pe_traverse(pe, __eeh_pe_state_clear, &state); +} + /* * Some PCI bridges (e.g. PLX bridges) have primary/secondary * buses assigned explicitly by firmware, and we probably have diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index 46fc0f4d8982c6..2405631e91a29d 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -20,6 +20,7 @@ */ #include +#include #include #include #include @@ -354,7 +355,7 @@ ret_from_syscall: SYNC MTMSRD(r10) lwz r9,TI_FLAGS(r12) - li r8,-_LAST_ERRNO + li r8,-MAX_ERRNO andi. r0,r9,(_TIF_SYSCALL_DOTRACE|_TIF_SINGLESTEP|_TIF_USER_WORK_MASK|_TIF_PERSYSCALL_MASK) bne- syscall_exit_work cmplw 0,r3,r8 @@ -457,6 +458,10 @@ syscall_dotrace: lwz r7,GPR7(r1) lwz r8,GPR8(r1) REST_NVGPRS(r1) + + cmplwi r0,NR_syscalls + /* Return code is already in r3 thanks to do_syscall_trace_enter() */ + bge- ret_from_syscall b syscall_dotrace_cont syscall_exit_work: diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S index 579e0f9a2d5700..a94f155db78e83 100644 --- a/arch/powerpc/kernel/entry_64.S +++ b/arch/powerpc/kernel/entry_64.S @@ -19,6 +19,7 @@ */ #include +#include #include #include #include @@ -150,8 +151,7 @@ END_FW_FTR_SECTION_IFSET(FW_FEATURE_SPLPAR) CURRENT_THREAD_INFO(r11, r1) ld r10,TI_FLAGS(r11) andi. r11,r10,_TIF_SYSCALL_DOTRACE - bne syscall_dotrace -.Lsyscall_dotrace_cont: + bne syscall_dotrace /* does not return */ cmpldi 0,r0,NR_syscalls bge- syscall_enosys @@ -207,7 +207,7 @@ system_call: /* label this so stack traces look sane */ #endif /* CONFIG_PPC_BOOK3E */ ld r9,TI_FLAGS(r12) - li r11,-_LAST_ERRNO + li r11,-MAX_ERRNO andi. r0,r9,(_TIF_SYSCALL_DOTRACE|_TIF_SINGLESTEP|_TIF_USER_WORK_MASK|_TIF_PERSYSCALL_MASK) bne- syscall_exit_work cmpld r3,r11 @@ -245,22 +245,34 @@ syscall_dotrace: bl save_nvgprs addi r3,r1,STACK_FRAME_OVERHEAD bl do_syscall_trace_enter + /* - * Restore argument registers possibly just changed. - * We use the return value of do_syscall_trace_enter - * for the call number to look up in the table (r0). + * We use the return value of do_syscall_trace_enter() as the syscall + * number. If the syscall was rejected for any reason do_syscall_trace_enter() + * returns an invalid syscall number and the test below against + * NR_syscalls will fail. */ mr r0,r3 + + /* Restore argument registers just clobbered and/or possibly changed. */ ld r3,GPR3(r1) ld r4,GPR4(r1) ld r5,GPR5(r1) ld r6,GPR6(r1) ld r7,GPR7(r1) ld r8,GPR8(r1) + + /* Repopulate r9 and r10 for the system_call path */ addi r9,r1,STACK_FRAME_OVERHEAD CURRENT_THREAD_INFO(r10, r1) ld r10,TI_FLAGS(r10) - b .Lsyscall_dotrace_cont + + cmpldi r0,NR_syscalls + blt+ system_call + + /* Return code is already in r3 thanks to do_syscall_trace_enter() */ + b .Lsyscall_exit + syscall_enosys: li r3,-ENOSYS @@ -277,7 +289,7 @@ syscall_exit_work: beq+ 0f REST_NVGPRS(r1) b 2f -0: cmpld r3,r11 /* r10 is -LAST_ERRNO */ +0: cmpld r3,r11 /* r11 is -MAX_ERRNO */ blt+ 1f andi. r0,r9,_TIF_NOERROR bne- 1f diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S index 3e68d1c6971854..f3bd5e747ed840 100644 --- a/arch/powerpc/kernel/exceptions-64e.S +++ b/arch/powerpc/kernel/exceptions-64e.S @@ -1313,11 +1313,14 @@ skpinv: addi r6,r6,1 /* Increment */ sync isync -/* The mapping only needs to be cache-coherent on SMP */ -#ifdef CONFIG_SMP -#define M_IF_SMP MAS2_M +/* + * The mapping only needs to be cache-coherent on SMP, except on + * Freescale e500mc derivatives where it's also needed for coherent DMA. + */ +#if defined(CONFIG_SMP) || defined(CONFIG_PPC_E500MC) +#define M_IF_NEEDED MAS2_M #else -#define M_IF_SMP 0 +#define M_IF_NEEDED 0 #endif /* 6. Setup KERNELBASE mapping in TLB[0] @@ -1332,7 +1335,7 @@ skpinv: addi r6,r6,1 /* Increment */ ori r6,r6,(MAS1_TSIZE(BOOK3E_PAGESZ_1GB))@l mtspr SPRN_MAS1,r6 - LOAD_REG_IMMEDIATE(r6, PAGE_OFFSET | M_IF_SMP) + LOAD_REG_IMMEDIATE(r6, PAGE_OFFSET | M_IF_NEEDED) mtspr SPRN_MAS2,r6 rlwinm r5,r5,0,0,25 diff --git a/arch/powerpc/kernel/fsl_booke_entry_mapping.S b/arch/powerpc/kernel/fsl_booke_entry_mapping.S index f22e7e44fbf321..83dd0f6776b3e6 100644 --- a/arch/powerpc/kernel/fsl_booke_entry_mapping.S +++ b/arch/powerpc/kernel/fsl_booke_entry_mapping.S @@ -152,11 +152,14 @@ skpinv: addi r6,r6,1 /* Increment */ tlbivax 0,r9 TLBSYNC -/* The mapping only needs to be cache-coherent on SMP */ -#ifdef CONFIG_SMP -#define M_IF_SMP MAS2_M +/* + * The mapping only needs to be cache-coherent on SMP, except on + * Freescale e500mc derivatives where it's also needed for coherent DMA. + */ +#if defined(CONFIG_SMP) || defined(CONFIG_PPC_E500MC) +#define M_IF_NEEDED MAS2_M #else -#define M_IF_SMP 0 +#define M_IF_NEEDED 0 #endif #if defined(ENTRY_MAPPING_BOOT_SETUP) @@ -167,8 +170,8 @@ skpinv: addi r6,r6,1 /* Increment */ lis r6,(MAS1_VALID|MAS1_IPROT)@h ori r6,r6,(MAS1_TSIZE(BOOK3E_PAGESZ_64M))@l mtspr SPRN_MAS1,r6 - lis r6,MAS2_VAL(PAGE_OFFSET, BOOK3E_PAGESZ_64M, M_IF_SMP)@h - ori r6,r6,MAS2_VAL(PAGE_OFFSET, BOOK3E_PAGESZ_64M, M_IF_SMP)@l + lis r6,MAS2_VAL(PAGE_OFFSET, BOOK3E_PAGESZ_64M, M_IF_NEEDED)@h + ori r6,r6,MAS2_VAL(PAGE_OFFSET, BOOK3E_PAGESZ_64M, M_IF_NEEDED)@l mtspr SPRN_MAS2,r6 mtspr SPRN_MAS3,r8 tlbwe diff --git a/arch/powerpc/kernel/jump_label.c b/arch/powerpc/kernel/jump_label.c index a1ed8a8c7cb42c..6472472093d084 100644 --- a/arch/powerpc/kernel/jump_label.c +++ b/arch/powerpc/kernel/jump_label.c @@ -17,7 +17,7 @@ void arch_jump_label_transform(struct jump_entry *entry, { u32 *addr = (u32 *)(unsigned long)entry->code; - if (type == JUMP_LABEL_ENABLE) + if (type == JUMP_LABEL_JMP) patch_branch(addr, entry->target, 0); else patch_instruction(addr, PPC_INST_NOP); diff --git a/arch/powerpc/kernel/kvm.c b/arch/powerpc/kernel/kvm.c index 33aa4ddf597dc7..9ad37f827a975f 100644 --- a/arch/powerpc/kernel/kvm.c +++ b/arch/powerpc/kernel/kvm.c @@ -649,7 +649,6 @@ static void kvm_check_ins(u32 *inst, u32 features) kvm_patch_ins_mtsrin(inst, inst_rt, inst_rb); } break; - break; #endif } diff --git a/arch/powerpc/kernel/misc_32.S b/arch/powerpc/kernel/misc_32.S index 7c6bb4b17b4960..ed3ab509facac9 100644 --- a/arch/powerpc/kernel/misc_32.S +++ b/arch/powerpc/kernel/misc_32.S @@ -595,25 +595,6 @@ _GLOBAL(copy_page) li r11,4 b 2b -/* - * void atomic_clear_mask(atomic_t mask, atomic_t *addr) - * void atomic_set_mask(atomic_t mask, atomic_t *addr); - */ -_GLOBAL(atomic_clear_mask) -10: lwarx r5,0,r4 - andc r5,r5,r3 - PPC405_ERR77(0,r4) - stwcx. r5,0,r4 - bne- 10b - blr -_GLOBAL(atomic_set_mask) -10: lwarx r5,0,r4 - or r5,r5,r3 - PPC405_ERR77(0,r4) - stwcx. r5,0,r4 - bne- 10b - blr - /* * Extended precision shifts. * diff --git a/arch/powerpc/kernel/misc_64.S b/arch/powerpc/kernel/misc_64.S index 4e314b90c75d69..6e4168cf469817 100644 --- a/arch/powerpc/kernel/misc_64.S +++ b/arch/powerpc/kernel/misc_64.S @@ -475,9 +475,18 @@ _GLOBAL(kexec_wait) #ifdef CONFIG_KEXEC /* use no memory without kexec */ lwz r4,0(r5) cmpwi 0,r4,0 - bnea 0x60 + beq 99b +#ifdef CONFIG_PPC_BOOK3S_64 + li r10,0x60 + mfmsr r11 + clrrdi r11,r11,1 /* Clear MSR_LE */ + mtsrr0 r10 + mtsrr1 r11 + rfid +#else + ba 0x60 +#endif #endif - b 99b /* this can be in text because we won't change it until we are * running in real anyways diff --git a/arch/powerpc/kernel/nvram_64.c b/arch/powerpc/kernel/nvram_64.c index 1e703f8ebad4e3..98ba106a59efff 100644 --- a/arch/powerpc/kernel/nvram_64.c +++ b/arch/powerpc/kernel/nvram_64.c @@ -541,10 +541,9 @@ static ssize_t nvram_pstore_read(u64 *id, enum pstore_type_id *type, time->tv_sec = be64_to_cpu(oops_hdr->timestamp); time->tv_nsec = 0; } - *buf = kmalloc(length, GFP_KERNEL); + *buf = kmemdup(buff + hdr_size, length, GFP_KERNEL); if (*buf == NULL) return -ENOMEM; - memcpy(*buf, buff + hdr_size, length); kfree(buff); if (err_type == ERR_TYPE_KERNEL_PANIC_GZ) @@ -582,9 +581,10 @@ static int nvram_pstore_init(void) spin_lock_init(&nvram_pstore_info.buf_lock); rc = pstore_register(&nvram_pstore_info); - if (rc != 0) - pr_err("nvram: pstore_register() failed, defaults to " - "kmsg_dump; returned %d\n", rc); + if (rc && (rc != -EPERM)) + /* Print error only when pstore.backend == nvram */ + pr_err("nvram: pstore_register() failed, returned %d. " + "Defaults to kmsg_dump\n", rc); return rc; } diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c index 02c1d5dcee4d8c..a1d0632d97c69f 100644 --- a/arch/powerpc/kernel/pci-common.c +++ b/arch/powerpc/kernel/pci-common.c @@ -823,23 +823,15 @@ static void pcibios_fixup_resources(struct pci_dev *dev) (reg.start == 0 && !pci_has_flag(PCI_PROBE_ONLY))) { /* Only print message if not re-assigning */ if (!pci_has_flag(PCI_REASSIGN_ALL_RSRC)) - pr_debug("PCI:%s Resource %d %016llx-%016llx [%x] " - "is unassigned\n", - pci_name(dev), i, - (unsigned long long)res->start, - (unsigned long long)res->end, - (unsigned int)res->flags); + pr_debug("PCI:%s Resource %d %pR is unassigned\n", + pci_name(dev), i, res); res->end -= res->start; res->start = 0; res->flags |= IORESOURCE_UNSET; continue; } - pr_debug("PCI:%s Resource %d %016llx-%016llx [%x]\n", - pci_name(dev), i, - (unsigned long long)res->start,\ - (unsigned long long)res->end, - (unsigned int)res->flags); + pr_debug("PCI:%s Resource %d %pR\n", pci_name(dev), i, res); } /* Call machine specific resource fixup */ @@ -943,11 +935,7 @@ static void pcibios_fixup_bridge(struct pci_bus *bus) continue; } - pr_debug("PCI:%s Bus rsrc %d %016llx-%016llx [%x]\n", - pci_name(dev), i, - (unsigned long long)res->start,\ - (unsigned long long)res->end, - (unsigned int)res->flags); + pr_debug("PCI:%s Bus rsrc %d %pR\n", pci_name(dev), i, res); /* Try to detect uninitialized P2P bridge resources, * and clear them out so they get re-assigned later @@ -1126,10 +1114,8 @@ static int reparent_resources(struct resource *parent, *pp = NULL; for (p = res->child; p != NULL; p = p->sibling) { p->parent = res; - pr_debug("PCI: Reparented %s [%llx..%llx] under %s\n", - p->name, - (unsigned long long)p->start, - (unsigned long long)p->end, res->name); + pr_debug("PCI: Reparented %s %pR under %s\n", + p->name, p, res->name); } return 0; } @@ -1198,14 +1184,9 @@ static void pcibios_allocate_bus_resources(struct pci_bus *bus) } } - pr_debug("PCI: %s (bus %d) bridge rsrc %d: %016llx-%016llx " - "[0x%x], parent %p (%s)\n", - bus->self ? pci_name(bus->self) : "PHB", - bus->number, i, - (unsigned long long)res->start, - (unsigned long long)res->end, - (unsigned int)res->flags, - pr, (pr && pr->name) ? pr->name : "nil"); + pr_debug("PCI: %s (bus %d) bridge rsrc %d: %pR, parent %p (%s)\n", + bus->self ? pci_name(bus->self) : "PHB", bus->number, + i, res, pr, (pr && pr->name) ? pr->name : "nil"); if (pr && !(pr->flags & IORESOURCE_UNSET)) { struct pci_dev *dev = bus->self; @@ -1247,11 +1228,8 @@ static inline void alloc_resource(struct pci_dev *dev, int idx) { struct resource *pr, *r = &dev->resource[idx]; - pr_debug("PCI: Allocating %s: Resource %d: %016llx..%016llx [%x]\n", - pci_name(dev), idx, - (unsigned long long)r->start, - (unsigned long long)r->end, - (unsigned int)r->flags); + pr_debug("PCI: Allocating %s: Resource %d: %pR\n", + pci_name(dev), idx, r); pr = pci_find_parent_resource(dev, r); if (!pr || (pr->flags & IORESOURCE_UNSET) || @@ -1259,11 +1237,7 @@ static inline void alloc_resource(struct pci_dev *dev, int idx) printk(KERN_WARNING "PCI: Cannot allocate resource region %d" " of device %s, will remap\n", idx, pci_name(dev)); if (pr) - pr_debug("PCI: parent is %p: %016llx-%016llx [%x]\n", - pr, - (unsigned long long)pr->start, - (unsigned long long)pr->end, - (unsigned int)pr->flags); + pr_debug("PCI: parent is %p: %pR\n", pr, pr); /* We'll assign a new address later */ r->flags |= IORESOURCE_UNSET; r->end -= r->start; @@ -1425,12 +1399,8 @@ void pcibios_claim_one_bus(struct pci_bus *bus) if (r->parent || !r->start || !r->flags) continue; - pr_debug("PCI: Claiming %s: " - "Resource %d: %016llx..%016llx [%x]\n", - pci_name(dev), i, - (unsigned long long)r->start, - (unsigned long long)r->end, - (unsigned int)r->flags); + pr_debug("PCI: Claiming %s: Resource %d: %pR\n", + pci_name(dev), i, r); if (pci_claim_resource(dev, i) == 0) continue; @@ -1514,11 +1484,8 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose, } else { offset = pcibios_io_space_offset(hose); - pr_debug("PCI: PHB IO resource = %08llx-%08llx [%lx] off 0x%08llx\n", - (unsigned long long)res->start, - (unsigned long long)res->end, - (unsigned long)res->flags, - (unsigned long long)offset); + pr_debug("PCI: PHB IO resource = %pR off 0x%08llx\n", + res, (unsigned long long)offset); pci_add_resource_offset(resources, res, offset); } @@ -1535,11 +1502,8 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose, offset = hose->mem_offset[i]; - pr_debug("PCI: PHB MEM resource %d = %08llx-%08llx [%lx] off 0x%08llx\n", i, - (unsigned long long)res->start, - (unsigned long long)res->end, - (unsigned long)res->flags, - (unsigned long long)offset); + pr_debug("PCI: PHB MEM resource %d = %pR off 0x%08llx\n", i, + res, (unsigned long long)offset); pci_add_resource_offset(resources, res, offset); } diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c index 64e6e9d9e65628..75b6676c1a0b91 100644 --- a/arch/powerpc/kernel/process.c +++ b/arch/powerpc/kernel/process.c @@ -86,7 +86,7 @@ void giveup_fpu_maybe_transactional(struct task_struct *tsk) if (tsk == current && tsk->thread.regs && MSR_TM_ACTIVE(tsk->thread.regs->msr) && !test_thread_flag(TIF_RESTORE_TM)) { - tsk->thread.tm_orig_msr = tsk->thread.regs->msr; + tsk->thread.ckpt_regs.msr = tsk->thread.regs->msr; set_thread_flag(TIF_RESTORE_TM); } @@ -104,7 +104,7 @@ void giveup_altivec_maybe_transactional(struct task_struct *tsk) if (tsk == current && tsk->thread.regs && MSR_TM_ACTIVE(tsk->thread.regs->msr) && !test_thread_flag(TIF_RESTORE_TM)) { - tsk->thread.tm_orig_msr = tsk->thread.regs->msr; + tsk->thread.ckpt_regs.msr = tsk->thread.regs->msr; set_thread_flag(TIF_RESTORE_TM); } @@ -540,7 +540,7 @@ static void tm_reclaim_thread(struct thread_struct *thr, * the thread will no longer be transactional. */ if (test_ti_thread_flag(ti, TIF_RESTORE_TM)) { - msr_diff = thr->tm_orig_msr & ~thr->regs->msr; + msr_diff = thr->ckpt_regs.msr & ~thr->regs->msr; if (msr_diff & MSR_FP) memcpy(&thr->transact_fp, &thr->fp_state, sizeof(struct thread_fp_state)); @@ -591,10 +591,10 @@ static inline void tm_reclaim_task(struct task_struct *tsk) /* Stash the original thread MSR, as giveup_fpu et al will * modify it. We hold onto it to see whether the task used * FP & vector regs. If the TIF_RESTORE_TM flag is set, - * tm_orig_msr is already set. + * ckpt_regs.msr is already set. */ if (!test_ti_thread_flag(task_thread_info(tsk), TIF_RESTORE_TM)) - thr->tm_orig_msr = thr->regs->msr; + thr->ckpt_regs.msr = thr->regs->msr; TM_DEBUG("--- tm_reclaim on pid %d (NIP=%lx, " "ccr=%lx, msr=%lx, trap=%lx)\n", @@ -663,7 +663,7 @@ static inline void tm_recheckpoint_new_task(struct task_struct *new) tm_restore_sprs(&new->thread); return; } - msr = new->thread.tm_orig_msr; + msr = new->thread.ckpt_regs.msr; /* Recheckpoint to restore original checkpointed register state. */ TM_DEBUG("*** tm_recheckpoint of pid %d " "(new->msr 0x%lx, new->origmsr 0x%lx)\n", @@ -723,7 +723,7 @@ void restore_tm_state(struct pt_regs *regs) if (!MSR_TM_ACTIVE(regs->msr)) return; - msr_diff = current->thread.tm_orig_msr & ~regs->msr; + msr_diff = current->thread.ckpt_regs.msr & ~regs->msr; msr_diff &= MSR_FP | MSR_VEC | MSR_VSX; if (msr_diff & MSR_FP) { fp_enable(); diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c index 8b888b12a4752a..bef76c5033e4a9 100644 --- a/arch/powerpc/kernel/prom.c +++ b/arch/powerpc/kernel/prom.c @@ -218,22 +218,18 @@ static void __init check_cpu_pa_features(unsigned long node) } #ifdef CONFIG_PPC_STD_MMU_64 -static void __init check_cpu_slb_size(unsigned long node) +static void __init init_mmu_slb_size(unsigned long node) { const __be32 *slb_size_ptr; - slb_size_ptr = of_get_flat_dt_prop(node, "slb-size", NULL); - if (slb_size_ptr != NULL) { - mmu_slb_size = be32_to_cpup(slb_size_ptr); - return; - } - slb_size_ptr = of_get_flat_dt_prop(node, "ibm,slb-size", NULL); - if (slb_size_ptr != NULL) { + slb_size_ptr = of_get_flat_dt_prop(node, "slb-size", NULL) ? : + of_get_flat_dt_prop(node, "ibm,slb-size", NULL); + + if (slb_size_ptr) mmu_slb_size = be32_to_cpup(slb_size_ptr); - } } #else -#define check_cpu_slb_size(node) do { } while(0) +#define init_mmu_slb_size(node) do { } while(0) #endif static struct feature_property { @@ -380,7 +376,7 @@ static int __init early_init_dt_scan_cpus(unsigned long node, check_cpu_feature_properties(node); check_cpu_pa_features(node); - check_cpu_slb_size(node); + init_mmu_slb_size(node); #ifdef CONFIG_PPC64 if (nthreads > 1) @@ -476,9 +472,10 @@ static int __init early_init_dt_scan_drconf_memory(unsigned long node) flags = of_read_number(&dm[3], 1); /* skip DRC index, pad, assoc. list index, flags */ dm += 4; - /* skip this block if the reserved bit is set in flags (0x80) - or if the block is not assigned to this partition (0x8) */ - if ((flags & 0x80) || !(flags & 0x8)) + /* skip this block if the reserved bit is set in flags + or if the block is not assigned to this partition */ + if ((flags & DRCONF_MEM_RESERVED) || + !(flags & DRCONF_MEM_ASSIGNED)) continue; size = memblock_size; rngs = 1; diff --git a/arch/powerpc/kernel/prom_init.c b/arch/powerpc/kernel/prom_init.c index fcca8077e6a29c..15099c41622e99 100644 --- a/arch/powerpc/kernel/prom_init.c +++ b/arch/powerpc/kernel/prom_init.c @@ -641,6 +641,15 @@ static void __init early_cmdline_parse(void) #define W(x) ((x) >> 24) & 0xff, ((x) >> 16) & 0xff, \ ((x) >> 8) & 0xff, (x) & 0xff +/* Firmware expects the value to be n - 1, where n is the # of vectors */ +#define NUM_VECTORS(n) ((n) - 1) + +/* + * Firmware expects 1 + n - 2, where n is the length of the option vector in + * bytes. The 1 accounts for the length byte itself, the - 2 .. ? + */ +#define VECTOR_LENGTH(n) (1 + (n) - 2) + unsigned char ibm_architecture_vec[] = { W(0xfffe0000), W(0x003a0000), /* POWER5/POWER5+ */ W(0xffff0000), W(0x003e0000), /* POWER6 */ @@ -651,16 +660,16 @@ unsigned char ibm_architecture_vec[] = { W(0xffffffff), W(0x0f000003), /* all 2.06-compliant */ W(0xffffffff), W(0x0f000002), /* all 2.05-compliant */ W(0xfffffffe), W(0x0f000001), /* all 2.04-compliant and earlier */ - 6 - 1, /* 6 option vectors */ + NUM_VECTORS(6), /* 6 option vectors */ /* option vector 1: processor architectures supported */ - 3 - 2, /* length */ + VECTOR_LENGTH(2), /* length */ 0, /* don't ignore, don't halt */ OV1_PPC_2_00 | OV1_PPC_2_01 | OV1_PPC_2_02 | OV1_PPC_2_03 | OV1_PPC_2_04 | OV1_PPC_2_05 | OV1_PPC_2_06 | OV1_PPC_2_07, /* option vector 2: Open Firmware options supported */ - 34 - 2, /* length */ + VECTOR_LENGTH(33), /* length */ OV2_REAL_MODE, 0, 0, W(0xffffffff), /* real_base */ @@ -674,17 +683,17 @@ unsigned char ibm_architecture_vec[] = { 48, /* max log_2(hash table size) */ /* option vector 3: processor options supported */ - 3 - 2, /* length */ + VECTOR_LENGTH(2), /* length */ 0, /* don't ignore, don't halt */ OV3_FP | OV3_VMX | OV3_DFP, /* option vector 4: IBM PAPR implementation */ - 3 - 2, /* length */ + VECTOR_LENGTH(2), /* length */ 0, /* don't halt */ OV4_MIN_ENT_CAP, /* minimum VP entitled capacity */ /* option vector 5: PAPR/OF options */ - 19 - 2, /* length */ + VECTOR_LENGTH(18), /* length */ 0, /* don't ignore, don't halt */ OV5_FEAT(OV5_LPAR) | OV5_FEAT(OV5_SPLPAR) | OV5_FEAT(OV5_LARGE_PAGES) | OV5_FEAT(OV5_DRCONF_MEMORY) | OV5_FEAT(OV5_DONATE_DEDICATE_CPU) | @@ -717,12 +726,12 @@ unsigned char ibm_architecture_vec[] = { OV5_FEAT(OV5_PFO_HW_RNG) | OV5_FEAT(OV5_PFO_HW_ENCR) | OV5_FEAT(OV5_PFO_HW_842), OV5_FEAT(OV5_SUB_PROCESSORS), + /* option vector 6: IBM PAPR hints */ - 4 - 2, /* length */ + VECTOR_LENGTH(3), /* length */ 0, 0, OV6_LINUX, - }; /* Old method - ELF header with PT_NOTE sections only works on BE */ diff --git a/arch/powerpc/kernel/ptrace.c b/arch/powerpc/kernel/ptrace.c index f21897b420576c..737c0d0b53ac43 100644 --- a/arch/powerpc/kernel/ptrace.c +++ b/arch/powerpc/kernel/ptrace.c @@ -1762,26 +1762,81 @@ long arch_ptrace(struct task_struct *child, long request, return ret; } -/* - * We must return the syscall number to actually look up in the table. - * This can be -1L to skip running any syscall at all. +#ifdef CONFIG_SECCOMP +static int do_seccomp(struct pt_regs *regs) +{ + if (!test_thread_flag(TIF_SECCOMP)) + return 0; + + /* + * The ABI we present to seccomp tracers is that r3 contains + * the syscall return value and orig_gpr3 contains the first + * syscall parameter. This is different to the ptrace ABI where + * both r3 and orig_gpr3 contain the first syscall parameter. + */ + regs->gpr[3] = -ENOSYS; + + /* + * We use the __ version here because we have already checked + * TIF_SECCOMP. If this fails, there is nothing left to do, we + * have already loaded -ENOSYS into r3, or seccomp has put + * something else in r3 (via SECCOMP_RET_ERRNO/TRACE). + */ + if (__secure_computing()) + return -1; + + /* + * The syscall was allowed by seccomp, restore the register + * state to what ptrace and audit expect. + * Note that we use orig_gpr3, which means a seccomp tracer can + * modify the first syscall parameter (in orig_gpr3) and also + * allow the syscall to proceed. + */ + regs->gpr[3] = regs->orig_gpr3; + + return 0; +} +#else +static inline int do_seccomp(struct pt_regs *regs) { return 0; } +#endif /* CONFIG_SECCOMP */ + +/** + * do_syscall_trace_enter() - Do syscall tracing on kernel entry. + * @regs: the pt_regs of the task to trace (current) + * + * Performs various types of tracing on syscall entry. This includes seccomp, + * ptrace, syscall tracepoints and audit. + * + * The pt_regs are potentially visible to userspace via ptrace, so their + * contents is ABI. + * + * One or more of the tracers may modify the contents of pt_regs, in particular + * to modify arguments or even the syscall number itself. + * + * It's also possible that a tracer can choose to reject the system call. In + * that case this function will return an illegal syscall number, and will put + * an appropriate return value in regs->r3. + * + * Return: the (possibly changed) syscall number. */ long do_syscall_trace_enter(struct pt_regs *regs) { - long ret = 0; + bool abort = false; user_exit(); - secure_computing_strict(regs->gpr[0]); + if (do_seccomp(regs)) + return -1; - if (test_thread_flag(TIF_SYSCALL_TRACE) && - tracehook_report_syscall_entry(regs)) + if (test_thread_flag(TIF_SYSCALL_TRACE)) { /* - * Tracing decided this syscall should not happen. - * We'll return a bogus call number to get an ENOSYS - * error, but leave the original number in regs->gpr[0]. + * The tracer may decide to abort the syscall, if so tracehook + * will return !0. Note that the tracer may also just change + * regs->gpr[0] to an invalid syscall number, that is handled + * below on the exit path. */ - ret = -1L; + abort = tracehook_report_syscall_entry(regs) != 0; + } if (unlikely(test_thread_flag(TIF_SYSCALL_TRACEPOINT))) trace_sys_enter(regs, regs->gpr[0]); @@ -1798,7 +1853,17 @@ long do_syscall_trace_enter(struct pt_regs *regs) regs->gpr[5] & 0xffffffff, regs->gpr[6] & 0xffffffff); - return ret ?: regs->gpr[0]; + if (abort || regs->gpr[0] >= NR_syscalls) { + /* + * If we are aborting explicitly, or if the syscall number is + * now invalid, set the return value to -ENOSYS. + */ + regs->gpr[3] = -ENOSYS; + return -1; + } + + /* Return the possibly modified but valid syscall number */ + return regs->gpr[0]; } void do_syscall_trace_leave(struct pt_regs *regs) diff --git a/arch/powerpc/kernel/rtas.c b/arch/powerpc/kernel/rtas.c index 7a488c108410b7..84bf934cf74874 100644 --- a/arch/powerpc/kernel/rtas.c +++ b/arch/powerpc/kernel/rtas.c @@ -478,8 +478,9 @@ unsigned int rtas_busy_delay_time(int status) if (status == RTAS_BUSY) { ms = 1; - } else if (status >= 9900 && status <= 9905) { - order = status - 9900; + } else if (status >= RTAS_EXTENDED_DELAY_MIN && + status <= RTAS_EXTENDED_DELAY_MAX) { + order = status - RTAS_EXTENDED_DELAY_MIN; for (ms = 1; order > 0; order--) ms *= 10; } @@ -584,6 +585,23 @@ int rtas_get_sensor(int sensor, int index, int *state) } EXPORT_SYMBOL(rtas_get_sensor); +int rtas_get_sensor_fast(int sensor, int index, int *state) +{ + int token = rtas_token("get-sensor-state"); + int rc; + + if (token == RTAS_UNKNOWN_SERVICE) + return -ENOENT; + + rc = rtas_call(token, 2, 2, state, sensor, index); + WARN_ON(rc == RTAS_BUSY || (rc >= RTAS_EXTENDED_DELAY_MIN && + rc <= RTAS_EXTENDED_DELAY_MAX)); + + if (rc < 0) + return rtas_error_rc(rc); + return rc; +} + bool rtas_indicator_present(int token, int *maxindex) { int proplen, count, i; @@ -641,7 +659,8 @@ int rtas_set_indicator_fast(int indicator, int index, int new_value) rc = rtas_call(token, 3, 1, NULL, indicator, index, new_value); - WARN_ON(rc == -2 || (rc >= 9900 && rc <= 9905)); + WARN_ON(rc == RTAS_BUSY || (rc >= RTAS_EXTENDED_DELAY_MIN && + rc <= RTAS_EXTENDED_DELAY_MAX)); if (rc < 0) return rtas_error_rc(rc); diff --git a/arch/powerpc/kernel/signal_32.c b/arch/powerpc/kernel/signal_32.c index da50e0c9c57e69..0dbee465af7a70 100644 --- a/arch/powerpc/kernel/signal_32.c +++ b/arch/powerpc/kernel/signal_32.c @@ -949,6 +949,11 @@ int copy_siginfo_to_user32(struct compat_siginfo __user *d, const siginfo_t *s) err |= __put_user(s->si_overrun, &d->si_overrun); err |= __put_user(s->si_int, &d->si_int); break; + case __SI_SYS >> 16: + err |= __put_user(ptr_to_compat(s->si_call_addr), &d->si_call_addr); + err |= __put_user(s->si_syscall, &d->si_syscall); + err |= __put_user(s->si_arch, &d->si_arch); + break; case __SI_RT >> 16: /* This is not generated by the kernel as of now. */ case __SI_MESGQ >> 16: err |= __put_user(s->si_int, &d->si_int); diff --git a/arch/powerpc/kernel/signal_64.c b/arch/powerpc/kernel/signal_64.c index c7c24d2e2bdbc6..20756dfb9f3462 100644 --- a/arch/powerpc/kernel/signal_64.c +++ b/arch/powerpc/kernel/signal_64.c @@ -73,6 +73,19 @@ static const char fmt32[] = KERN_INFO \ static const char fmt64[] = KERN_INFO \ "%s[%d]: bad frame in %s: %016lx nip %016lx lr %016lx\n"; +/* + * This computes a quad word aligned pointer inside the vmx_reserve array + * element. For historical reasons sigcontext might not be quad word aligned, + * but the location we write the VMX regs to must be. See the comment in + * sigcontext for more detail. + */ +#ifdef CONFIG_ALTIVEC +static elf_vrreg_t __user *sigcontext_vmx_regs(struct sigcontext __user *sc) +{ + return (elf_vrreg_t __user *) (((unsigned long)sc->vmx_reserve + 15) & ~0xful); +} +#endif + /* * Set up the sigcontext for the signal frame. */ @@ -90,7 +103,7 @@ static long setup_sigcontext(struct sigcontext __user *sc, struct pt_regs *regs, * v_regs pointer or not */ #ifdef CONFIG_ALTIVEC - elf_vrreg_t __user *v_regs = (elf_vrreg_t __user *)(((unsigned long)sc->vmx_reserve + 15) & ~0xful); + elf_vrreg_t __user *v_regs = sigcontext_vmx_regs(sc); #endif unsigned long msr = regs->msr; long err = 0; @@ -181,10 +194,8 @@ static long setup_tm_sigcontexts(struct sigcontext __user *sc, * v_regs pointer or not. */ #ifdef CONFIG_ALTIVEC - elf_vrreg_t __user *v_regs = (elf_vrreg_t __user *) - (((unsigned long)sc->vmx_reserve + 15) & ~0xful); - elf_vrreg_t __user *tm_v_regs = (elf_vrreg_t __user *) - (((unsigned long)tm_sc->vmx_reserve + 15) & ~0xful); + elf_vrreg_t __user *v_regs = sigcontext_vmx_regs(sc); + elf_vrreg_t __user *tm_v_regs = sigcontext_vmx_regs(tm_sc); #endif unsigned long msr = regs->msr; long err = 0; diff --git a/arch/powerpc/kernel/trace_clock.c b/arch/powerpc/kernel/trace_clock.c new file mode 100644 index 00000000000000..49170690946de6 --- /dev/null +++ b/arch/powerpc/kernel/trace_clock.c @@ -0,0 +1,15 @@ +/* + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2, as + * published by the Free Software Foundation. + * + * Copyright (C) 2015 Naveen N. Rao, IBM Corporation + */ + +#include +#include + +u64 notrace trace_clock_ppc_tb(void) +{ + return get_tb(); +} diff --git a/arch/powerpc/lib/checksum_32.S b/arch/powerpc/lib/checksum_32.S index 7874e8a80455c9..6d67e057f15ecd 100644 --- a/arch/powerpc/lib/checksum_32.S +++ b/arch/powerpc/lib/checksum_32.S @@ -40,22 +40,6 @@ _GLOBAL(ip_fast_csum) srwi r3,r3,16 blr -/* - * Compute checksum of TCP or UDP pseudo-header: - * csum_tcpudp_magic(saddr, daddr, len, proto, sum) - */ -_GLOBAL(csum_tcpudp_magic) - rlwimi r5,r6,16,0,15 /* put proto in upper half of len */ - addc r0,r3,r4 /* add 4 32-bit words together */ - adde r0,r0,r5 - adde r0,r0,r7 - addze r0,r0 /* add in final carry */ - rlwinm r3,r0,16,0,31 /* fold two halves together */ - add r3,r0,r3 - not r3,r3 - srwi r3,r3,16 - blr - /* * computes the checksum of a memory block at buff, length len, * and adds in "sum" (32-bit) diff --git a/arch/powerpc/lib/checksum_64.S b/arch/powerpc/lib/checksum_64.S index 57a0720650576a..f3ef35436612b7 100644 --- a/arch/powerpc/lib/checksum_64.S +++ b/arch/powerpc/lib/checksum_64.S @@ -44,27 +44,6 @@ _GLOBAL(ip_fast_csum) srwi r3,r3,16 blr -/* - * Compute checksum of TCP or UDP pseudo-header: - * csum_tcpudp_magic(r3=saddr, r4=daddr, r5=len, r6=proto, r7=sum) - * No real gain trying to do this specially for 64 bit, but - * the 32 bit addition may spill into the upper bits of - * the doubleword so we still must fold it down from 64. - */ -_GLOBAL(csum_tcpudp_magic) - rlwimi r5,r6,16,0,15 /* put proto in upper half of len */ - addc r0,r3,r4 /* add 4 32-bit words together */ - adde r0,r0,r5 - adde r0,r0,r7 - rldicl r4,r0,32,0 /* fold 64 bit value */ - add r0,r4,r0 - srdi r0,r0,32 - rlwinm r3,r0,16,0,31 /* fold two halves together */ - add r3,r0,r3 - not r3,r3 - srwi r3,r3,16 - blr - /* * Computes the checksum of a memory block at buff, length len, * and adds in "sum" (32-bit). diff --git a/arch/powerpc/lib/copy_32.S b/arch/powerpc/lib/copy_32.S index 6813f80d1eec38..2ef50c62947096 100644 --- a/arch/powerpc/lib/copy_32.S +++ b/arch/powerpc/lib/copy_32.S @@ -69,9 +69,15 @@ CACHELINE_BYTES = L1_CACHE_BYTES LG_CACHELINE_BYTES = L1_CACHE_SHIFT CACHELINE_MASK = (L1_CACHE_BYTES-1) +/* + * Use dcbz on the complete cache lines in the destination + * to set them to zero. This requires that the destination + * area is cacheable. -- paulus + */ _GLOBAL(memset) rlwimi r4,r4,8,16,23 rlwimi r4,r4,16,0,15 + addi r6,r3,-4 cmplwi 0,r5,4 blt 7f @@ -80,7 +86,29 @@ _GLOBAL(memset) andi. r0,r6,3 add r5,r0,r5 subf r6,r0,r6 - srwi r0,r5,2 + cmplwi 0,r4,0 + bne 2f /* Use normal procedure if r4 is not zero */ + + clrlwi r7,r6,32-LG_CACHELINE_BYTES + add r8,r7,r5 + srwi r9,r8,LG_CACHELINE_BYTES + addic. r9,r9,-1 /* total number of complete cachelines */ + ble 2f + xori r0,r7,CACHELINE_MASK & ~3 + srwi. r0,r0,2 + beq 3f + mtctr r0 +4: stwu r4,4(r6) + bdnz 4b +3: mtctr r9 + li r7,4 +10: dcbz r7,r6 + addi r6,r6,CACHELINE_BYTES + bdnz 10b + clrlwi r5,r8,32-LG_CACHELINE_BYTES + addi r5,r5,4 + +2: srwi r0,r5,2 mtctr r0 bdz 6f 1: stwu r4,4(r6) @@ -94,12 +122,91 @@ _GLOBAL(memset) bdnz 8b blr +/* + * This version uses dcbz on the complete cache lines in the + * destination area to reduce memory traffic. This requires that + * the destination area is cacheable. + * We only use this version if the source and dest don't overlap. + * -- paulus. + */ _GLOBAL(memmove) cmplw 0,r3,r4 bgt backwards_memcpy /* fall through */ _GLOBAL(memcpy) + add r7,r3,r5 /* test if the src & dst overlap */ + add r8,r4,r5 + cmplw 0,r4,r7 + cmplw 1,r3,r8 + crand 0,0,4 /* cr0.lt &= cr1.lt */ + blt generic_memcpy /* if regions overlap */ + + addi r4,r4,-4 + addi r6,r3,-4 + neg r0,r3 + andi. r0,r0,CACHELINE_MASK /* # bytes to start of cache line */ + beq 58f + + cmplw 0,r5,r0 /* is this more than total to do? */ + blt 63f /* if not much to do */ + andi. r8,r0,3 /* get it word-aligned first */ + subf r5,r0,r5 + mtctr r8 + beq+ 61f +70: lbz r9,4(r4) /* do some bytes */ + addi r4,r4,1 + addi r6,r6,1 + stb r9,3(r6) + bdnz 70b +61: srwi. r0,r0,2 + mtctr r0 + beq 58f +72: lwzu r9,4(r4) /* do some words */ + stwu r9,4(r6) + bdnz 72b + +58: srwi. r0,r5,LG_CACHELINE_BYTES /* # complete cachelines */ + clrlwi r5,r5,32-LG_CACHELINE_BYTES + li r11,4 + mtctr r0 + beq 63f +53: + dcbz r11,r6 + COPY_16_BYTES +#if L1_CACHE_BYTES >= 32 + COPY_16_BYTES +#if L1_CACHE_BYTES >= 64 + COPY_16_BYTES + COPY_16_BYTES +#if L1_CACHE_BYTES >= 128 + COPY_16_BYTES + COPY_16_BYTES + COPY_16_BYTES + COPY_16_BYTES +#endif +#endif +#endif + bdnz 53b + +63: srwi. r0,r5,2 + mtctr r0 + beq 64f +30: lwzu r0,4(r4) + stwu r0,4(r6) + bdnz 30b + +64: andi. r0,r5,3 + mtctr r0 + beq+ 65f + addi r4,r4,3 + addi r6,r6,3 +40: lbzu r0,1(r4) + stbu r0,1(r6) + bdnz 40b +65: blr + +_GLOBAL(generic_memcpy) srwi. r7,r5,3 addi r6,r3,-4 addi r4,r4,-4 diff --git a/arch/powerpc/mm/fsl_booke_mmu.c b/arch/powerpc/mm/fsl_booke_mmu.c index 9c90e66cffb6da..354ba3c09ef3e9 100644 --- a/arch/powerpc/mm/fsl_booke_mmu.c +++ b/arch/powerpc/mm/fsl_booke_mmu.c @@ -112,7 +112,7 @@ static void settlbcam(int index, unsigned long virt, phys_addr_t phys, tsize = __ilog2(size) - 10; -#ifdef CONFIG_SMP +#if defined(CONFIG_SMP) || defined(CONFIG_PPC_E500MC) if ((flags & _PAGE_NO_CACHE) == 0) flags |= _PAGE_COHERENT; #endif diff --git a/arch/powerpc/mm/hash_low_64.S b/arch/powerpc/mm/hash_low_64.S index 463174a4a64755..3b49e329590197 100644 --- a/arch/powerpc/mm/hash_low_64.S +++ b/arch/powerpc/mm/hash_low_64.S @@ -701,7 +701,7 @@ htab_pte_insert_failure: #endif /* CONFIG_PPC_64K_PAGES */ -#ifdef CONFIG_PPC_HAS_HASH_64K +#ifdef CONFIG_PPC_64K_PAGES /***************************************************************************** * * @@ -993,7 +993,7 @@ ht64_pte_insert_failure: b ht64_bail -#endif /* CONFIG_PPC_HAS_HASH_64K */ +#endif /* CONFIG_PPC_64K_PAGES */ /***************************************************************************** diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c index 5ec987f65b2c95..aee70171355b9b 100644 --- a/arch/powerpc/mm/hash_utils_64.c +++ b/arch/powerpc/mm/hash_utils_64.c @@ -640,7 +640,7 @@ extern u32 ht64_call_hpte_updatepp[]; static void __init htab_finish_init(void) { -#ifdef CONFIG_PPC_HAS_HASH_64K +#ifdef CONFIG_PPC_64K_PAGES patch_branch(ht64_call_hpte_insert1, ppc_function_entry(ppc_md.hpte_insert), BRANCH_SET_LINK); @@ -653,7 +653,7 @@ static void __init htab_finish_init(void) patch_branch(ht64_call_hpte_updatepp, ppc_function_entry(ppc_md.hpte_updatepp), BRANCH_SET_LINK); -#endif /* CONFIG_PPC_HAS_HASH_64K */ +#endif /* CONFIG_PPC_64K_PAGES */ patch_branch(htab_call_hpte_insert1, ppc_function_entry(ppc_md.hpte_insert), @@ -1151,12 +1151,12 @@ int hash_page_mm(struct mm_struct *mm, unsigned long ea, check_paca_psize(ea, mm, psize, user_region); #endif /* CONFIG_PPC_64K_PAGES */ -#ifdef CONFIG_PPC_HAS_HASH_64K +#ifdef CONFIG_PPC_64K_PAGES if (psize == MMU_PAGE_64K) rc = __hash_page_64K(ea, access, vsid, ptep, trap, flags, ssize); else -#endif /* CONFIG_PPC_HAS_HASH_64K */ +#endif /* CONFIG_PPC_64K_PAGES */ { int spp = subpage_protection(mm, ea); if (access & spp) @@ -1264,12 +1264,12 @@ void hash_preload(struct mm_struct *mm, unsigned long ea, update_flags |= HPTE_LOCAL_UPDATE; /* Hash it in */ -#ifdef CONFIG_PPC_HAS_HASH_64K +#ifdef CONFIG_PPC_64K_PAGES if (mm->context.user_psize == MMU_PAGE_64K) rc = __hash_page_64K(ea, access, vsid, ptep, trap, update_flags, ssize); else -#endif /* CONFIG_PPC_HAS_HASH_64K */ +#endif /* CONFIG_PPC_64K_PAGES */ rc = __hash_page_4K(ea, access, vsid, ptep, trap, update_flags, ssize, subpage_protection(mm, ea)); diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c index bb0bd7025cb88f..06c14523b787a4 100644 --- a/arch/powerpc/mm/hugetlbpage.c +++ b/arch/powerpc/mm/hugetlbpage.c @@ -808,14 +808,6 @@ static int __init add_huge_page_size(unsigned long long size) if ((mmu_psize = shift_to_mmu_psize(shift)) < 0) return -EINVAL; -#ifdef CONFIG_SPU_FS_64K_LS - /* Disable support for 64K huge pages when 64K SPU local store - * support is enabled as the current implementation conflicts. - */ - if (shift == PAGE_SHIFT_64K) - return -EINVAL; -#endif /* CONFIG_SPU_FS_64K_LS */ - BUG_ON(mmu_psize_defs[mmu_psize].shift != shift); /* Return if huge page size has already been setup */ diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index 0f11819d8f1dc0..e1fe333da94680 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -414,17 +414,17 @@ void flush_dcache_icache_page(struct page *page) return; } #endif -#ifdef CONFIG_BOOKE - { +#if defined(CONFIG_8xx) || defined(CONFIG_PPC64) + /* On 8xx there is no need to kmap since highmem is not supported */ + __flush_dcache_icache(page_address(page)); +#else + if (IS_ENABLED(CONFIG_BOOKE) || sizeof(phys_addr_t) > sizeof(void *)) { void *start = kmap_atomic(page); __flush_dcache_icache(start); kunmap_atomic(start); + } else { + __flush_dcache_icache_phys(page_to_pfn(page) << PAGE_SHIFT); } -#elif defined(CONFIG_8xx) || defined(CONFIG_PPC64) - /* On 8xx there is no need to kmap since highmem is not supported */ - __flush_dcache_icache(page_address(page)); -#else - __flush_dcache_icache_phys(page_to_pfn(page) << PAGE_SHIFT); #endif } EXPORT_SYMBOL(flush_dcache_icache_page); diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index 5e80621d9324c7..8b9502adaf794f 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -225,7 +225,7 @@ static void initialize_distance_lookup_table(int nid, for (i = 0; i < distance_ref_points_depth; i++) { const __be32 *entry; - entry = &associativity[be32_to_cpu(distance_ref_points[i])]; + entry = &associativity[be32_to_cpu(distance_ref_points[i]) - 1]; distance_lookup_table[nid][i] = of_read_number(entry, 1); } } @@ -248,8 +248,12 @@ static int associativity_to_nid(const __be32 *associativity) nid = -1; if (nid > 0 && - of_read_number(associativity, 1) >= distance_ref_points_depth) - initialize_distance_lookup_table(nid, associativity); + of_read_number(associativity, 1) >= distance_ref_points_depth) { + /* + * Skip the length field and send start of associativity array + */ + initialize_distance_lookup_table(nid, associativity + 1); + } out: return nid; @@ -507,6 +511,12 @@ static int of_drconf_to_nid_single(struct of_drconf_cell *drmem, if (nid == 0xffff || nid >= MAX_NUMNODES) nid = default_nid; + + if (nid > 0) { + index = drmem->aa_index * aa->array_sz; + initialize_distance_lookup_table(nid, + &aa->arrays[index]); + } } return nid; diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c index 876232d6412688..e92cb2146b1862 100644 --- a/arch/powerpc/mm/pgtable_64.c +++ b/arch/powerpc/mm/pgtable_64.c @@ -149,17 +149,7 @@ int map_kernel_page(unsigned long ea, unsigned long pa, int flags) #endif /* !CONFIG_PPC_MMU_NOHASH */ } -#ifdef CONFIG_PPC_BOOK3E_64 - /* - * With hardware tablewalk, a sync is needed to ensure that - * subsequent accesses see the PTE we just wrote. Unlike userspace - * mappings, we can't tolerate spurious faults, so make sure - * the new PTE will be seen the first time. - */ - mb(); -#else smp_wmb(); -#endif return 0; } diff --git a/arch/powerpc/mm/slb.c b/arch/powerpc/mm/slb.c index 6e450ca6652684..8a32a2be3c5339 100644 --- a/arch/powerpc/mm/slb.c +++ b/arch/powerpc/mm/slb.c @@ -41,9 +41,9 @@ static void slb_allocate(unsigned long ea) (((ssize) == MMU_SEGSIZE_256M)? ESID_MASK: ESID_MASK_1T) static inline unsigned long mk_esid_data(unsigned long ea, int ssize, - unsigned long slot) + unsigned long entry) { - return (ea & slb_esid_mask(ssize)) | SLB_ESID_V | slot; + return (ea & slb_esid_mask(ssize)) | SLB_ESID_V | entry; } static inline unsigned long mk_vsid_data(unsigned long ea, int ssize, @@ -249,11 +249,24 @@ void switch_slb(struct task_struct *tsk, struct mm_struct *mm) static inline void patch_slb_encoding(unsigned int *insn_addr, unsigned int immed) { - int insn = (*insn_addr & 0xffff0000) | immed; + + /* + * This function patches either an li or a cmpldi instruction with + * a new immediate value. This relies on the fact that both li + * (which is actually addi) and cmpldi both take a 16-bit immediate + * value, and it is situated in the same location in the instruction, + * ie. bits 16-31 (Big endian bit order) or the lower 16 bits. + * The signedness of the immediate operand differs between the two + * instructions however this code is only ever patching a small value, + * much less than 1 << 15, so we can get away with it. + * To patch the value we read the existing instruction, clear the + * immediate value, and or in our new value, then write the instruction + * back. + */ + unsigned int insn = (*insn_addr & 0xffff0000) | immed; patch_instruction(insn_addr, insn); } -extern u32 slb_compare_rr_to_size[]; extern u32 slb_miss_kernel_load_linear[]; extern u32 slb_miss_kernel_load_io[]; extern u32 slb_compare_rr_to_size[]; @@ -309,12 +322,11 @@ void slb_initialize(void) lflags = SLB_VSID_KERNEL | linear_llp; vflags = SLB_VSID_KERNEL | vmalloc_llp; - /* Invalidate the entire SLB (even slot 0) & all the ERATS */ + /* Invalidate the entire SLB (even entry 0) & all the ERATS */ asm volatile("isync":::"memory"); asm volatile("slbmte %0,%0"::"r" (0) : "memory"); asm volatile("isync; slbia; isync":::"memory"); create_shadowed_slbe(PAGE_OFFSET, mmu_kernel_ssize, lflags, 0); - create_shadowed_slbe(VMALLOC_START, mmu_kernel_ssize, vflags, 1); /* For the boot cpu, we're running on the stack in init_thread_union, diff --git a/arch/powerpc/mm/tlb_low_64e.S b/arch/powerpc/mm/tlb_low_64e.S index 765b419883f22e..e4185581c5a7d3 100644 --- a/arch/powerpc/mm/tlb_low_64e.S +++ b/arch/powerpc/mm/tlb_low_64e.S @@ -308,11 +308,11 @@ BEGIN_FTR_SECTION /* CPU_FTR_SMT */ * * MAS6:IND should be already set based on MAS4 */ -1: lbarx r15,0,r11 lhz r10,PACAPACAINDEX(r13) - cmpdi r15,0 - cmpdi cr1,r15,1 /* set cr1.eq = 0 for non-recursive */ addi r10,r10,1 + crclr cr1*4+eq /* set cr1.eq = 0 for non-recursive */ +1: lbarx r15,0,r11 + cmpdi r15,0 bne 2f stbcx. r10,0,r11 bne 1b @@ -320,9 +320,9 @@ BEGIN_FTR_SECTION /* CPU_FTR_SMT */ .subsection 1 2: cmpd cr1,r15,r10 /* recursive lock due to mcheck/crit/etc? */ beq cr1,3b /* unlock will happen if cr1.eq = 0 */ - lbz r15,0(r11) +10: lbz r15,0(r11) cmpdi r15,0 - bne 2b + bne 10b b 1b .previous diff --git a/arch/powerpc/oprofile/op_model_power4.c b/arch/powerpc/oprofile/op_model_power4.c index 962fe7b3e3fb19..4b32e9404bbef6 100644 --- a/arch/powerpc/oprofile/op_model_power4.c +++ b/arch/powerpc/oprofile/op_model_power4.c @@ -207,7 +207,7 @@ static int power4_start(struct op_counter_config *ctr) unsigned int mmcr0; /* set the PMM bit (see comment below) */ - mtmsrd(mfmsr() | MSR_PMM); + mtmsr(mfmsr() | MSR_PMM); for (i = 0; i < cur_cpu_spec->num_pmcs; ++i) { if (ctr[i].enabled) { @@ -377,7 +377,7 @@ static void power4_handle_interrupt(struct pt_regs *regs, is_kernel = get_kernel(pc, mmcra); /* set the PMM bit (see comment below) */ - mtmsrd(mfmsr() | MSR_PMM); + mtmsr(mfmsr() | MSR_PMM); /* Check that the SIAR valid bit in MMCRA is set to 1. */ if ((mmcra & MMCRA_SIAR_VALID_MASK) == MMCRA_SIAR_VALID_MASK) diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c index d90893b76e7ceb..b0382f3f109558 100644 --- a/arch/powerpc/perf/core-book3s.c +++ b/arch/powerpc/perf/core-book3s.c @@ -53,7 +53,7 @@ struct cpu_hw_events { /* BHRB bits */ u64 bhrb_filter; /* BHRB HW branch filter */ - int bhrb_users; + unsigned int bhrb_users; void *bhrb_context; struct perf_branch_stack bhrb_stack; struct perf_branch_entry bhrb_entries[BHRB_MAX_ENTRIES]; @@ -369,8 +369,8 @@ static void power_pmu_bhrb_disable(struct perf_event *event) if (!ppmu->bhrb_nr) return; + WARN_ON_ONCE(!cpuhw->bhrb_users); cpuhw->bhrb_users--; - WARN_ON_ONCE(cpuhw->bhrb_users < 0); perf_sched_cb_dec(event->ctx->pmu); if (!cpuhw->disabled && !cpuhw->bhrb_users) { diff --git a/arch/powerpc/perf/hv-24x7.c b/arch/powerpc/perf/hv-24x7.c index df956295c2a7ab..527c8b98e97ee5 100644 --- a/arch/powerpc/perf/hv-24x7.c +++ b/arch/powerpc/perf/hv-24x7.c @@ -416,7 +416,7 @@ static struct attribute *event_to_attr(unsigned ix, } static struct attribute *event_to_desc_attr(struct hv_24x7_event_data *event, - int nonce) + int nonce) { int nl, dl; char *name = event_name(event, &nl); @@ -444,7 +444,7 @@ event_to_long_desc_attr(struct hv_24x7_event_data *event, int nonce) } static ssize_t event_data_to_attrs(unsigned ix, struct attribute **attrs, - struct hv_24x7_event_data *event, int nonce) + struct hv_24x7_event_data *event, int nonce) { unsigned i; @@ -512,7 +512,7 @@ static int memord(const void *d1, size_t s1, const void *d2, size_t s2) } static int ev_uniq_ord(const void *v1, size_t s1, unsigned d1, const void *v2, - size_t s2, unsigned d2) + size_t s2, unsigned d2) { int r = memord(v1, s1, v2, s2); @@ -526,7 +526,7 @@ static int ev_uniq_ord(const void *v1, size_t s1, unsigned d1, const void *v2, } static int event_uniq_add(struct rb_root *root, const char *name, int nl, - unsigned domain) + unsigned domain) { struct rb_node **new = &(root->rb_node), *parent = NULL; struct event_uniq *data; @@ -650,8 +650,8 @@ static ssize_t catalog_event_len_validate(struct hv_24x7_event_data *event, #define MAX_4K (SIZE_MAX / 4096) static int create_events_from_catalog(struct attribute ***events_, - struct attribute ***event_descs_, - struct attribute ***event_long_descs_) + struct attribute ***event_descs_, + struct attribute ***event_long_descs_) { unsigned long hret; size_t catalog_len, catalog_page_len, event_entry_count, @@ -1008,8 +1008,8 @@ static const struct attribute_group *attr_groups[] = { }; static void log_24x7_hcall(struct hv_24x7_request_buffer *request_buffer, - struct hv_24x7_data_result_buffer *result_buffer, - unsigned long ret) + struct hv_24x7_data_result_buffer *result_buffer, + unsigned long ret) { struct hv_24x7_request *req; @@ -1026,7 +1026,7 @@ static void log_24x7_hcall(struct hv_24x7_request_buffer *request_buffer, * Start the process for a new H_GET_24x7_DATA hcall. */ static void init_24x7_request(struct hv_24x7_request_buffer *request_buffer, - struct hv_24x7_data_result_buffer *result_buffer) + struct hv_24x7_data_result_buffer *result_buffer) { memset(request_buffer, 0, 4096); @@ -1041,7 +1041,7 @@ static void init_24x7_request(struct hv_24x7_request_buffer *request_buffer, * by 'init_24x7_request()' and 'add_event_to_24x7_request()'. */ static int make_24x7_request(struct hv_24x7_request_buffer *request_buffer, - struct hv_24x7_data_result_buffer *result_buffer) + struct hv_24x7_data_result_buffer *result_buffer) { unsigned long ret; @@ -1104,7 +1104,6 @@ static unsigned long single_24x7_request(struct perf_event *event, u64 *count) unsigned long ret; struct hv_24x7_request_buffer *request_buffer; struct hv_24x7_data_result_buffer *result_buffer; - struct hv_24x7_result *resb; BUILD_BUG_ON(sizeof(*request_buffer) > 4096); BUILD_BUG_ON(sizeof(*result_buffer) > 4096); @@ -1125,8 +1124,7 @@ static unsigned long single_24x7_request(struct perf_event *event, u64 *count) } /* process result from hcall */ - resb = &result_buffer->results[0]; - *count = be64_to_cpu(resb->elements[0].element_data[0]); + *count = be64_to_cpu(result_buffer->results[0].elements[0].element_data[0]); out: put_cpu_var(hv_24x7_reqb); diff --git a/arch/powerpc/platforms/512x/Kconfig b/arch/powerpc/platforms/512x/Kconfig index 5aa3f4b5332c57..48bf38d0de3591 100644 --- a/arch/powerpc/platforms/512x/Kconfig +++ b/arch/powerpc/platforms/512x/Kconfig @@ -7,8 +7,8 @@ config PPC_MPC512x select PPC_PCI_CHOICE select FSL_PCI if PCI select ARCH_WANT_OPTIONAL_GPIOLIB - select USB_EHCI_BIG_ENDIAN_MMIO - select USB_EHCI_BIG_ENDIAN_DESC + select USB_EHCI_BIG_ENDIAN_MMIO if USB_EHCI_HCD + select USB_EHCI_BIG_ENDIAN_DESC if USB_EHCI_HCD config MPC5121_ADS bool "Freescale MPC5121E ADS" diff --git a/arch/powerpc/platforms/85xx/c293pcie.c b/arch/powerpc/platforms/85xx/c293pcie.c index 84476b646005aa..61bc851e9a8e8c 100644 --- a/arch/powerpc/platforms/85xx/c293pcie.c +++ b/arch/powerpc/platforms/85xx/c293pcie.c @@ -66,10 +66,6 @@ define_machine(c293_pcie) { .probe = c293_pcie_probe, .setup_arch = c293_pcie_setup_arch, .init_IRQ = c293_pcie_pic_init, -#ifdef CONFIG_PCI - .pcibios_fixup_bus = fsl_pcibios_fixup_bus, - .pcibios_fixup_phb = fsl_pcibios_fixup_phb, -#endif .get_irq = mpic_get_irq, .restart = fsl_rstcr_restart, .calibrate_decr = generic_calibrate_decr, diff --git a/arch/powerpc/platforms/85xx/corenet_generic.c b/arch/powerpc/platforms/85xx/corenet_generic.c index bd839dc287fe61..b39557120cbb06 100644 --- a/arch/powerpc/platforms/85xx/corenet_generic.c +++ b/arch/powerpc/platforms/85xx/corenet_generic.c @@ -153,6 +153,8 @@ static const char * const boards[] __initconst = { "fsl,T1023RDB", "fsl,T1024QDS", "fsl,T1024RDB", + "fsl,T1040D4RDB", + "fsl,T1042D4RDB", "fsl,T1040QDS", "fsl,T1042QDS", "fsl,T1040RDB", diff --git a/arch/powerpc/platforms/cell/Kconfig b/arch/powerpc/platforms/cell/Kconfig index 2f23133ab3d10c..b0ac1773cea698 100644 --- a/arch/powerpc/platforms/cell/Kconfig +++ b/arch/powerpc/platforms/cell/Kconfig @@ -57,21 +57,6 @@ config SPU_FS Units on machines implementing the Broadband Processor Architecture. -config SPU_FS_64K_LS - bool "Use 64K pages to map SPE local store" - # we depend on PPC_MM_SLICES for now rather than selecting - # it because we depend on hugetlbfs hooks being present. We - # will fix that when the generic code has been improved to - # not require hijacking hugetlbfs hooks. - depends on SPU_FS && PPC_MM_SLICES && !PPC_64K_PAGES - default y - select PPC_HAS_HASH_64K - help - This option causes SPE local stores to be mapped in process - address spaces using 64K pages while the rest of the kernel - uses 4K pages. This can improve performances of applications - using multiple SPEs by lowering the TLB pressure on them. - config SPU_BASE bool default n diff --git a/arch/powerpc/platforms/cell/spufs/file.c b/arch/powerpc/platforms/cell/spufs/file.c index d966bbe58b8f36..5038fd578e65ac 100644 --- a/arch/powerpc/platforms/cell/spufs/file.c +++ b/arch/powerpc/platforms/cell/spufs/file.c @@ -239,23 +239,6 @@ spufs_mem_mmap_fault(struct vm_area_struct *vma, struct vm_fault *vmf) unsigned long address = (unsigned long)vmf->virtual_address; unsigned long pfn, offset; -#ifdef CONFIG_SPU_FS_64K_LS - struct spu_state *csa = &ctx->csa; - int psize; - - /* Check what page size we are using */ - psize = get_slice_psize(vma->vm_mm, address); - - /* Some sanity checking */ - BUG_ON(csa->use_big_pages != (psize == MMU_PAGE_64K)); - - /* Wow, 64K, cool, we need to align the address though */ - if (csa->use_big_pages) { - BUG_ON(vma->vm_start & 0xffff); - address &= ~0xfffful; - } -#endif /* CONFIG_SPU_FS_64K_LS */ - offset = vmf->pgoff << PAGE_SHIFT; if (offset >= LS_SIZE) return VM_FAULT_SIGBUS; @@ -310,22 +293,6 @@ static const struct vm_operations_struct spufs_mem_mmap_vmops = { static int spufs_mem_mmap(struct file *file, struct vm_area_struct *vma) { -#ifdef CONFIG_SPU_FS_64K_LS - struct spu_context *ctx = file->private_data; - struct spu_state *csa = &ctx->csa; - - /* Sanity check VMA alignment */ - if (csa->use_big_pages) { - pr_debug("spufs_mem_mmap 64K, start=0x%lx, end=0x%lx," - " pgoff=0x%lx\n", vma->vm_start, vma->vm_end, - vma->vm_pgoff); - if (vma->vm_start & 0xffff) - return -EINVAL; - if (vma->vm_pgoff & 0xf) - return -EINVAL; - } -#endif /* CONFIG_SPU_FS_64K_LS */ - if (!(vma->vm_flags & VM_SHARED)) return -EINVAL; @@ -336,25 +303,6 @@ static int spufs_mem_mmap(struct file *file, struct vm_area_struct *vma) return 0; } -#ifdef CONFIG_SPU_FS_64K_LS -static unsigned long spufs_get_unmapped_area(struct file *file, - unsigned long addr, unsigned long len, unsigned long pgoff, - unsigned long flags) -{ - struct spu_context *ctx = file->private_data; - struct spu_state *csa = &ctx->csa; - - /* If not using big pages, fallback to normal MM g_u_a */ - if (!csa->use_big_pages) - return current->mm->get_unmapped_area(file, addr, len, - pgoff, flags); - - /* Else, try to obtain a 64K pages slice */ - return slice_get_unmapped_area(addr, len, flags, - MMU_PAGE_64K, 1); -} -#endif /* CONFIG_SPU_FS_64K_LS */ - static const struct file_operations spufs_mem_fops = { .open = spufs_mem_open, .release = spufs_mem_release, @@ -362,9 +310,6 @@ static const struct file_operations spufs_mem_fops = { .write = spufs_mem_write, .llseek = generic_file_llseek, .mmap = spufs_mem_mmap, -#ifdef CONFIG_SPU_FS_64K_LS - .get_unmapped_area = spufs_get_unmapped_area, -#endif }; static int spufs_ps_fault(struct vm_area_struct *vma, diff --git a/arch/powerpc/platforms/cell/spufs/lscsa_alloc.c b/arch/powerpc/platforms/cell/spufs/lscsa_alloc.c index 147069938cfea9..b847e94035664d 100644 --- a/arch/powerpc/platforms/cell/spufs/lscsa_alloc.c +++ b/arch/powerpc/platforms/cell/spufs/lscsa_alloc.c @@ -31,7 +31,7 @@ #include "spufs.h" -static int spu_alloc_lscsa_std(struct spu_state *csa) +int spu_alloc_lscsa(struct spu_state *csa) { struct spu_lscsa *lscsa; unsigned char *p; @@ -48,7 +48,7 @@ static int spu_alloc_lscsa_std(struct spu_state *csa) return 0; } -static void spu_free_lscsa_std(struct spu_state *csa) +void spu_free_lscsa(struct spu_state *csa) { /* Clear reserved bit before vfree. */ unsigned char *p; @@ -61,123 +61,3 @@ static void spu_free_lscsa_std(struct spu_state *csa) vfree(csa->lscsa); } - -#ifdef CONFIG_SPU_FS_64K_LS - -#define SPU_64K_PAGE_SHIFT 16 -#define SPU_64K_PAGE_ORDER (SPU_64K_PAGE_SHIFT - PAGE_SHIFT) -#define SPU_64K_PAGE_COUNT (1ul << SPU_64K_PAGE_ORDER) - -int spu_alloc_lscsa(struct spu_state *csa) -{ - struct page **pgarray; - unsigned char *p; - int i, j, n_4k; - - /* Check availability of 64K pages */ - if (!spu_64k_pages_available()) - goto fail; - - csa->use_big_pages = 1; - - pr_debug("spu_alloc_lscsa(csa=0x%p), trying to allocate 64K pages\n", - csa); - - /* First try to allocate our 64K pages. We need 5 of them - * with the current implementation. In the future, we should try - * to separate the lscsa with the actual local store image, thus - * allowing us to require only 4 64K pages per context - */ - for (i = 0; i < SPU_LSCSA_NUM_BIG_PAGES; i++) { - /* XXX This is likely to fail, we should use a special pool - * similar to what hugetlbfs does. - */ - csa->lscsa_pages[i] = alloc_pages(GFP_KERNEL, - SPU_64K_PAGE_ORDER); - if (csa->lscsa_pages[i] == NULL) - goto fail; - } - - pr_debug(" success ! creating vmap...\n"); - - /* Now we need to create a vmalloc mapping of these for the kernel - * and SPU context switch code to use. Currently, we stick to a - * normal kernel vmalloc mapping, which in our case will be 4K - */ - n_4k = SPU_64K_PAGE_COUNT * SPU_LSCSA_NUM_BIG_PAGES; - pgarray = kmalloc(sizeof(struct page *) * n_4k, GFP_KERNEL); - if (pgarray == NULL) - goto fail; - for (i = 0; i < SPU_LSCSA_NUM_BIG_PAGES; i++) - for (j = 0; j < SPU_64K_PAGE_COUNT; j++) - /* We assume all the struct page's are contiguous - * which should be hopefully the case for an order 4 - * allocation.. - */ - pgarray[i * SPU_64K_PAGE_COUNT + j] = - csa->lscsa_pages[i] + j; - csa->lscsa = vmap(pgarray, n_4k, VM_USERMAP, PAGE_KERNEL); - kfree(pgarray); - if (csa->lscsa == NULL) - goto fail; - - memset(csa->lscsa, 0, sizeof(struct spu_lscsa)); - - /* Set LS pages reserved to allow for user-space mapping. - * - * XXX isn't that a bit obsolete ? I think we should just - * make sure the page count is high enough. Anyway, won't harm - * for now - */ - for (p = csa->lscsa->ls; p < csa->lscsa->ls + LS_SIZE; p += PAGE_SIZE) - SetPageReserved(vmalloc_to_page(p)); - - pr_debug(" all good !\n"); - - return 0; -fail: - pr_debug("spufs: failed to allocate lscsa 64K pages, falling back\n"); - spu_free_lscsa(csa); - return spu_alloc_lscsa_std(csa); -} - -void spu_free_lscsa(struct spu_state *csa) -{ - unsigned char *p; - int i; - - if (!csa->use_big_pages) { - spu_free_lscsa_std(csa); - return; - } - csa->use_big_pages = 0; - - if (csa->lscsa == NULL) - goto free_pages; - - for (p = csa->lscsa->ls; p < csa->lscsa->ls + LS_SIZE; p += PAGE_SIZE) - ClearPageReserved(vmalloc_to_page(p)); - - vunmap(csa->lscsa); - csa->lscsa = NULL; - - free_pages: - - for (i = 0; i < SPU_LSCSA_NUM_BIG_PAGES; i++) - if (csa->lscsa_pages[i]) - __free_pages(csa->lscsa_pages[i], SPU_64K_PAGE_ORDER); -} - -#else /* CONFIG_SPU_FS_64K_LS */ - -int spu_alloc_lscsa(struct spu_state *csa) -{ - return spu_alloc_lscsa_std(csa); -} - -void spu_free_lscsa(struct spu_state *csa) -{ - spu_free_lscsa_std(csa); -} - -#endif /* !defined(CONFIG_SPU_FS_64K_LS) */ diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c b/arch/powerpc/platforms/powernv/eeh-powernv.c index 7cf0df859d0536..3bb6acb763394e 100644 --- a/arch/powerpc/platforms/powernv/eeh-powernv.c +++ b/arch/powerpc/platforms/powernv/eeh-powernv.c @@ -1394,11 +1394,19 @@ static int pnv_eeh_next_error(struct eeh_pe **pe) */ if (pnv_eeh_get_pe(hose, be64_to_cpu(frozen_pe_no), pe)) { - /* Try best to clear it */ pr_info("EEH: Clear non-existing PHB#%x-PE#%llx\n", - hose->global_number, frozen_pe_no); + hose->global_number, be64_to_cpu(frozen_pe_no)); pr_info("EEH: PHB location: %s\n", eeh_pe_loc_get(phb_pe)); + + /* Dump PHB diag-data */ + rc = opal_pci_get_phb_diag_data2(phb->opal_id, + phb->diag.blob, PNV_PCI_DIAG_BUF_SIZE); + if (rc == OPAL_SUCCESS) + pnv_pci_dump_phb_diag_data(hose, + phb->diag.blob); + + /* Try best to clear it */ opal_pci_eeh_freeze_clear(phb->opal_id, frozen_pe_no, OPAL_EEH_ACTION_CLEAR_FREEZE_ALL); diff --git a/arch/powerpc/platforms/powernv/opal-hmi.c b/arch/powerpc/platforms/powernv/opal-hmi.c index a8f49d380449bf..d000f4e219814e 100644 --- a/arch/powerpc/platforms/powernv/opal-hmi.c +++ b/arch/powerpc/platforms/powernv/opal-hmi.c @@ -35,9 +35,134 @@ struct OpalHmiEvtNode { struct list_head list; struct OpalHMIEvent hmi_evt; }; + +struct xstop_reason { + uint32_t xstop_reason; + const char *unit_failed; + const char *description; +}; + static LIST_HEAD(opal_hmi_evt_list); static DEFINE_SPINLOCK(opal_hmi_evt_lock); +static void print_core_checkstop_reason(const char *level, + struct OpalHMIEvent *hmi_evt) +{ + int i; + static const struct xstop_reason xstop_reason[] = { + { CORE_CHECKSTOP_IFU_REGFILE, "IFU", + "RegFile core check stop" }, + { CORE_CHECKSTOP_IFU_LOGIC, "IFU", "Logic core check stop" }, + { CORE_CHECKSTOP_PC_DURING_RECOV, "PC", + "Core checkstop during recovery" }, + { CORE_CHECKSTOP_ISU_REGFILE, "ISU", + "RegFile core check stop (mapper error)" }, + { CORE_CHECKSTOP_ISU_LOGIC, "ISU", "Logic core check stop" }, + { CORE_CHECKSTOP_FXU_LOGIC, "FXU", "Logic core check stop" }, + { CORE_CHECKSTOP_VSU_LOGIC, "VSU", "Logic core check stop" }, + { CORE_CHECKSTOP_PC_RECOV_IN_MAINT_MODE, "PC", + "Recovery in maintenance mode" }, + { CORE_CHECKSTOP_LSU_REGFILE, "LSU", + "RegFile core check stop" }, + { CORE_CHECKSTOP_PC_FWD_PROGRESS, "PC", + "Forward Progress Error" }, + { CORE_CHECKSTOP_LSU_LOGIC, "LSU", "Logic core check stop" }, + { CORE_CHECKSTOP_PC_LOGIC, "PC", "Logic core check stop" }, + { CORE_CHECKSTOP_PC_HYP_RESOURCE, "PC", + "Hypervisor Resource error - core check stop" }, + { CORE_CHECKSTOP_PC_HANG_RECOV_FAILED, "PC", + "Hang Recovery Failed (core check stop)" }, + { CORE_CHECKSTOP_PC_AMBI_HANG_DETECTED, "PC", + "Ambiguous Hang Detected (unknown source)" }, + { CORE_CHECKSTOP_PC_DEBUG_TRIG_ERR_INJ, "PC", + "Debug Trigger Error inject" }, + { CORE_CHECKSTOP_PC_SPRD_HYP_ERR_INJ, "PC", + "Hypervisor check stop via SPRC/SPRD" }, + }; + + /* Validity check */ + if (!hmi_evt->u.xstop_error.xstop_reason) { + printk("%s Unknown Core check stop.\n", level); + return; + } + + printk("%s CPU PIR: %08x\n", level, + be32_to_cpu(hmi_evt->u.xstop_error.u.pir)); + for (i = 0; i < ARRAY_SIZE(xstop_reason); i++) + if (be32_to_cpu(hmi_evt->u.xstop_error.xstop_reason) & + xstop_reason[i].xstop_reason) + printk("%s [Unit: %-3s] %s\n", level, + xstop_reason[i].unit_failed, + xstop_reason[i].description); +} + +static void print_nx_checkstop_reason(const char *level, + struct OpalHMIEvent *hmi_evt) +{ + int i; + static const struct xstop_reason xstop_reason[] = { + { NX_CHECKSTOP_SHM_INVAL_STATE_ERR, "DMA & Engine", + "SHM invalid state error" }, + { NX_CHECKSTOP_DMA_INVAL_STATE_ERR_1, "DMA & Engine", + "DMA invalid state error bit 15" }, + { NX_CHECKSTOP_DMA_INVAL_STATE_ERR_2, "DMA & Engine", + "DMA invalid state error bit 16" }, + { NX_CHECKSTOP_DMA_CH0_INVAL_STATE_ERR, "DMA & Engine", + "Channel 0 invalid state error" }, + { NX_CHECKSTOP_DMA_CH1_INVAL_STATE_ERR, "DMA & Engine", + "Channel 1 invalid state error" }, + { NX_CHECKSTOP_DMA_CH2_INVAL_STATE_ERR, "DMA & Engine", + "Channel 2 invalid state error" }, + { NX_CHECKSTOP_DMA_CH3_INVAL_STATE_ERR, "DMA & Engine", + "Channel 3 invalid state error" }, + { NX_CHECKSTOP_DMA_CH4_INVAL_STATE_ERR, "DMA & Engine", + "Channel 4 invalid state error" }, + { NX_CHECKSTOP_DMA_CH5_INVAL_STATE_ERR, "DMA & Engine", + "Channel 5 invalid state error" }, + { NX_CHECKSTOP_DMA_CH6_INVAL_STATE_ERR, "DMA & Engine", + "Channel 6 invalid state error" }, + { NX_CHECKSTOP_DMA_CH7_INVAL_STATE_ERR, "DMA & Engine", + "Channel 7 invalid state error" }, + { NX_CHECKSTOP_DMA_CRB_UE, "DMA & Engine", + "UE error on CRB(CSB address, CCB)" }, + { NX_CHECKSTOP_DMA_CRB_SUE, "DMA & Engine", + "SUE error on CRB(CSB address, CCB)" }, + { NX_CHECKSTOP_PBI_ISN_UE, "PowerBus Interface", + "CRB Kill ISN received while holding ISN with UE error" }, + }; + + /* Validity check */ + if (!hmi_evt->u.xstop_error.xstop_reason) { + printk("%s Unknown NX check stop.\n", level); + return; + } + + printk("%s NX checkstop on CHIP ID: %x\n", level, + be32_to_cpu(hmi_evt->u.xstop_error.u.chip_id)); + for (i = 0; i < ARRAY_SIZE(xstop_reason); i++) + if (be32_to_cpu(hmi_evt->u.xstop_error.xstop_reason) & + xstop_reason[i].xstop_reason) + printk("%s [Unit: %-3s] %s\n", level, + xstop_reason[i].unit_failed, + xstop_reason[i].description); +} + +static void print_checkstop_reason(const char *level, + struct OpalHMIEvent *hmi_evt) +{ + switch (hmi_evt->u.xstop_error.xstop_type) { + case CHECKSTOP_TYPE_CORE: + print_core_checkstop_reason(level, hmi_evt); + break; + case CHECKSTOP_TYPE_NX: + print_nx_checkstop_reason(level, hmi_evt); + break; + case CHECKSTOP_TYPE_UNKNOWN: + printk("%s Unknown Malfunction Alert.\n", level); + break; + } +} + static void print_hmi_event_info(struct OpalHMIEvent *hmi_evt) { const char *level, *sevstr, *error_info; @@ -95,6 +220,13 @@ static void print_hmi_event_info(struct OpalHMIEvent *hmi_evt) (hmi_evt->type == OpalHMI_ERROR_TFMR_PARITY)) printk("%s TFMR: %016llx\n", level, be64_to_cpu(hmi_evt->tfmr)); + + if (hmi_evt->version < OpalHMIEvt_V2) + return; + + /* OpalHMIEvt_V2 and above provides reason for malfunction alert. */ + if (hmi_evt->type == OpalHMI_ERROR_MALFUNC_ALERT) + print_checkstop_reason(level, hmi_evt); } static void hmi_event_handler(struct work_struct *work) @@ -103,6 +235,8 @@ static void hmi_event_handler(struct work_struct *work) struct OpalHMIEvent *hmi_evt; struct OpalHmiEvtNode *msg_node; uint8_t disposition; + struct opal_msg msg; + int unrecoverable = 0; spin_lock_irqsave(&opal_hmi_evt_lock, flags); while (!list_empty(&opal_hmi_evt_list)) { @@ -118,14 +252,53 @@ static void hmi_event_handler(struct work_struct *work) /* * Check if HMI event has been recovered or not. If not - * then we can't continue, invoke panic. + * then kernel can't continue, we need to panic. + * But before we do that, display all the HMI event + * available on the list and set unrecoverable flag to 1. */ if (disposition != OpalHMI_DISPOSITION_RECOVERED) - panic("Unrecoverable HMI exception"); + unrecoverable = 1; spin_lock_irqsave(&opal_hmi_evt_lock, flags); } spin_unlock_irqrestore(&opal_hmi_evt_lock, flags); + + if (unrecoverable) { + int ret; + + /* Pull all HMI events from OPAL before we panic. */ + while (opal_get_msg(__pa(&msg), sizeof(msg)) == OPAL_SUCCESS) { + u32 type; + + type = be32_to_cpu(msg.msg_type); + + /* skip if not HMI event */ + if (type != OPAL_MSG_HMI_EVT) + continue; + + /* HMI event info starts from param[0] */ + hmi_evt = (struct OpalHMIEvent *)&msg.params[0]; + print_hmi_event_info(hmi_evt); + } + + /* + * Unrecoverable HMI exception. We need to inform BMC/OCC + * about this error so that it can collect relevant data + * for error analysis before rebooting. + */ + ret = opal_cec_reboot2(OPAL_REBOOT_PLATFORM_ERROR, + "Unrecoverable HMI exception"); + if (ret == OPAL_UNSUPPORTED) { + pr_emerg("Reboot type %d not supported\n", + OPAL_REBOOT_PLATFORM_ERROR); + } + + /* + * Fall through and panic if opal_cec_reboot2() returns + * OPAL_UNSUPPORTED. + */ + panic("Unrecoverable HMI exception"); + } } static DECLARE_WORK(hmi_event_work, hmi_event_handler); diff --git a/arch/powerpc/platforms/powernv/opal-power.c b/arch/powerpc/platforms/powernv/opal-power.c index ac46c2c24f99e2..58dc3308237f6a 100644 --- a/arch/powerpc/platforms/powernv/opal-power.c +++ b/arch/powerpc/platforms/powernv/opal-power.c @@ -9,9 +9,12 @@ * 2 of the License, or (at your option) any later version. */ +#define pr_fmt(fmt) "opal-power: " fmt + #include #include #include +#include #include #include @@ -19,30 +22,116 @@ #define SOFT_OFF 0x00 #define SOFT_REBOOT 0x01 +/* Detect EPOW event */ +static bool detect_epow(void) +{ + u16 epow; + int i, rc; + __be16 epow_classes; + __be16 opal_epow_status[OPAL_SYSEPOW_MAX] = {0}; + + /* + * Check for EPOW event. Kernel sends supported EPOW classes info + * to OPAL. OPAL returns EPOW info along with classes present. + */ + epow_classes = cpu_to_be16(OPAL_SYSEPOW_MAX); + rc = opal_get_epow_status(opal_epow_status, &epow_classes); + if (rc != OPAL_SUCCESS) { + pr_err("Failed to get EPOW event information\n"); + return false; + } + + /* Look for EPOW events present */ + for (i = 0; i < be16_to_cpu(epow_classes); i++) { + epow = be16_to_cpu(opal_epow_status[i]); + + /* Filter events which do not need shutdown. */ + if (i == OPAL_SYSEPOW_POWER) + epow &= ~(OPAL_SYSPOWER_CHNG | OPAL_SYSPOWER_FAIL | + OPAL_SYSPOWER_INCL); + if (epow) + return true; + } + + return false; +} + +/* Check for existing EPOW, DPO events */ +static bool poweroff_pending(void) +{ + int rc; + __be64 opal_dpo_timeout; + + /* Check for DPO event */ + rc = opal_get_dpo_status(&opal_dpo_timeout); + if (rc == OPAL_SUCCESS) { + pr_info("Existing DPO event detected.\n"); + return true; + } + + /* Check for EPOW event */ + if (detect_epow()) { + pr_info("Existing EPOW event detected.\n"); + return true; + } + + return false; +} + +/* OPAL power-control events notifier */ static int opal_power_control_event(struct notifier_block *nb, - unsigned long msg_type, void *msg) + unsigned long msg_type, void *msg) { - struct opal_msg *power_msg = msg; uint64_t type; - type = be64_to_cpu(power_msg->params[0]); - - switch (type) { - case SOFT_REBOOT: - pr_info("OPAL: reboot requested\n"); - orderly_reboot(); + switch (msg_type) { + case OPAL_MSG_EPOW: + if (detect_epow()) { + pr_info("EPOW msg received. Powering off system\n"); + orderly_poweroff(true); + } break; - case SOFT_OFF: - pr_info("OPAL: poweroff requested\n"); + case OPAL_MSG_DPO: + pr_info("DPO msg received. Powering off system\n"); orderly_poweroff(true); break; + case OPAL_MSG_SHUTDOWN: + type = be64_to_cpu(((struct opal_msg *)msg)->params[0]); + switch (type) { + case SOFT_REBOOT: + pr_info("Reboot requested\n"); + orderly_reboot(); + break; + case SOFT_OFF: + pr_info("Poweroff requested\n"); + orderly_poweroff(true); + break; + default: + pr_err("Unknown power-control type %llu\n", type); + } + break; default: - pr_err("OPAL: power control type unexpected %016llx\n", type); + pr_err("Unknown OPAL message type %lu\n", msg_type); } return 0; } +/* OPAL EPOW event notifier block */ +static struct notifier_block opal_epow_nb = { + .notifier_call = opal_power_control_event, + .next = NULL, + .priority = 0, +}; + +/* OPAL DPO event notifier block */ +static struct notifier_block opal_dpo_nb = { + .notifier_call = opal_power_control_event, + .next = NULL, + .priority = 0, +}; + +/* OPAL power-control event notifier block */ static struct notifier_block opal_power_control_nb = { .notifier_call = opal_power_control_event, .next = NULL, @@ -51,16 +140,40 @@ static struct notifier_block opal_power_control_nb = { static int __init opal_power_control_init(void) { - int ret; + int ret, supported = 0; + struct device_node *np; + /* Register OPAL power-control events notifier */ ret = opal_message_notifier_register(OPAL_MSG_SHUTDOWN, - &opal_power_control_nb); - if (ret) { - pr_err("%s: Can't register OPAL event notifier (%d)\n", - __func__, ret); - return ret; + &opal_power_control_nb); + if (ret) + pr_err("Failed to register SHUTDOWN notifier, ret = %d\n", ret); + + /* Determine OPAL EPOW, DPO support */ + np = of_find_node_by_path("/ibm,opal/epow"); + if (np) { + supported = of_device_is_compatible(np, "ibm,opal-v3-epow"); + of_node_put(np); } + if (!supported) + return 0; + pr_info("OPAL EPOW, DPO support detected.\n"); + + /* Register EPOW event notifier */ + ret = opal_message_notifier_register(OPAL_MSG_EPOW, &opal_epow_nb); + if (ret) + pr_err("Failed to register EPOW notifier, ret = %d\n", ret); + + /* Register DPO event notifier */ + ret = opal_message_notifier_register(OPAL_MSG_DPO, &opal_dpo_nb); + if (ret) + pr_err("Failed to register DPO notifier, ret = %d\n", ret); + + /* Check for any pending EPOW or DPO events. */ + if (poweroff_pending()) + orderly_poweroff(true); + return 0; } machine_subsys_initcall(powernv, opal_power_control_init); diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S index d6a7b8252e4da2..b7a464fef7a77b 100644 --- a/arch/powerpc/platforms/powernv/opal-wrappers.S +++ b/arch/powerpc/platforms/powernv/opal-wrappers.S @@ -202,6 +202,7 @@ OPAL_CALL(opal_rtc_read, OPAL_RTC_READ); OPAL_CALL(opal_rtc_write, OPAL_RTC_WRITE); OPAL_CALL(opal_cec_power_down, OPAL_CEC_POWER_DOWN); OPAL_CALL(opal_cec_reboot, OPAL_CEC_REBOOT); +OPAL_CALL(opal_cec_reboot2, OPAL_CEC_REBOOT2); OPAL_CALL(opal_read_nvram, OPAL_READ_NVRAM); OPAL_CALL(opal_write_nvram, OPAL_WRITE_NVRAM); OPAL_CALL(opal_handle_interrupt, OPAL_HANDLE_INTERRUPT); @@ -249,6 +250,7 @@ OPAL_CALL(opal_pci_reinit, OPAL_PCI_REINIT); OPAL_CALL(opal_pci_mask_pe_error, OPAL_PCI_MASK_PE_ERROR); OPAL_CALL(opal_set_slot_led_status, OPAL_SET_SLOT_LED_STATUS); OPAL_CALL(opal_get_epow_status, OPAL_GET_EPOW_STATUS); +OPAL_CALL(opal_get_dpo_status, OPAL_GET_DPO_STATUS); OPAL_CALL(opal_set_system_attention_led, OPAL_SET_SYSTEM_ATTENTION_LED); OPAL_CALL(opal_pci_next_error, OPAL_PCI_NEXT_ERROR); OPAL_CALL(opal_pci_poll, OPAL_PCI_POLL); @@ -297,3 +299,5 @@ OPAL_CALL(opal_flash_read, OPAL_FLASH_READ); OPAL_CALL(opal_flash_write, OPAL_FLASH_WRITE); OPAL_CALL(opal_flash_erase, OPAL_FLASH_ERASE); OPAL_CALL(opal_prd_msg, OPAL_PRD_MSG); +OPAL_CALL(opal_leds_get_ind, OPAL_LEDS_GET_INDICATOR); +OPAL_CALL(opal_leds_set_ind, OPAL_LEDS_SET_INDICATOR); diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c index f084afa0e3baeb..230f3a7cdea45f 100644 --- a/arch/powerpc/platforms/powernv/opal.c +++ b/arch/powerpc/platforms/powernv/opal.c @@ -441,6 +441,7 @@ static int opal_recover_mce(struct pt_regs *regs, int opal_machine_check(struct pt_regs *regs) { struct machine_check_event evt; + int ret; if (!get_mce_event(&evt, MCE_EVENT_RELEASE)) return 0; @@ -455,6 +456,40 @@ int opal_machine_check(struct pt_regs *regs) if (opal_recover_mce(regs, &evt)) return 1; + + /* + * Unrecovered machine check, we are heading to panic path. + * + * We may have hit this MCE in very early stage of kernel + * initialization even before opal-prd has started running. If + * this is the case then this MCE error may go un-noticed or + * un-analyzed if we go down panic path. We need to inform + * BMC/OCC about this error so that they can collect relevant + * data for error analysis before rebooting. + * Use opal_cec_reboot2(OPAL_REBOOT_PLATFORM_ERROR) to do so. + * This function may not return on BMC based system. + */ + ret = opal_cec_reboot2(OPAL_REBOOT_PLATFORM_ERROR, + "Unrecoverable Machine Check exception"); + if (ret == OPAL_UNSUPPORTED) { + pr_emerg("Reboot type %d not supported\n", + OPAL_REBOOT_PLATFORM_ERROR); + } + + /* + * We reached here. There can be three possibilities: + * 1. We are running on a firmware level that do not support + * opal_cec_reboot2() + * 2. We are running on a firmware level that do not support + * OPAL_REBOOT_PLATFORM_ERROR reboot type. + * 3. We are running on FSP based system that does not need opal + * to trigger checkstop explicitly for error analysis. The FSP + * PRD component would have already got notified about this + * error through other channels. + * + * In any case, let us just fall through. We anyway heading + * down to panic path. + */ return 0; } @@ -648,7 +683,7 @@ static void opal_init_heartbeat(void) static int __init opal_init(void) { - struct device_node *np, *consoles; + struct device_node *np, *consoles, *leds; int rc; opal_node = of_find_node_by_path("/ibm,opal"); @@ -689,6 +724,13 @@ static int __init opal_init(void) /* Setup a heatbeat thread if requested by OPAL */ opal_init_heartbeat(); + /* Create leds platform devices */ + leds = of_find_node_by_path("/ibm,opal/leds"); + if (leds) { + of_platform_device_create(leds, "opal_leds", NULL); + of_node_put(leds); + } + /* Create "opal" kobject under /sys/firmware */ rc = opal_sysfs_init(); if (rc == 0) { @@ -841,3 +883,6 @@ EXPORT_SYMBOL_GPL(opal_rtc_write); EXPORT_SYMBOL_GPL(opal_tpo_read); EXPORT_SYMBOL_GPL(opal_tpo_write); EXPORT_SYMBOL_GPL(opal_i2c_request); +/* Export these symbols for PowerNV LED class driver */ +EXPORT_SYMBOL_GPL(opal_leds_get_ind); +EXPORT_SYMBOL_GPL(opal_leds_set_ind); diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c index 85cbc96eff6cbd..2927cd5c830309 100644 --- a/arch/powerpc/platforms/powernv/pci-ioda.c +++ b/arch/powerpc/platforms/powernv/pci-ioda.c @@ -140,11 +140,9 @@ static void pnv_ioda_reserve_pe(struct pnv_phb *phb, int pe_no) return; } - if (test_and_set_bit(pe_no, phb->ioda.pe_alloc)) { - pr_warn("%s: PE %d was assigned on PHB#%x\n", - __func__, pe_no, phb->hose->global_number); - return; - } + if (test_and_set_bit(pe_no, phb->ioda.pe_alloc)) + pr_debug("%s: PE %d was reserved on PHB#%x\n", + __func__, pe_no, phb->hose->global_number); phb->ioda.pe_array[pe_no].phb = phb; phb->ioda.pe_array[pe_no].pe_number = pe_no; @@ -231,61 +229,60 @@ static int pnv_ioda2_init_m64(struct pnv_phb *phb) return -EIO; } -static void pnv_ioda2_reserve_m64_pe(struct pnv_phb *phb) +static void pnv_ioda2_reserve_dev_m64_pe(struct pci_dev *pdev, + unsigned long *pe_bitmap) { - resource_size_t sgsz = phb->ioda.m64_segsize; - struct pci_dev *pdev; + struct pci_controller *hose = pci_bus_to_host(pdev->bus); + struct pnv_phb *phb = hose->private_data; struct resource *r; - int base, step, i; - - /* - * Root bus always has full M64 range and root port has - * M64 range used in reality. So we're checking root port - * instead of root bus. - */ - list_for_each_entry(pdev, &phb->hose->bus->devices, bus_list) { - for (i = 0; i < PCI_BRIDGE_RESOURCE_NUM; i++) { - r = &pdev->resource[PCI_BRIDGE_RESOURCES + i]; - if (!r->parent || - !pnv_pci_is_mem_pref_64(r->flags)) - continue; + resource_size_t base, sgsz, start, end; + int segno, i; + + base = phb->ioda.m64_base; + sgsz = phb->ioda.m64_segsize; + for (i = 0; i <= PCI_ROM_RESOURCE; i++) { + r = &pdev->resource[i]; + if (!r->parent || !pnv_pci_is_mem_pref_64(r->flags)) + continue; - base = (r->start - phb->ioda.m64_base) / sgsz; - for (step = 0; step < resource_size(r) / sgsz; step++) - pnv_ioda_reserve_pe(phb, base + step); + start = _ALIGN_DOWN(r->start - base, sgsz); + end = _ALIGN_UP(r->end - base, sgsz); + for (segno = start / sgsz; segno < end / sgsz; segno++) { + if (pe_bitmap) + set_bit(segno, pe_bitmap); + else + pnv_ioda_reserve_pe(phb, segno); } } } -static int pnv_ioda2_pick_m64_pe(struct pnv_phb *phb, - struct pci_bus *bus, int all) +static void pnv_ioda2_reserve_m64_pe(struct pci_bus *bus, + unsigned long *pe_bitmap, + bool all) { - resource_size_t segsz = phb->ioda.m64_segsize; struct pci_dev *pdev; - struct resource *r; + + list_for_each_entry(pdev, &bus->devices, bus_list) { + pnv_ioda2_reserve_dev_m64_pe(pdev, pe_bitmap); + + if (all && pdev->subordinate) + pnv_ioda2_reserve_m64_pe(pdev->subordinate, + pe_bitmap, all); + } +} + +static int pnv_ioda2_pick_m64_pe(struct pci_bus *bus, bool all) +{ + struct pci_controller *hose = pci_bus_to_host(bus); + struct pnv_phb *phb = hose->private_data; struct pnv_ioda_pe *master_pe, *pe; unsigned long size, *pe_alloc; - bool found; - int start, i, j; + int i; /* Root bus shouldn't use M64 */ if (pci_is_root_bus(bus)) return IODA_INVALID_PE; - /* We support only one M64 window on each bus */ - found = false; - pci_bus_for_each_resource(bus, r, i) { - if (r && r->parent && - pnv_pci_is_mem_pref_64(r->flags)) { - found = true; - break; - } - } - - /* No M64 window found ? */ - if (!found) - return IODA_INVALID_PE; - /* Allocate bitmap */ size = _ALIGN_UP(phb->ioda.total_pe / 8, sizeof(unsigned long)); pe_alloc = kzalloc(size, GFP_KERNEL); @@ -295,35 +292,8 @@ static int pnv_ioda2_pick_m64_pe(struct pnv_phb *phb, return IODA_INVALID_PE; } - /* - * Figure out reserved PE numbers by the PE - * the its child PEs. - */ - start = (r->start - phb->ioda.m64_base) / segsz; - for (i = 0; i < resource_size(r) / segsz; i++) - set_bit(start + i, pe_alloc); - - if (all) - goto done; - - /* - * If the PE doesn't cover all subordinate buses, - * we need subtract from reserved PEs for children. - */ - list_for_each_entry(pdev, &bus->devices, bus_list) { - if (!pdev->subordinate) - continue; - - pci_bus_for_each_resource(pdev->subordinate, r, i) { - if (!r || !r->parent || - !pnv_pci_is_mem_pref_64(r->flags)) - continue; - - start = (r->start - phb->ioda.m64_base) / segsz; - for (j = 0; j < resource_size(r) / segsz ; j++) - clear_bit(start + j, pe_alloc); - } - } + /* Figure out reserved PE numbers by the PE */ + pnv_ioda2_reserve_m64_pe(bus, pe_alloc, all); /* * the current bus might not own M64 window and that's all @@ -339,7 +309,6 @@ static int pnv_ioda2_pick_m64_pe(struct pnv_phb *phb, * Figure out the master PE and put all slave PEs to master * PE's list to form compound PE. */ -done: master_pe = NULL; i = -1; while ((i = find_next_bit(pe_alloc, phb->ioda.total_pe, i + 1)) < @@ -653,7 +622,7 @@ static int pnv_ioda_set_peltv(struct pnv_phb *phb, pdev = pe->pdev->bus->self; #ifdef CONFIG_PCI_IOV else if (pe->flags & PNV_IODA_PE_VF) - pdev = pe->parent_dev->bus->self; + pdev = pe->parent_dev; #endif /* CONFIG_PCI_IOV */ while (pdev) { struct pci_dn *pdn = pci_get_pdn(pdev); @@ -732,7 +701,7 @@ static int pnv_ioda_deconfigure_pe(struct pnv_phb *phb, struct pnv_ioda_pe *pe) parent = parent->bus->self; } - opal_pci_eeh_freeze_set(phb->opal_id, pe->pe_number, + opal_pci_eeh_freeze_clear(phb->opal_id, pe->pe_number, OPAL_EEH_ACTION_CLEAR_FREEZE_ALL); /* Disassociate PE in PELT */ @@ -946,8 +915,9 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset) res2 = *res; res->start += size * offset; - dev_info(&dev->dev, "VF BAR%d: %pR shifted to %pR (enabling %d VFs shifted by %d)\n", - i, &res2, res, num_vfs, offset); + dev_info(&dev->dev, "VF BAR%d: %pR shifted to %pR (%sabling %d VFs shifted by %d)\n", + i, &res2, res, (offset > 0) ? "En" : "Dis", + num_vfs, offset); pci_update_resource(dev, i + PCI_IOV_RESOURCES); } return 0; @@ -1050,7 +1020,7 @@ static void pnv_ioda_setup_same_PE(struct pci_bus *bus, struct pnv_ioda_pe *pe) * subordinate PCI devices and buses. The second type of PE is normally * orgiriated by PCIe-to-PCI bridge or PLX switch downstream ports. */ -static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, int all) +static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all) { struct pci_controller *hose = pci_bus_to_host(bus); struct pnv_phb *phb = hose->private_data; @@ -1059,7 +1029,7 @@ static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, int all) /* Check if PE is determined by M64 */ if (phb->pick_m64_pe) - pe_num = phb->pick_m64_pe(phb, bus, all); + pe_num = phb->pick_m64_pe(bus, all); /* The PE number isn't pinned by M64 */ if (pe_num == IODA_INVALID_PE) @@ -1117,12 +1087,12 @@ static void pnv_ioda_setup_PEs(struct pci_bus *bus) { struct pci_dev *dev; - pnv_ioda_setup_bus_PE(bus, 0); + pnv_ioda_setup_bus_PE(bus, false); list_for_each_entry(dev, &bus->devices, bus_list) { if (dev->subordinate) { if (pci_pcie_type(dev) == PCI_EXP_TYPE_PCI_BRIDGE) - pnv_ioda_setup_bus_PE(dev->subordinate, 1); + pnv_ioda_setup_bus_PE(dev->subordinate, true); else pnv_ioda_setup_PEs(dev->subordinate); } @@ -1147,7 +1117,7 @@ static void pnv_pci_ioda_setup_PEs(void) /* M64 layout might affect PE allocation */ if (phb->reserve_m64_pe) - phb->reserve_m64_pe(phb); + phb->reserve_m64_pe(hose->bus, NULL, true); pnv_ioda_setup_PEs(hose->bus); } @@ -1590,6 +1560,7 @@ static void pnv_pci_ioda_dma_dev_setup(struct pnv_phb *phb, struct pci_dev *pdev pe = &phb->ioda.pe_array[pdn->pe_number]; WARN_ON(get_dma_ops(&pdev->dev) != &dma_iommu_ops); + set_dma_offset(&pdev->dev, pe->tce_bypass_base); set_iommu_table_base(&pdev->dev, pe->table_group.tables[0]); /* * Note: iommu_add_device() will fail here as @@ -1620,19 +1591,18 @@ static int pnv_pci_ioda_dma_set_mask(struct pci_dev *pdev, u64 dma_mask) if (bypass) { dev_info(&pdev->dev, "Using 64-bit DMA iommu bypass\n"); set_dma_ops(&pdev->dev, &dma_direct_ops); - set_dma_offset(&pdev->dev, pe->tce_bypass_base); } else { dev_info(&pdev->dev, "Using 32-bit DMA via iommu\n"); set_dma_ops(&pdev->dev, &dma_iommu_ops); - set_iommu_table_base(&pdev->dev, pe->table_group.tables[0]); } *pdev->dev.dma_mask = dma_mask; return 0; } -static u64 pnv_pci_ioda_dma_get_required_mask(struct pnv_phb *phb, - struct pci_dev *pdev) +static u64 pnv_pci_ioda_dma_get_required_mask(struct pci_dev *pdev) { + struct pci_controller *hose = pci_bus_to_host(pdev->bus); + struct pnv_phb *phb = hose->private_data; struct pci_dn *pdn = pci_get_pdn(pdev); struct pnv_ioda_pe *pe; u64 end, mask; @@ -1659,6 +1629,7 @@ static void pnv_ioda_setup_bus_dma(struct pnv_ioda_pe *pe, list_for_each_entry(dev, &bus->devices, bus_list) { set_iommu_table_base(&dev->dev, pe->table_group.tables[0]); + set_dma_offset(&dev->dev, pe->tce_bypass_base); iommu_add_device(&dev->dev); if ((pe->flags & PNV_IODA_PE_BUS_ALL) && dev->subordinate) @@ -3057,6 +3028,7 @@ static const struct pci_controller_ops pnv_pci_ioda_controller_ops = { .window_alignment = pnv_pci_window_alignment, .reset_secondary_bus = pnv_pci_reset_secondary_bus, .dma_set_mask = pnv_pci_ioda_dma_set_mask, + .dma_get_required_mask = pnv_pci_ioda_dma_get_required_mask, .shutdown = pnv_pci_ioda_shutdown, }; @@ -3203,7 +3175,6 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np, /* Setup TCEs */ phb->dma_dev_setup = pnv_pci_ioda_dma_dev_setup; - phb->dma_get_required_mask = pnv_pci_ioda_dma_get_required_mask; /* Setup MSI support */ pnv_pci_init_ioda_msis(phb); diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c index bc6d4e02e29c1d..9b2480b265c00b 100644 --- a/arch/powerpc/platforms/powernv/pci.c +++ b/arch/powerpc/platforms/powernv/pci.c @@ -761,17 +761,6 @@ void pnv_pci_dma_dev_setup(struct pci_dev *pdev) phb->dma_dev_setup(phb, pdev); } -u64 pnv_pci_dma_get_required_mask(struct pci_dev *pdev) -{ - struct pci_controller *hose = pci_bus_to_host(pdev->bus); - struct pnv_phb *phb = hose->private_data; - - if (phb && phb->dma_get_required_mask) - return phb->dma_get_required_mask(phb, pdev); - - return __dma_get_required_mask(&pdev->dev); -} - void pnv_pci_shutdown(void) { struct pci_controller *hose; diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h index 8ef2d28aded0f6..c8ff50e907662c 100644 --- a/arch/powerpc/platforms/powernv/pci.h +++ b/arch/powerpc/platforms/powernv/pci.h @@ -105,13 +105,12 @@ struct pnv_phb { unsigned int hwirq, unsigned int virq, unsigned int is_64, struct msi_msg *msg); void (*dma_dev_setup)(struct pnv_phb *phb, struct pci_dev *pdev); - u64 (*dma_get_required_mask)(struct pnv_phb *phb, - struct pci_dev *pdev); void (*fixup_phb)(struct pci_controller *hose); u32 (*bdfn_to_pe)(struct pnv_phb *phb, struct pci_bus *bus, u32 devfn); int (*init_m64)(struct pnv_phb *phb); - void (*reserve_m64_pe)(struct pnv_phb *phb); - int (*pick_m64_pe)(struct pnv_phb *phb, struct pci_bus *bus, int all); + void (*reserve_m64_pe)(struct pci_bus *bus, + unsigned long *pe_bitmap, bool all); + int (*pick_m64_pe)(struct pci_bus *bus, bool all); int (*get_pe_state)(struct pnv_phb *phb, int pe_no); void (*freeze_pe)(struct pnv_phb *phb, int pe_no); int (*unfreeze_pe)(struct pnv_phb *phb, int pe_no, int opt); diff --git a/arch/powerpc/platforms/powernv/powernv.h b/arch/powerpc/platforms/powernv/powernv.h index 9269e30e4ca0e7..6dbc0a1da1f61d 100644 --- a/arch/powerpc/platforms/powernv/powernv.h +++ b/arch/powerpc/platforms/powernv/powernv.h @@ -12,15 +12,9 @@ struct pci_dev; #ifdef CONFIG_PCI extern void pnv_pci_init(void); extern void pnv_pci_shutdown(void); -extern u64 pnv_pci_dma_get_required_mask(struct pci_dev *pdev); #else static inline void pnv_pci_init(void) { } static inline void pnv_pci_shutdown(void) { } - -static inline u64 pnv_pci_dma_get_required_mask(struct pci_dev *pdev) -{ - return 0; -} #endif extern u32 pnv_get_supported_cpuidle_states(void); diff --git a/arch/powerpc/platforms/powernv/rng.c b/arch/powerpc/platforms/powernv/rng.c index 6eb808ff637e71..5dcbdea1afac00 100644 --- a/arch/powerpc/platforms/powernv/rng.c +++ b/arch/powerpc/platforms/powernv/rng.c @@ -128,7 +128,7 @@ static __init int rng_create(struct device_node *dn) pr_info_once("Registering arch random hook.\n"); - ppc_md.get_random_long = powernv_get_random_long; + ppc_md.get_random_seed = powernv_get_random_long; return 0; } diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c index 53737e019ae362..685b3cbe1362c0 100644 --- a/arch/powerpc/platforms/powernv/setup.c +++ b/arch/powerpc/platforms/powernv/setup.c @@ -165,14 +165,6 @@ static void pnv_progress(char *s, unsigned short hex) { } -static u64 pnv_dma_get_required_mask(struct device *dev) -{ - if (dev_is_pci(dev)) - return pnv_pci_dma_get_required_mask(to_pci_dev(dev)); - - return __dma_get_required_mask(dev); -} - static void pnv_shutdown(void) { /* Let the PCI code clear up IODA tables */ @@ -243,6 +235,13 @@ static void pnv_kexec_cpu_down(int crash_shutdown, int secondary) } else { /* Primary waits for the secondaries to have reached OPAL */ pnv_kexec_wait_secondaries_down(); + + /* + * We might be running as little-endian - now that interrupts + * are disabled, reset the HILE bit to big-endian so we don't + * take interrupts in the wrong endian later + */ + opal_reinit_cpus(OPAL_REINIT_CPUS_HILE_BE); } } #endif /* CONFIG_KEXEC */ @@ -314,7 +313,6 @@ define_machine(powernv) { .machine_shutdown = pnv_shutdown, .power_save = power7_idle, .calibrate_decr = generic_calibrate_decr, - .dma_get_required_mask = pnv_dma_get_required_mask, #ifdef CONFIG_KEXEC .kexec_cpu_down = pnv_kexec_cpu_down, #endif diff --git a/arch/powerpc/platforms/powernv/subcore.c b/arch/powerpc/platforms/powernv/subcore.c index f60f80ada90394..503a73f593599b 100644 --- a/arch/powerpc/platforms/powernv/subcore.c +++ b/arch/powerpc/platforms/powernv/subcore.c @@ -190,7 +190,7 @@ static void unsplit_core(void) hid0 = mfspr(SPRN_HID0); hid0 &= ~HID0_POWER8_DYNLPARDIS; - mtspr(SPRN_HID0, hid0); + update_power8_hid0(hid0); update_hid_in_slw(hid0); while (mfspr(SPRN_HID0) & mask) @@ -227,7 +227,7 @@ static void split_core(int new_mode) /* Write new mode */ hid0 = mfspr(SPRN_HID0); hid0 |= HID0_POWER8_DYNLPARDIS | split_parms[i].value; - mtspr(SPRN_HID0, hid0); + update_power8_hid0(hid0); update_hid_in_slw(hid0); /* Wait for it to happen */ diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c index 0ced387e14638f..e9ff44cd5d86e3 100644 --- a/arch/powerpc/platforms/pseries/hotplug-memory.c +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c @@ -92,13 +92,12 @@ static struct property *dlpar_clone_drconf_property(struct device_node *dn) return NULL; new_prop->name = kstrdup(prop->name, GFP_KERNEL); - new_prop->value = kmalloc(prop->length, GFP_KERNEL); + new_prop->value = kmemdup(prop->value, prop->length, GFP_KERNEL); if (!new_prop->name || !new_prop->value) { dlpar_free_drconf_property(new_prop); return NULL; } - memcpy(new_prop->value, prop->value, prop->length); new_prop->length = prop->length; /* Convert the property to cpu endian-ness */ diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c index 10510dea16b31a..0946b98d75d41f 100644 --- a/arch/powerpc/platforms/pseries/iommu.c +++ b/arch/powerpc/platforms/pseries/iommu.c @@ -1253,11 +1253,10 @@ static int dma_set_mask_pSeriesLP(struct device *dev, u64 dma_mask) } } - /* fall back on iommu ops, restore table pointer with ops */ + /* fall back on iommu ops */ if (!ddw_enabled && get_dma_ops(dev) != &dma_iommu_ops) { dev_info(dev, "Restoring 32-bit DMA via iommu\n"); set_dma_ops(dev, &dma_iommu_ops); - pci_dma_dev_setup_pSeriesLP(pdev); } check_mask: diff --git a/arch/powerpc/platforms/pseries/ras.c b/arch/powerpc/platforms/pseries/ras.c index 02e4a17455164d..3b6647e574b6d0 100644 --- a/arch/powerpc/platforms/pseries/ras.c +++ b/arch/powerpc/platforms/pseries/ras.c @@ -189,7 +189,8 @@ static irqreturn_t ras_epow_interrupt(int irq, void *dev_id) int state; int critical; - status = rtas_get_sensor(EPOW_SENSOR_TOKEN, EPOW_SENSOR_INDEX, &state); + status = rtas_get_sensor_fast(EPOW_SENSOR_TOKEN, EPOW_SENSOR_INDEX, + &state); if (state > 3) critical = 1; /* Time Critical */ diff --git a/arch/powerpc/platforms/pseries/rng.c b/arch/powerpc/platforms/pseries/rng.c index e09608770909ca..31ca557af60bc8 100644 --- a/arch/powerpc/platforms/pseries/rng.c +++ b/arch/powerpc/platforms/pseries/rng.c @@ -38,7 +38,7 @@ static __init int rng_init(void) pr_info("Registering arch random hook.\n"); - ppc_md.get_random_long = pseries_get_random_long; + ppc_md.get_random_seed = pseries_get_random_long; return 0; } diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c index df6a7041922b67..39a74fad3e0452 100644 --- a/arch/powerpc/platforms/pseries/setup.c +++ b/arch/powerpc/platforms/pseries/setup.c @@ -254,19 +254,26 @@ static void __init pseries_discover_pic(void) static int pci_dn_reconfig_notifier(struct notifier_block *nb, unsigned long action, void *data) { struct of_reconfig_data *rd = data; - struct device_node *np = rd->dn; - struct pci_dn *pci = NULL; + struct device_node *parent, *np = rd->dn; + struct pci_dn *pdn; int err = NOTIFY_OK; switch (action) { case OF_RECONFIG_ATTACH_NODE: - pci = np->parent->data; - if (pci) { - update_dn_pci_info(np, pci->phb); - - /* Create EEH device for the OF node */ - eeh_dev_init(PCI_DN(np), pci->phb); + parent = of_get_parent(np); + pdn = parent ? PCI_DN(parent) : NULL; + if (pdn) { + /* Create pdn and EEH device */ + update_dn_pci_info(np, pdn->phb); + eeh_dev_init(PCI_DN(np), pdn->phb); } + + of_node_put(parent); + break; + case OF_RECONFIG_DETACH_NODE: + pdn = PCI_DN(np); + if (pdn) + list_del(&pdn->list); break; default: err = NOTIFY_DONE; diff --git a/arch/powerpc/sysdev/cpm_common.c b/arch/powerpc/sysdev/cpm_common.c index 4f786957129040..e2ea51961979c0 100644 --- a/arch/powerpc/sysdev/cpm_common.c +++ b/arch/powerpc/sysdev/cpm_common.c @@ -147,7 +147,7 @@ unsigned long cpm_muram_alloc(unsigned long size, unsigned long align) spin_lock_irqsave(&cpm_muram_lock, flags); cpm_muram_info.alignment = align; start = rh_alloc(&cpm_muram_info, size, "commproc"); - memset(cpm_muram_addr(start), 0, size); + memset_io(cpm_muram_addr(start), 0, size); spin_unlock_irqrestore(&cpm_muram_lock, flags); return start; diff --git a/arch/powerpc/sysdev/dart_iommu.c b/arch/powerpc/sysdev/dart_iommu.c index 90bcdfeedf4829..b7348637eae00a 100644 --- a/arch/powerpc/sysdev/dart_iommu.c +++ b/arch/powerpc/sysdev/dart_iommu.c @@ -313,20 +313,11 @@ static void iommu_table_dart_setup(void) set_bit(iommu_table_dart.it_size - 1, iommu_table_dart.it_map); } -static void dma_dev_setup_dart(struct device *dev) -{ - /* We only have one iommu table on the mac for now, which makes - * things simple. Setup all PCI devices to point to this table - */ - if (get_dma_ops(dev) == &dma_direct_ops) - set_dma_offset(dev, DART_U4_BYPASS_BASE); - else - set_iommu_table_base(dev, &iommu_table_dart); -} - static void pci_dma_dev_setup_dart(struct pci_dev *dev) { - dma_dev_setup_dart(&dev->dev); + if (dart_is_u4) + set_dma_offset(&dev->dev, DART_U4_BYPASS_BASE); + set_iommu_table_base(&dev->dev, &iommu_table_dart); } static void pci_dma_bus_setup_dart(struct pci_bus *bus) @@ -370,7 +361,6 @@ static int dart_dma_set_mask(struct device *dev, u64 dma_mask) dev_info(dev, "Using 32-bit DMA via iommu\n"); set_dma_ops(dev, &dma_iommu_ops); } - dma_dev_setup_dart(dev); *dev->dma_mask = dma_mask; return 0; diff --git a/arch/powerpc/sysdev/ppc4xx_hsta_msi.c b/arch/powerpc/sysdev/ppc4xx_hsta_msi.c index af3c144b92c14f..52a93dcae26224 100644 --- a/arch/powerpc/sysdev/ppc4xx_hsta_msi.c +++ b/arch/powerpc/sysdev/ppc4xx_hsta_msi.c @@ -132,7 +132,7 @@ static int hsta_msi_probe(struct platform_device *pdev) struct pci_controller *phb; mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); - if (IS_ERR(mem)) { + if (!mem) { dev_err(dev, "Unable to get mmio space\n"); return -EINVAL; } @@ -157,7 +157,7 @@ static int hsta_msi_probe(struct platform_device *pdev) goto out; ppc4xx_hsta_msi.irq_map = kmalloc(sizeof(int) * irq_count, GFP_KERNEL); - if (IS_ERR(ppc4xx_hsta_msi.irq_map)) { + if (!ppc4xx_hsta_msi.irq_map) { ret = -ENOMEM; goto out1; } diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c index e599259d84fc83..6ef1231c6e9c08 100644 --- a/arch/powerpc/xmon/xmon.c +++ b/arch/powerpc/xmon/xmon.c @@ -1987,7 +1987,6 @@ memex(void) case '^': adrs -= size; break; - break; case '/': if (nslash > 0) adrs -= 1 << nslash; @@ -2731,7 +2730,7 @@ static void xmon_print_symbol(unsigned long address, const char *mid, void dump_segments(void) { int i; - unsigned long esid,vsid,valid; + unsigned long esid,vsid; unsigned long llp; printf("SLB contents of cpu 0x%x\n", smp_processor_id()); @@ -2739,10 +2738,9 @@ void dump_segments(void) for (i = 0; i < mmu_slb_size; i++) { asm volatile("slbmfee %0,%1" : "=r" (esid) : "r" (i)); asm volatile("slbmfev %0,%1" : "=r" (vsid) : "r" (i)); - valid = (esid & SLB_ESID_V); - if (valid | esid | vsid) { + if (esid || vsid) { printf("%02d %016lx %016lx", i, esid, vsid); - if (valid) { + if (esid & SLB_ESID_V) { llp = vsid & SLB_VSID_LLP; if (vsid & SLB_VSID_B_1T) { printf(" 1T ESID=%9lx VSID=%13lx LLP:%3lx \n", diff --git a/arch/s390/include/asm/atomic.h b/arch/s390/include/asm/atomic.h index adbe3802e377bc..117fa5c921c1b0 100644 --- a/arch/s390/include/asm/atomic.h +++ b/arch/s390/include/asm/atomic.h @@ -27,6 +27,7 @@ #define __ATOMIC_OR "lao" #define __ATOMIC_AND "lan" #define __ATOMIC_ADD "laa" +#define __ATOMIC_XOR "lax" #define __ATOMIC_BARRIER "bcr 14,0\n" #define __ATOMIC_LOOP(ptr, op_val, op_string, __barrier) \ @@ -49,6 +50,7 @@ #define __ATOMIC_OR "or" #define __ATOMIC_AND "nr" #define __ATOMIC_ADD "ar" +#define __ATOMIC_XOR "xr" #define __ATOMIC_BARRIER "\n" #define __ATOMIC_LOOP(ptr, op_val, op_string, __barrier) \ @@ -118,15 +120,17 @@ static inline void atomic_add(int i, atomic_t *v) #define atomic_dec_return(_v) atomic_sub_return(1, _v) #define atomic_dec_and_test(_v) (atomic_sub_return(1, _v) == 0) -static inline void atomic_clear_mask(unsigned int mask, atomic_t *v) -{ - __ATOMIC_LOOP(v, ~mask, __ATOMIC_AND, __ATOMIC_NO_BARRIER); +#define ATOMIC_OP(op, OP) \ +static inline void atomic_##op(int i, atomic_t *v) \ +{ \ + __ATOMIC_LOOP(v, i, __ATOMIC_##OP, __ATOMIC_NO_BARRIER); \ } -static inline void atomic_set_mask(unsigned int mask, atomic_t *v) -{ - __ATOMIC_LOOP(v, mask, __ATOMIC_OR, __ATOMIC_NO_BARRIER); -} +ATOMIC_OP(and, AND) +ATOMIC_OP(or, OR) +ATOMIC_OP(xor, XOR) + +#undef ATOMIC_OP #define atomic_xchg(v, new) (xchg(&((v)->counter), new)) @@ -167,6 +171,7 @@ static inline int __atomic_add_unless(atomic_t *v, int a, int u) #define __ATOMIC64_OR "laog" #define __ATOMIC64_AND "lang" #define __ATOMIC64_ADD "laag" +#define __ATOMIC64_XOR "laxg" #define __ATOMIC64_BARRIER "bcr 14,0\n" #define __ATOMIC64_LOOP(ptr, op_val, op_string, __barrier) \ @@ -189,6 +194,7 @@ static inline int __atomic_add_unless(atomic_t *v, int a, int u) #define __ATOMIC64_OR "ogr" #define __ATOMIC64_AND "ngr" #define __ATOMIC64_ADD "agr" +#define __ATOMIC64_XOR "xgr" #define __ATOMIC64_BARRIER "\n" #define __ATOMIC64_LOOP(ptr, op_val, op_string, __barrier) \ @@ -247,16 +253,6 @@ static inline void atomic64_add(long long i, atomic64_t *v) __ATOMIC64_LOOP(v, i, __ATOMIC64_ADD, __ATOMIC64_NO_BARRIER); } -static inline void atomic64_clear_mask(unsigned long mask, atomic64_t *v) -{ - __ATOMIC64_LOOP(v, ~mask, __ATOMIC64_AND, __ATOMIC64_NO_BARRIER); -} - -static inline void atomic64_set_mask(unsigned long mask, atomic64_t *v) -{ - __ATOMIC64_LOOP(v, mask, __ATOMIC64_OR, __ATOMIC64_NO_BARRIER); -} - #define atomic64_xchg(v, new) (xchg(&((v)->counter), new)) static inline long long atomic64_cmpxchg(atomic64_t *v, @@ -270,6 +266,17 @@ static inline long long atomic64_cmpxchg(atomic64_t *v, return old; } +#define ATOMIC64_OP(op, OP) \ +static inline void atomic64_##op(long i, atomic64_t *v) \ +{ \ + __ATOMIC64_LOOP(v, i, __ATOMIC64_##OP, __ATOMIC64_NO_BARRIER); \ +} + +ATOMIC64_OP(and, AND) +ATOMIC64_OP(or, OR) +ATOMIC64_OP(xor, XOR) + +#undef ATOMIC64_OP #undef __ATOMIC64_LOOP static inline int atomic64_add_unless(atomic64_t *v, long long i, long long u) diff --git a/arch/s390/include/asm/barrier.h b/arch/s390/include/asm/barrier.h index e6f8615a11eb5c..d48fe016233160 100644 --- a/arch/s390/include/asm/barrier.h +++ b/arch/s390/include/asm/barrier.h @@ -42,12 +42,12 @@ do { \ compiletime_assert_atomic_type(*p); \ barrier(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ barrier(); \ ___p1; \ diff --git a/arch/s390/include/asm/jump_label.h b/arch/s390/include/asm/jump_label.h index 69972b7957ee40..7f9fd5e3f1bff6 100644 --- a/arch/s390/include/asm/jump_label.h +++ b/arch/s390/include/asm/jump_label.h @@ -12,14 +12,29 @@ * We use a brcl 0,2 instruction for jump labels at compile time so it * can be easily distinguished from a hotpatch generated instruction. */ -static __always_inline bool arch_static_branch(struct static_key *key) +static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { asm_volatile_goto("0: brcl 0,"__stringify(JUMP_LABEL_NOP_OFFSET)"\n" ".pushsection __jump_table, \"aw\"\n" ".balign 8\n" ".quad 0b, %l[label], %0\n" ".popsection\n" - : : "X" (key) : : label); + : : "X" (&((char *)key)[branch]) : : label); + + return false; +label: + return true; +} + +static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch) +{ + asm_volatile_goto("0: brcl 15, %l[label]\n" + ".pushsection __jump_table, \"aw\"\n" + ".balign 8\n" + ".quad 0b, %l[label], %0\n" + ".popsection\n" + : : "X" (&((char *)key)[branch]) : : label); + return false; label: return true; diff --git a/arch/s390/kernel/jump_label.c b/arch/s390/kernel/jump_label.c index c9dac2139f59ef..083b05f5f5ab6f 100644 --- a/arch/s390/kernel/jump_label.c +++ b/arch/s390/kernel/jump_label.c @@ -61,7 +61,7 @@ static void __jump_label_transform(struct jump_entry *entry, { struct insn old, new; - if (type == JUMP_LABEL_ENABLE) { + if (type == JUMP_LABEL_JMP) { jump_label_make_nop(entry, &old); jump_label_make_branch(entry, &new); } else { diff --git a/arch/s390/kernel/time.c b/arch/s390/kernel/time.c index 52524b9083c36a..017c3a9bfc280e 100644 --- a/arch/s390/kernel/time.c +++ b/arch/s390/kernel/time.c @@ -378,7 +378,7 @@ static void disable_sync_clock(void *dummy) * increase the "sequence" counter to avoid the race of an * etr event and the complete recovery against get_sync_clock. */ - atomic_clear_mask(0x80000000, sw_ptr); + atomic_andnot(0x80000000, sw_ptr); atomic_inc(sw_ptr); } @@ -389,7 +389,7 @@ static void disable_sync_clock(void *dummy) static void enable_sync_clock(void) { atomic_t *sw_ptr = this_cpu_ptr(&clock_sync_word); - atomic_set_mask(0x80000000, sw_ptr); + atomic_or(0x80000000, sw_ptr); } /* diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index b277d50dcf76a4..5c2c169395c3ca 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -173,20 +173,20 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu) static void __set_cpu_idle(struct kvm_vcpu *vcpu) { - atomic_set_mask(CPUSTAT_WAIT, &vcpu->arch.sie_block->cpuflags); + atomic_or(CPUSTAT_WAIT, &vcpu->arch.sie_block->cpuflags); set_bit(vcpu->vcpu_id, vcpu->arch.local_int.float_int->idle_mask); } static void __unset_cpu_idle(struct kvm_vcpu *vcpu) { - atomic_clear_mask(CPUSTAT_WAIT, &vcpu->arch.sie_block->cpuflags); + atomic_andnot(CPUSTAT_WAIT, &vcpu->arch.sie_block->cpuflags); clear_bit(vcpu->vcpu_id, vcpu->arch.local_int.float_int->idle_mask); } static void __reset_intercept_indicators(struct kvm_vcpu *vcpu) { - atomic_clear_mask(CPUSTAT_IO_INT | CPUSTAT_EXT_INT | CPUSTAT_STOP_INT, - &vcpu->arch.sie_block->cpuflags); + atomic_andnot(CPUSTAT_IO_INT | CPUSTAT_EXT_INT | CPUSTAT_STOP_INT, + &vcpu->arch.sie_block->cpuflags); vcpu->arch.sie_block->lctl = 0x0000; vcpu->arch.sie_block->ictl &= ~(ICTL_LPSW | ICTL_STCTL | ICTL_PINT); @@ -199,7 +199,7 @@ static void __reset_intercept_indicators(struct kvm_vcpu *vcpu) static void __set_cpuflag(struct kvm_vcpu *vcpu, u32 flag) { - atomic_set_mask(flag, &vcpu->arch.sie_block->cpuflags); + atomic_or(flag, &vcpu->arch.sie_block->cpuflags); } static void set_intercept_indicators_io(struct kvm_vcpu *vcpu) @@ -928,7 +928,7 @@ void kvm_s390_clear_local_irqs(struct kvm_vcpu *vcpu) spin_unlock(&li->lock); /* clear pending external calls set by sigp interpretation facility */ - atomic_clear_mask(CPUSTAT_ECALL_PEND, li->cpuflags); + atomic_andnot(CPUSTAT_ECALL_PEND, li->cpuflags); vcpu->kvm->arch.sca->cpu[vcpu->vcpu_id].sigp_ctrl = 0; } @@ -1026,7 +1026,7 @@ static int __inject_pfault_init(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq) li->irq.ext = irq->u.ext; set_bit(IRQ_PEND_PFAULT_INIT, &li->pending_irqs); - atomic_set_mask(CPUSTAT_EXT_INT, li->cpuflags); + atomic_or(CPUSTAT_EXT_INT, li->cpuflags); return 0; } @@ -1041,7 +1041,7 @@ static int __inject_extcall_sigpif(struct kvm_vcpu *vcpu, uint16_t src_id) /* another external call is pending */ return -EBUSY; } - atomic_set_mask(CPUSTAT_ECALL_PEND, &vcpu->arch.sie_block->cpuflags); + atomic_or(CPUSTAT_ECALL_PEND, &vcpu->arch.sie_block->cpuflags); return 0; } @@ -1067,7 +1067,7 @@ static int __inject_extcall(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq) if (test_and_set_bit(IRQ_PEND_EXT_EXTERNAL, &li->pending_irqs)) return -EBUSY; *extcall = irq->u.extcall; - atomic_set_mask(CPUSTAT_EXT_INT, li->cpuflags); + atomic_or(CPUSTAT_EXT_INT, li->cpuflags); return 0; } @@ -1139,7 +1139,7 @@ static int __inject_sigp_emergency(struct kvm_vcpu *vcpu, set_bit(irq->u.emerg.code, li->sigp_emerg_pending); set_bit(IRQ_PEND_EXT_EMERGENCY, &li->pending_irqs); - atomic_set_mask(CPUSTAT_EXT_INT, li->cpuflags); + atomic_or(CPUSTAT_EXT_INT, li->cpuflags); return 0; } @@ -1183,7 +1183,7 @@ static int __inject_ckc(struct kvm_vcpu *vcpu) 0, 0); set_bit(IRQ_PEND_EXT_CLOCK_COMP, &li->pending_irqs); - atomic_set_mask(CPUSTAT_EXT_INT, li->cpuflags); + atomic_or(CPUSTAT_EXT_INT, li->cpuflags); return 0; } @@ -1196,7 +1196,7 @@ static int __inject_cpu_timer(struct kvm_vcpu *vcpu) 0, 0); set_bit(IRQ_PEND_EXT_CPU_TIMER, &li->pending_irqs); - atomic_set_mask(CPUSTAT_EXT_INT, li->cpuflags); + atomic_or(CPUSTAT_EXT_INT, li->cpuflags); return 0; } @@ -1375,13 +1375,13 @@ static void __floating_irq_kick(struct kvm *kvm, u64 type) spin_lock(&li->lock); switch (type) { case KVM_S390_MCHK: - atomic_set_mask(CPUSTAT_STOP_INT, li->cpuflags); + atomic_or(CPUSTAT_STOP_INT, li->cpuflags); break; case KVM_S390_INT_IO_MIN...KVM_S390_INT_IO_MAX: - atomic_set_mask(CPUSTAT_IO_INT, li->cpuflags); + atomic_or(CPUSTAT_IO_INT, li->cpuflags); break; default: - atomic_set_mask(CPUSTAT_EXT_INT, li->cpuflags); + atomic_or(CPUSTAT_EXT_INT, li->cpuflags); break; } spin_unlock(&li->lock); diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 98df53c0134398..c91eb941b444ee 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -1333,12 +1333,12 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) save_access_regs(vcpu->arch.host_acrs); restore_access_regs(vcpu->run->s.regs.acrs); gmap_enable(vcpu->arch.gmap); - atomic_set_mask(CPUSTAT_RUNNING, &vcpu->arch.sie_block->cpuflags); + atomic_or(CPUSTAT_RUNNING, &vcpu->arch.sie_block->cpuflags); } void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) { - atomic_clear_mask(CPUSTAT_RUNNING, &vcpu->arch.sie_block->cpuflags); + atomic_andnot(CPUSTAT_RUNNING, &vcpu->arch.sie_block->cpuflags); gmap_disable(vcpu->arch.gmap); save_fpu_regs(); @@ -1443,9 +1443,9 @@ int kvm_arch_vcpu_setup(struct kvm_vcpu *vcpu) CPUSTAT_STOPPED); if (test_kvm_facility(vcpu->kvm, 78)) - atomic_set_mask(CPUSTAT_GED2, &vcpu->arch.sie_block->cpuflags); + atomic_or(CPUSTAT_GED2, &vcpu->arch.sie_block->cpuflags); else if (test_kvm_facility(vcpu->kvm, 8)) - atomic_set_mask(CPUSTAT_GED, &vcpu->arch.sie_block->cpuflags); + atomic_or(CPUSTAT_GED, &vcpu->arch.sie_block->cpuflags); kvm_s390_vcpu_setup_model(vcpu); @@ -1557,24 +1557,24 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu) void kvm_s390_vcpu_block(struct kvm_vcpu *vcpu) { - atomic_set_mask(PROG_BLOCK_SIE, &vcpu->arch.sie_block->prog20); + atomic_or(PROG_BLOCK_SIE, &vcpu->arch.sie_block->prog20); exit_sie(vcpu); } void kvm_s390_vcpu_unblock(struct kvm_vcpu *vcpu) { - atomic_clear_mask(PROG_BLOCK_SIE, &vcpu->arch.sie_block->prog20); + atomic_andnot(PROG_BLOCK_SIE, &vcpu->arch.sie_block->prog20); } static void kvm_s390_vcpu_request(struct kvm_vcpu *vcpu) { - atomic_set_mask(PROG_REQUEST, &vcpu->arch.sie_block->prog20); + atomic_or(PROG_REQUEST, &vcpu->arch.sie_block->prog20); exit_sie(vcpu); } static void kvm_s390_vcpu_request_handled(struct kvm_vcpu *vcpu) { - atomic_clear_mask(PROG_REQUEST, &vcpu->arch.sie_block->prog20); + atomic_or(PROG_REQUEST, &vcpu->arch.sie_block->prog20); } /* @@ -1583,7 +1583,7 @@ static void kvm_s390_vcpu_request_handled(struct kvm_vcpu *vcpu) * return immediately. */ void exit_sie(struct kvm_vcpu *vcpu) { - atomic_set_mask(CPUSTAT_STOP_INT, &vcpu->arch.sie_block->cpuflags); + atomic_or(CPUSTAT_STOP_INT, &vcpu->arch.sie_block->cpuflags); while (vcpu->arch.sie_block->prog0c & PROG_IN_SIE) cpu_relax(); } @@ -1807,19 +1807,19 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu, if (dbg->control & KVM_GUESTDBG_ENABLE) { vcpu->guest_debug = dbg->control; /* enforce guest PER */ - atomic_set_mask(CPUSTAT_P, &vcpu->arch.sie_block->cpuflags); + atomic_or(CPUSTAT_P, &vcpu->arch.sie_block->cpuflags); if (dbg->control & KVM_GUESTDBG_USE_HW_BP) rc = kvm_s390_import_bp_data(vcpu, dbg); } else { - atomic_clear_mask(CPUSTAT_P, &vcpu->arch.sie_block->cpuflags); + atomic_andnot(CPUSTAT_P, &vcpu->arch.sie_block->cpuflags); vcpu->arch.guestdbg.last_bp = 0; } if (rc) { vcpu->guest_debug = 0; kvm_s390_clear_bp_data(vcpu); - atomic_clear_mask(CPUSTAT_P, &vcpu->arch.sie_block->cpuflags); + atomic_andnot(CPUSTAT_P, &vcpu->arch.sie_block->cpuflags); } return rc; @@ -1894,7 +1894,7 @@ static int kvm_s390_handle_requests(struct kvm_vcpu *vcpu) if (kvm_check_request(KVM_REQ_ENABLE_IBS, vcpu)) { if (!ibs_enabled(vcpu)) { trace_kvm_s390_enable_disable_ibs(vcpu->vcpu_id, 1); - atomic_set_mask(CPUSTAT_IBS, + atomic_or(CPUSTAT_IBS, &vcpu->arch.sie_block->cpuflags); } goto retry; @@ -1903,7 +1903,7 @@ static int kvm_s390_handle_requests(struct kvm_vcpu *vcpu) if (kvm_check_request(KVM_REQ_DISABLE_IBS, vcpu)) { if (ibs_enabled(vcpu)) { trace_kvm_s390_enable_disable_ibs(vcpu->vcpu_id, 0); - atomic_clear_mask(CPUSTAT_IBS, + atomic_andnot(CPUSTAT_IBS, &vcpu->arch.sie_block->cpuflags); } goto retry; @@ -2419,7 +2419,7 @@ void kvm_s390_vcpu_start(struct kvm_vcpu *vcpu) __disable_ibs_on_all_vcpus(vcpu->kvm); } - atomic_clear_mask(CPUSTAT_STOPPED, &vcpu->arch.sie_block->cpuflags); + atomic_andnot(CPUSTAT_STOPPED, &vcpu->arch.sie_block->cpuflags); /* * Another VCPU might have used IBS while we were offline. * Let's play safe and flush the VCPU at startup. @@ -2445,7 +2445,7 @@ void kvm_s390_vcpu_stop(struct kvm_vcpu *vcpu) /* SIGP STOP and SIGP STOP AND STORE STATUS has been fully processed */ kvm_s390_clear_stop_irq(vcpu); - atomic_set_mask(CPUSTAT_STOPPED, &vcpu->arch.sie_block->cpuflags); + atomic_or(CPUSTAT_STOPPED, &vcpu->arch.sie_block->cpuflags); __disable_ibs_on_vcpu(vcpu); for (i = 0; i < online_vcpus; i++) { diff --git a/arch/s390/lib/uaccess.c b/arch/s390/lib/uaccess.c index 0d002a746bec15..ae4de559e3a042 100644 --- a/arch/s390/lib/uaccess.c +++ b/arch/s390/lib/uaccess.c @@ -15,7 +15,7 @@ #include #include -static struct static_key have_mvcos = STATIC_KEY_INIT_FALSE; +static DEFINE_STATIC_KEY_FALSE(have_mvcos); static inline unsigned long copy_from_user_mvcos(void *x, const void __user *ptr, unsigned long size) @@ -104,7 +104,7 @@ static inline unsigned long copy_from_user_mvcp(void *x, const void __user *ptr, unsigned long __copy_from_user(void *to, const void __user *from, unsigned long n) { - if (static_key_false(&have_mvcos)) + if (static_branch_likely(&have_mvcos)) return copy_from_user_mvcos(to, from, n); return copy_from_user_mvcp(to, from, n); } @@ -177,7 +177,7 @@ static inline unsigned long copy_to_user_mvcs(void __user *ptr, const void *x, unsigned long __copy_to_user(void __user *to, const void *from, unsigned long n) { - if (static_key_false(&have_mvcos)) + if (static_branch_likely(&have_mvcos)) return copy_to_user_mvcos(to, from, n); return copy_to_user_mvcs(to, from, n); } @@ -240,7 +240,7 @@ static inline unsigned long copy_in_user_mvc(void __user *to, const void __user unsigned long __copy_in_user(void __user *to, const void __user *from, unsigned long n) { - if (static_key_false(&have_mvcos)) + if (static_branch_likely(&have_mvcos)) return copy_in_user_mvcos(to, from, n); return copy_in_user_mvc(to, from, n); } @@ -312,7 +312,7 @@ static inline unsigned long clear_user_xc(void __user *to, unsigned long size) unsigned long __clear_user(void __user *to, unsigned long size) { - if (static_key_false(&have_mvcos)) + if (static_branch_likely(&have_mvcos)) return clear_user_mvcos(to, size); return clear_user_xc(to, size); } @@ -373,7 +373,7 @@ EXPORT_SYMBOL(__strncpy_from_user); static int __init uaccess_init(void) { if (test_facility(27)) - static_key_slow_inc(&have_mvcos); + static_branch_enable(&have_mvcos); return 0; } early_initcall(uaccess_init); diff --git a/arch/sh/include/asm/atomic-grb.h b/arch/sh/include/asm/atomic-grb.h index 97a5fda83450e7..b94df40e5f2d2b 100644 --- a/arch/sh/include/asm/atomic-grb.h +++ b/arch/sh/include/asm/atomic-grb.h @@ -48,47 +48,12 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP -static inline void atomic_clear_mask(unsigned int mask, atomic_t *v) -{ - int tmp; - unsigned int _mask = ~mask; - - __asm__ __volatile__ ( - " .align 2 \n\t" - " mova 1f, r0 \n\t" /* r0 = end point */ - " mov r15, r1 \n\t" /* r1 = saved sp */ - " mov #-6, r15 \n\t" /* LOGIN: r15 = size */ - " mov.l @%1, %0 \n\t" /* load old value */ - " and %2, %0 \n\t" /* add */ - " mov.l %0, @%1 \n\t" /* store new value */ - "1: mov r1, r15 \n\t" /* LOGOUT */ - : "=&r" (tmp), - "+r" (v) - : "r" (_mask) - : "memory" , "r0", "r1"); -} - -static inline void atomic_set_mask(unsigned int mask, atomic_t *v) -{ - int tmp; - - __asm__ __volatile__ ( - " .align 2 \n\t" - " mova 1f, r0 \n\t" /* r0 = end point */ - " mov r15, r1 \n\t" /* r1 = saved sp */ - " mov #-6, r15 \n\t" /* LOGIN: r15 = size */ - " mov.l @%1, %0 \n\t" /* load old value */ - " or %2, %0 \n\t" /* or */ - " mov.l %0, @%1 \n\t" /* store new value */ - "1: mov r1, r15 \n\t" /* LOGOUT */ - : "=&r" (tmp), - "+r" (v) - : "r" (mask) - : "memory" , "r0", "r1"); -} - #endif /* __ASM_SH_ATOMIC_GRB_H */ diff --git a/arch/sh/include/asm/atomic-irq.h b/arch/sh/include/asm/atomic-irq.h index 61d107523f0649..23fcdad5773eea 100644 --- a/arch/sh/include/asm/atomic-irq.h +++ b/arch/sh/include/asm/atomic-irq.h @@ -37,27 +37,12 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add, +=) ATOMIC_OPS(sub, -=) +ATOMIC_OP(and, &=) +ATOMIC_OP(or, |=) +ATOMIC_OP(xor, ^=) #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP -static inline void atomic_clear_mask(unsigned int mask, atomic_t *v) -{ - unsigned long flags; - - raw_local_irq_save(flags); - v->counter &= ~mask; - raw_local_irq_restore(flags); -} - -static inline void atomic_set_mask(unsigned int mask, atomic_t *v) -{ - unsigned long flags; - - raw_local_irq_save(flags); - v->counter |= mask; - raw_local_irq_restore(flags); -} - #endif /* __ASM_SH_ATOMIC_IRQ_H */ diff --git a/arch/sh/include/asm/atomic-llsc.h b/arch/sh/include/asm/atomic-llsc.h index 8575dccb9ef781..33d34b16d4d67b 100644 --- a/arch/sh/include/asm/atomic-llsc.h +++ b/arch/sh/include/asm/atomic-llsc.h @@ -52,37 +52,12 @@ static inline int atomic_##op##_return(int i, atomic_t *v) \ ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP -static inline void atomic_clear_mask(unsigned int mask, atomic_t *v) -{ - unsigned long tmp; - - __asm__ __volatile__ ( -"1: movli.l @%2, %0 ! atomic_clear_mask \n" -" and %1, %0 \n" -" movco.l %0, @%2 \n" -" bf 1b \n" - : "=&z" (tmp) - : "r" (~mask), "r" (&v->counter) - : "t"); -} - -static inline void atomic_set_mask(unsigned int mask, atomic_t *v) -{ - unsigned long tmp; - - __asm__ __volatile__ ( -"1: movli.l @%2, %0 ! atomic_set_mask \n" -" or %1, %0 \n" -" movco.l %0, @%2 \n" -" bf 1b \n" - : "=&z" (tmp) - : "r" (mask), "r" (&v->counter) - : "t"); -} - #endif /* __ASM_SH_ATOMIC_LLSC_H */ diff --git a/arch/sparc/include/asm/atomic_32.h b/arch/sparc/include/asm/atomic_32.h index 0e69b7e7a439ce..7dcbebbcaec6f0 100644 --- a/arch/sparc/include/asm/atomic_32.h +++ b/arch/sparc/include/asm/atomic_32.h @@ -17,10 +17,12 @@ #include #include - #define ATOMIC_INIT(i) { (i) } int atomic_add_return(int, atomic_t *); +void atomic_and(int, atomic_t *); +void atomic_or(int, atomic_t *); +void atomic_xor(int, atomic_t *); int atomic_cmpxchg(atomic_t *, int, int); int atomic_xchg(atomic_t *, int); int __atomic_add_unless(atomic_t *, int, int); diff --git a/arch/sparc/include/asm/atomic_64.h b/arch/sparc/include/asm/atomic_64.h index 4082749913ce06..917084ace49dee 100644 --- a/arch/sparc/include/asm/atomic_64.h +++ b/arch/sparc/include/asm/atomic_64.h @@ -33,6 +33,10 @@ long atomic64_##op##_return(long, atomic64_t *); ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP diff --git a/arch/sparc/include/asm/barrier_64.h b/arch/sparc/include/asm/barrier_64.h index 809941e33e1217..14a928601657da 100644 --- a/arch/sparc/include/asm/barrier_64.h +++ b/arch/sparc/include/asm/barrier_64.h @@ -60,12 +60,12 @@ do { __asm__ __volatile__("ba,pt %%xcc, 1f\n\t" \ do { \ compiletime_assert_atomic_type(*p); \ barrier(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ barrier(); \ ___p1; \ diff --git a/arch/sparc/include/asm/jump_label.h b/arch/sparc/include/asm/jump_label.h index cc9b04a2b11b63..62d0354d172746 100644 --- a/arch/sparc/include/asm/jump_label.h +++ b/arch/sparc/include/asm/jump_label.h @@ -7,16 +7,33 @@ #define JUMP_LABEL_NOP_SIZE 4 -static __always_inline bool arch_static_branch(struct static_key *key) +static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { - asm_volatile_goto("1:\n\t" - "nop\n\t" - "nop\n\t" - ".pushsection __jump_table, \"aw\"\n\t" - ".align 4\n\t" - ".word 1b, %l[l_yes], %c0\n\t" - ".popsection \n\t" - : : "i" (key) : : l_yes); + asm_volatile_goto("1:\n\t" + "nop\n\t" + "nop\n\t" + ".pushsection __jump_table, \"aw\"\n\t" + ".align 4\n\t" + ".word 1b, %l[l_yes], %c0\n\t" + ".popsection \n\t" + : : "i" (&((char *)key)[branch]) : : l_yes); + + return false; +l_yes: + return true; +} + +static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch) +{ + asm_volatile_goto("1:\n\t" + "b %l[l_yes]\n\t" + "nop\n\t" + ".pushsection __jump_table, \"aw\"\n\t" + ".align 4\n\t" + ".word 1b, %l[l_yes], %c0\n\t" + ".popsection \n\t" + : : "i" (&((char *)key)[branch]) : : l_yes); + return false; l_yes: return true; diff --git a/arch/sparc/kernel/jump_label.c b/arch/sparc/kernel/jump_label.c index 48565c11e82ae2..59bbeff550243d 100644 --- a/arch/sparc/kernel/jump_label.c +++ b/arch/sparc/kernel/jump_label.c @@ -16,7 +16,7 @@ void arch_jump_label_transform(struct jump_entry *entry, u32 val; u32 *insn = (u32 *) (unsigned long) entry->code; - if (type == JUMP_LABEL_ENABLE) { + if (type == JUMP_LABEL_JMP) { s32 off = (s32)entry->target - (s32)entry->code; #ifdef CONFIG_SPARC64 diff --git a/arch/sparc/lib/atomic32.c b/arch/sparc/lib/atomic32.c index 71cd65ab200c87..b9d63c0a7aabc2 100644 --- a/arch/sparc/lib/atomic32.c +++ b/arch/sparc/lib/atomic32.c @@ -27,22 +27,38 @@ static DEFINE_SPINLOCK(dummy); #endif /* SMP */ -#define ATOMIC_OP(op, cop) \ +#define ATOMIC_OP_RETURN(op, c_op) \ int atomic_##op##_return(int i, atomic_t *v) \ { \ int ret; \ unsigned long flags; \ spin_lock_irqsave(ATOMIC_HASH(v), flags); \ \ - ret = (v->counter cop i); \ + ret = (v->counter c_op i); \ \ spin_unlock_irqrestore(ATOMIC_HASH(v), flags); \ return ret; \ } \ EXPORT_SYMBOL(atomic_##op##_return); -ATOMIC_OP(add, +=) +#define ATOMIC_OP(op, c_op) \ +void atomic_##op(int i, atomic_t *v) \ +{ \ + unsigned long flags; \ + spin_lock_irqsave(ATOMIC_HASH(v), flags); \ + \ + v->counter c_op i; \ + \ + spin_unlock_irqrestore(ATOMIC_HASH(v), flags); \ +} \ +EXPORT_SYMBOL(atomic_##op); + +ATOMIC_OP_RETURN(add, +=) +ATOMIC_OP(and, &=) +ATOMIC_OP(or, |=) +ATOMIC_OP(xor, ^=) +#undef ATOMIC_OP_RETURN #undef ATOMIC_OP int atomic_xchg(atomic_t *v, int new) diff --git a/arch/sparc/lib/atomic_64.S b/arch/sparc/lib/atomic_64.S index 05dac43907d119..d6b0363f345bfb 100644 --- a/arch/sparc/lib/atomic_64.S +++ b/arch/sparc/lib/atomic_64.S @@ -47,6 +47,9 @@ ENDPROC(atomic_##op##_return); ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN @@ -84,6 +87,9 @@ ENDPROC(atomic64_##op##_return); ATOMIC64_OPS(add) ATOMIC64_OPS(sub) +ATOMIC64_OP(and) +ATOMIC64_OP(or) +ATOMIC64_OP(xor) #undef ATOMIC64_OPS #undef ATOMIC64_OP_RETURN diff --git a/arch/sparc/lib/ksyms.c b/arch/sparc/lib/ksyms.c index 8069ce12f20b13..8eb454cfe05c9f 100644 --- a/arch/sparc/lib/ksyms.c +++ b/arch/sparc/lib/ksyms.c @@ -111,6 +111,9 @@ EXPORT_SYMBOL(atomic64_##op##_return); ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN diff --git a/arch/tile/Kconfig b/arch/tile/Kconfig index 9def1f52d03a4f..2ba12d76172344 100644 --- a/arch/tile/Kconfig +++ b/arch/tile/Kconfig @@ -32,6 +32,7 @@ config TILE select EDAC_SUPPORT select GENERIC_STRNCPY_FROM_USER select GENERIC_STRNLEN_USER + select HAVE_ARCH_SECCOMP_FILTER # FIXME: investigate whether we need/want these options. # select HAVE_IOREMAP_PROT @@ -221,6 +222,22 @@ config COMPAT If enabled, the kernel will support running TILE-Gx binaries that were built with the -m32 option. +config SECCOMP + bool "Enable seccomp to safely compute untrusted bytecode" + depends on PROC_FS + help + This kernel feature is useful for number crunching applications + that may need to compute untrusted bytecode during their + execution. By using pipes or other transports made available to + the process as file descriptors supporting the read/write + syscalls, it's possible to isolate those applications in + their own address space using seccomp. Once seccomp is + enabled via prctl, it cannot be disabled and the task is only + allowed to execute a few safe syscalls defined by each seccomp + mode. + + If unsure, say N. + config SYSVIPC_COMPAT def_bool y depends on COMPAT && SYSVIPC diff --git a/arch/tile/include/asm/Kbuild b/arch/tile/include/asm/Kbuild index d8a84316347127..ba35c41c71fff3 100644 --- a/arch/tile/include/asm/Kbuild +++ b/arch/tile/include/asm/Kbuild @@ -28,6 +28,7 @@ generic-y += poll.h generic-y += posix_types.h generic-y += preempt.h generic-y += resource.h +generic-y += seccomp.h generic-y += sembuf.h generic-y += serial.h generic-y += shmbuf.h diff --git a/arch/tile/include/asm/atomic_32.h b/arch/tile/include/asm/atomic_32.h index 1b109fad9fff3c..d320ce253d8630 100644 --- a/arch/tile/include/asm/atomic_32.h +++ b/arch/tile/include/asm/atomic_32.h @@ -34,6 +34,19 @@ static inline void atomic_add(int i, atomic_t *v) _atomic_xchg_add(&v->counter, i); } +#define ATOMIC_OP(op) \ +unsigned long _atomic_##op(volatile unsigned long *p, unsigned long mask); \ +static inline void atomic_##op(int i, atomic_t *v) \ +{ \ + _atomic_##op((unsigned long *)&v->counter, i); \ +} + +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + +#undef ATOMIC_OP + /** * atomic_add_return - add integer and return * @v: pointer of type atomic_t @@ -113,6 +126,17 @@ static inline void atomic64_add(long long i, atomic64_t *v) _atomic64_xchg_add(&v->counter, i); } +#define ATOMIC64_OP(op) \ +long long _atomic64_##op(long long *v, long long n); \ +static inline void atomic64_##op(long long i, atomic64_t *v) \ +{ \ + _atomic64_##op(&v->counter, i); \ +} + +ATOMIC64_OP(and) +ATOMIC64_OP(or) +ATOMIC64_OP(xor) + /** * atomic64_add_return - add integer and return * @v: pointer of type atomic64_t @@ -225,6 +249,7 @@ extern struct __get_user __atomic_xchg_add(volatile int *p, int *lock, int n); extern struct __get_user __atomic_xchg_add_unless(volatile int *p, int *lock, int o, int n); extern struct __get_user __atomic_or(volatile int *p, int *lock, int n); +extern struct __get_user __atomic_and(volatile int *p, int *lock, int n); extern struct __get_user __atomic_andn(volatile int *p, int *lock, int n); extern struct __get_user __atomic_xor(volatile int *p, int *lock, int n); extern long long __atomic64_cmpxchg(volatile long long *p, int *lock, @@ -234,6 +259,9 @@ extern long long __atomic64_xchg_add(volatile long long *p, int *lock, long long n); extern long long __atomic64_xchg_add_unless(volatile long long *p, int *lock, long long o, long long n); +extern long long __atomic64_and(volatile long long *p, int *lock, long long n); +extern long long __atomic64_or(volatile long long *p, int *lock, long long n); +extern long long __atomic64_xor(volatile long long *p, int *lock, long long n); /* Return failure from the atomic wrappers. */ struct __get_user __atomic_bad_address(int __user *addr); diff --git a/arch/tile/include/asm/atomic_64.h b/arch/tile/include/asm/atomic_64.h index 0496970cef8296..096a56d6ead429 100644 --- a/arch/tile/include/asm/atomic_64.h +++ b/arch/tile/include/asm/atomic_64.h @@ -58,6 +58,26 @@ static inline int __atomic_add_unless(atomic_t *v, int a, int u) return oldval; } +static inline void atomic_and(int i, atomic_t *v) +{ + __insn_fetchand4((void *)&v->counter, i); +} + +static inline void atomic_or(int i, atomic_t *v) +{ + __insn_fetchor4((void *)&v->counter, i); +} + +static inline void atomic_xor(int i, atomic_t *v) +{ + int guess, oldval = v->counter; + do { + guess = oldval; + __insn_mtspr(SPR_CMPEXCH_VALUE, guess); + oldval = __insn_cmpexch4(&v->counter, guess ^ i); + } while (guess != oldval); +} + /* Now the true 64-bit operations. */ #define ATOMIC64_INIT(i) { (i) } @@ -91,6 +111,26 @@ static inline long atomic64_add_unless(atomic64_t *v, long a, long u) return oldval != u; } +static inline void atomic64_and(long i, atomic64_t *v) +{ + __insn_fetchand((void *)&v->counter, i); +} + +static inline void atomic64_or(long i, atomic64_t *v) +{ + __insn_fetchor((void *)&v->counter, i); +} + +static inline void atomic64_xor(long i, atomic64_t *v) +{ + long guess, oldval = v->counter; + do { + guess = oldval; + __insn_mtspr(SPR_CMPEXCH_VALUE, guess); + oldval = __insn_cmpexch(&v->counter, guess ^ i); + } while (guess != oldval); +} + #define atomic64_sub_return(i, v) atomic64_add_return(-(i), (v)) #define atomic64_sub(i, v) atomic64_add(-(i), (v)) #define atomic64_inc_return(v) atomic64_add_return(1, (v)) diff --git a/arch/tile/include/asm/elf.h b/arch/tile/include/asm/elf.h index 41d9878a968688..c505d77e4d06a8 100644 --- a/arch/tile/include/asm/elf.h +++ b/arch/tile/include/asm/elf.h @@ -22,6 +22,7 @@ #include #include +#include #include #include @@ -30,9 +31,6 @@ typedef unsigned long elf_greg_t; #define ELF_NGREG (sizeof(struct pt_regs) / sizeof(elf_greg_t)) typedef elf_greg_t elf_gregset_t[ELF_NGREG]; -#define EM_TILEPRO 188 -#define EM_TILEGX 191 - /* Provide a nominal data structure. */ #define ELF_NFPREG 0 typedef double elf_fpreg_t; diff --git a/arch/tile/include/asm/syscall.h b/arch/tile/include/asm/syscall.h index 9644b88f133da7..373d73064ea13d 100644 --- a/arch/tile/include/asm/syscall.h +++ b/arch/tile/include/asm/syscall.h @@ -20,6 +20,8 @@ #include #include +#include +#include #include /* The array of function pointers for syscalls. */ @@ -61,7 +63,15 @@ static inline void syscall_set_return_value(struct task_struct *task, struct pt_regs *regs, int error, long val) { - regs->regs[0] = (long) error ?: val; + if (error) { + /* R0 is the passed-in negative error, R1 is positive. */ + regs->regs[0] = error; + regs->regs[1] = -error; + } else { + /* R1 set to zero to indicate no error. */ + regs->regs[0] = val; + regs->regs[1] = 0; + } } static inline void syscall_get_arguments(struct task_struct *task, @@ -82,4 +92,20 @@ static inline void syscall_set_arguments(struct task_struct *task, memcpy(®s[i], args, n * sizeof(args[0])); } +/* + * We don't care about endianness (__AUDIT_ARCH_LE bit) here because + * tile has the same system calls both on little- and big- endian. + */ +static inline int syscall_get_arch(void) +{ + if (is_compat_task()) + return AUDIT_ARCH_TILEGX32; + +#ifdef CONFIG_TILEGX + return AUDIT_ARCH_TILEGX; +#else + return AUDIT_ARCH_TILEPRO; +#endif +} + #endif /* _ASM_TILE_SYSCALL_H */ diff --git a/arch/tile/include/uapi/arch/opcode_tilegx.h b/arch/tile/include/uapi/arch/opcode_tilegx.h index d76ff2db745e2e..9e46eaa847d4eb 100644 --- a/arch/tile/include/uapi/arch/opcode_tilegx.h +++ b/arch/tile/include/uapi/arch/opcode_tilegx.h @@ -830,11 +830,11 @@ enum ADDX_RRR_0_OPCODE_X0 = 2, ADDX_RRR_0_OPCODE_X1 = 2, ADDX_RRR_0_OPCODE_Y0 = 0, - ADDX_SPECIAL_0_OPCODE_Y1 = 0, + ADDX_RRR_0_OPCODE_Y1 = 0, ADD_RRR_0_OPCODE_X0 = 3, ADD_RRR_0_OPCODE_X1 = 3, ADD_RRR_0_OPCODE_Y0 = 1, - ADD_SPECIAL_0_OPCODE_Y1 = 1, + ADD_RRR_0_OPCODE_Y1 = 1, ANDI_IMM8_OPCODE_X0 = 3, ANDI_IMM8_OPCODE_X1 = 3, ANDI_OPCODE_Y0 = 2, @@ -995,6 +995,7 @@ enum LD4U_ADD_IMM8_OPCODE_X1 = 12, LD4U_OPCODE_Y2 = 2, LD4U_UNARY_OPCODE_X1 = 20, + LDNA_ADD_IMM8_OPCODE_X1 = 21, LDNA_UNARY_OPCODE_X1 = 21, LDNT1S_ADD_IMM8_OPCODE_X1 = 13, LDNT1S_UNARY_OPCODE_X1 = 22, @@ -1015,7 +1016,6 @@ enum LD_UNARY_OPCODE_X1 = 29, LNK_UNARY_OPCODE_X1 = 30, LNK_UNARY_OPCODE_Y1 = 14, - LWNA_ADD_IMM8_OPCODE_X1 = 21, MFSPR_IMM8_OPCODE_X1 = 22, MF_UNARY_OPCODE_X1 = 31, MM_BF_OPCODE_X0 = 7, diff --git a/arch/tile/kernel/intvec_32.S b/arch/tile/kernel/intvec_32.S index cdbda45a4e4bba..fbbe2ea882ea72 100644 --- a/arch/tile/kernel/intvec_32.S +++ b/arch/tile/kernel/intvec_32.S @@ -1224,6 +1224,7 @@ handle_syscall: jal do_syscall_trace_enter } FEEDBACK_REENTER(handle_syscall) + blz r0, .Lsyscall_sigreturn_skip /* * We always reload our registers from the stack at this diff --git a/arch/tile/kernel/intvec_64.S b/arch/tile/kernel/intvec_64.S index 800b91d3f9dc79..58964d209d4db5 100644 --- a/arch/tile/kernel/intvec_64.S +++ b/arch/tile/kernel/intvec_64.S @@ -1247,6 +1247,7 @@ handle_syscall: jal do_syscall_trace_enter } FEEDBACK_REENTER(handle_syscall) + bltz r0, .Lsyscall_sigreturn_skip /* * We always reload our registers from the stack at this diff --git a/arch/tile/kernel/ptrace.c b/arch/tile/kernel/ptrace.c index f84eed8243da54..bdc126faf7412f 100644 --- a/arch/tile/kernel/ptrace.c +++ b/arch/tile/kernel/ptrace.c @@ -262,6 +262,9 @@ int do_syscall_trace_enter(struct pt_regs *regs) if (work & _TIF_NOHZ) user_exit(); + if (secure_computing() == -1) + return -1; + if (work & _TIF_SYSCALL_TRACE) { if (tracehook_report_syscall_entry(regs)) regs->regs[TREG_SYSCALL_NR] = -1; diff --git a/arch/tile/kernel/time.c b/arch/tile/kernel/time.c index 00178ecf9aeab4..178989e6d3e3ae 100644 --- a/arch/tile/kernel/time.c +++ b/arch/tile/kernel/time.c @@ -140,10 +140,10 @@ static int tile_timer_set_next_event(unsigned long ticks, * Whenever anyone tries to change modes, we just mask interrupts * and wait for the next event to get set. */ -static void tile_timer_set_mode(enum clock_event_mode mode, - struct clock_event_device *evt) +static int tile_timer_shutdown(struct clock_event_device *evt) { arch_local_irq_mask_now(INT_TILE_TIMER); + return 0; } /* @@ -157,7 +157,9 @@ static DEFINE_PER_CPU(struct clock_event_device, tile_timer) = { .rating = 100, .irq = -1, .set_next_event = tile_timer_set_next_event, - .set_mode = tile_timer_set_mode, + .set_state_shutdown = tile_timer_shutdown, + .set_state_oneshot = tile_timer_shutdown, + .tick_resume = tile_timer_shutdown, }; void setup_tile_timer(void) diff --git a/arch/tile/kernel/vdso/Makefile b/arch/tile/kernel/vdso/Makefile index a025f63d54cd96..c54fff37b5ff07 100644 --- a/arch/tile/kernel/vdso/Makefile +++ b/arch/tile/kernel/vdso/Makefile @@ -54,7 +54,7 @@ $(obj)/built-in.o: $(obj)/vdso-syms.o $(obj)/built-in.o: ld_flags += -R $(obj)/vdso-syms.o SYSCFLAGS_vdso.so.dbg = -shared -s -Wl,-soname=linux-vdso.so.1 \ - $(call cc-ldoption, -Wl$(comma)--hash-style=sysv) + $(call cc-ldoption, -Wl$(comma)--hash-style=both) SYSCFLAGS_vdso_syms.o = -r $(obj)/vdso-syms.o: $(src)/vdso.lds $(obj)/vrt_sigreturn.o FORCE $(call if_changed,vdsold) @@ -113,6 +113,6 @@ $(obj)/vrt_sigreturn32.o: $(obj)/vrt_sigreturn.S $(obj)/vdso32.o: $(obj)/vdso32.so SYSCFLAGS_vdso32.so.dbg = -m32 -shared -s -Wl,-soname=linux-vdso32.so.1 \ - $(call cc-ldoption, -Wl$(comma)--hash-style=sysv) + $(call cc-ldoption, -Wl$(comma)--hash-style=both) $(obj)/vdso32.so.dbg: $(src)/vdso.lds $(obj-vdso32) $(call if_changed,vdsold) diff --git a/arch/tile/lib/atomic_32.c b/arch/tile/lib/atomic_32.c index c89b211fd9e7c0..298df1e9912a15 100644 --- a/arch/tile/lib/atomic_32.c +++ b/arch/tile/lib/atomic_32.c @@ -94,6 +94,12 @@ unsigned long _atomic_or(volatile unsigned long *p, unsigned long mask) } EXPORT_SYMBOL(_atomic_or); +unsigned long _atomic_and(volatile unsigned long *p, unsigned long mask) +{ + return __atomic_and((int *)p, __atomic_setup(p), mask).val; +} +EXPORT_SYMBOL(_atomic_and); + unsigned long _atomic_andn(volatile unsigned long *p, unsigned long mask) { return __atomic_andn((int *)p, __atomic_setup(p), mask).val; @@ -136,6 +142,23 @@ long long _atomic64_cmpxchg(long long *v, long long o, long long n) } EXPORT_SYMBOL(_atomic64_cmpxchg); +long long _atomic64_and(long long *v, long long n) +{ + return __atomic64_and(v, __atomic_setup(v), n); +} +EXPORT_SYMBOL(_atomic64_and); + +long long _atomic64_or(long long *v, long long n) +{ + return __atomic64_or(v, __atomic_setup(v), n); +} +EXPORT_SYMBOL(_atomic64_or); + +long long _atomic64_xor(long long *v, long long n) +{ + return __atomic64_xor(v, __atomic_setup(v), n); +} +EXPORT_SYMBOL(_atomic64_xor); /* * If any of the atomic or futex routines hit a bad address (not in diff --git a/arch/tile/lib/atomic_asm_32.S b/arch/tile/lib/atomic_asm_32.S index 6bda3132cd619e..f611265633d695 100644 --- a/arch/tile/lib/atomic_asm_32.S +++ b/arch/tile/lib/atomic_asm_32.S @@ -178,6 +178,7 @@ atomic_op _xchg_add, 32, "add r24, r22, r2" atomic_op _xchg_add_unless, 32, \ "sne r26, r22, r2; { bbns r26, 3f; add r24, r22, r3 }" atomic_op _or, 32, "or r24, r22, r2" +atomic_op _and, 32, "and r24, r22, r2" atomic_op _andn, 32, "nor r2, r2, zero; and r24, r22, r2" atomic_op _xor, 32, "xor r24, r22, r2" @@ -191,6 +192,9 @@ atomic_op 64_xchg_add_unless, 64, \ { bbns r26, 3f; add r24, r22, r4 }; \ { bbns r27, 3f; add r25, r23, r5 }; \ slt_u r26, r24, r22; add r25, r25, r26" +atomic_op 64_or, 64, "{ or r24, r22, r2; or r25, r23, r3 }" +atomic_op 64_and, 64, "{ and r24, r22, r2; and r25, r23, r3 }" +atomic_op 64_xor, 64, "{ xor r24, r22, r2; xor r25, r23, r3 }" jrp lr /* happy backtracer */ diff --git a/arch/x86/include/asm/atomic.h b/arch/x86/include/asm/atomic.h index e9168955c42f4e..fb52aa644aabb4 100644 --- a/arch/x86/include/asm/atomic.h +++ b/arch/x86/include/asm/atomic.h @@ -182,6 +182,21 @@ static inline int atomic_xchg(atomic_t *v, int new) return xchg(&v->counter, new); } +#define ATOMIC_OP(op) \ +static inline void atomic_##op(int i, atomic_t *v) \ +{ \ + asm volatile(LOCK_PREFIX #op"l %1,%0" \ + : "+m" (v->counter) \ + : "ir" (i) \ + : "memory"); \ +} + +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + +#undef ATOMIC_OP + /** * __atomic_add_unless - add unless the number is already a given value * @v: pointer of type atomic_t @@ -219,16 +234,6 @@ static __always_inline short int atomic_inc_short(short int *v) return *v; } -/* These are x86-specific, used by some header files */ -#define atomic_clear_mask(mask, addr) \ - asm volatile(LOCK_PREFIX "andl %0,%1" \ - : : "r" (~(mask)), "m" (*(addr)) : "memory") - -#define atomic_set_mask(mask, addr) \ - asm volatile(LOCK_PREFIX "orl %0,%1" \ - : : "r" ((unsigned)(mask)), "m" (*(addr)) \ - : "memory") - #ifdef CONFIG_X86_32 # include #else diff --git a/arch/x86/include/asm/atomic64_32.h b/arch/x86/include/asm/atomic64_32.h index b154de75c90cb5..a11c30b77fb57d 100644 --- a/arch/x86/include/asm/atomic64_32.h +++ b/arch/x86/include/asm/atomic64_32.h @@ -313,4 +313,18 @@ static inline long long atomic64_dec_if_positive(atomic64_t *v) #undef alternative_atomic64 #undef __alternative_atomic64 +#define ATOMIC64_OP(op, c_op) \ +static inline void atomic64_##op(long long i, atomic64_t *v) \ +{ \ + long long old, c = 0; \ + while ((old = atomic64_cmpxchg(v, c, c c_op i)) != c) \ + c = old; \ +} + +ATOMIC64_OP(and, &) +ATOMIC64_OP(or, |) +ATOMIC64_OP(xor, ^) + +#undef ATOMIC64_OP + #endif /* _ASM_X86_ATOMIC64_32_H */ diff --git a/arch/x86/include/asm/atomic64_64.h b/arch/x86/include/asm/atomic64_64.h index b965f9e03f2a04..50e33eff58de7f 100644 --- a/arch/x86/include/asm/atomic64_64.h +++ b/arch/x86/include/asm/atomic64_64.h @@ -220,4 +220,19 @@ static inline long atomic64_dec_if_positive(atomic64_t *v) return dec; } +#define ATOMIC64_OP(op) \ +static inline void atomic64_##op(long i, atomic64_t *v) \ +{ \ + asm volatile(LOCK_PREFIX #op"q %1,%0" \ + : "+m" (v->counter) \ + : "er" (i) \ + : "memory"); \ +} + +ATOMIC64_OP(and) +ATOMIC64_OP(or) +ATOMIC64_OP(xor) + +#undef ATOMIC64_OP + #endif /* _ASM_X86_ATOMIC64_64_H */ diff --git a/arch/x86/include/asm/barrier.h b/arch/x86/include/asm/barrier.h index 818cb87882255e..0681d2532527f6 100644 --- a/arch/x86/include/asm/barrier.h +++ b/arch/x86/include/asm/barrier.h @@ -57,12 +57,12 @@ do { \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ ___p1; \ @@ -74,12 +74,12 @@ do { \ do { \ compiletime_assert_atomic_type(*p); \ barrier(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ barrier(); \ ___p1; \ diff --git a/arch/x86/include/asm/jump_label.h b/arch/x86/include/asm/jump_label.h index a4c1cf7e93f812..5daeca3d0f9e5b 100644 --- a/arch/x86/include/asm/jump_label.h +++ b/arch/x86/include/asm/jump_label.h @@ -16,15 +16,32 @@ # define STATIC_KEY_INIT_NOP GENERIC_NOP5_ATOMIC #endif -static __always_inline bool arch_static_branch(struct static_key *key) +static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { asm_volatile_goto("1:" ".byte " __stringify(STATIC_KEY_INIT_NOP) "\n\t" ".pushsection __jump_table, \"aw\" \n\t" _ASM_ALIGN "\n\t" - _ASM_PTR "1b, %l[l_yes], %c0 \n\t" + _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t" ".popsection \n\t" - : : "i" (key) : : l_yes); + : : "i" (key), "i" (branch) : : l_yes); + + return false; +l_yes: + return true; +} + +static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch) +{ + asm_volatile_goto("1:" + ".byte 0xe9\n\t .long %l[l_yes] - 2f\n\t" + "2:\n\t" + ".pushsection __jump_table, \"aw\" \n\t" + _ASM_ALIGN "\n\t" + _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t" + ".popsection \n\t" + : : "i" (key), "i" (branch) : : l_yes); + return false; l_yes: return true; diff --git a/arch/x86/include/asm/qrwlock.h b/arch/x86/include/asm/qrwlock.h index ae0e241e228b80..c537cbb038a768 100644 --- a/arch/x86/include/asm/qrwlock.h +++ b/arch/x86/include/asm/qrwlock.h @@ -2,16 +2,6 @@ #define _ASM_X86_QRWLOCK_H #include - -#ifndef CONFIG_X86_PPRO_FENCE -#define queue_write_unlock queue_write_unlock -static inline void queue_write_unlock(struct qrwlock *lock) -{ - barrier(); - ACCESS_ONCE(*(u8 *)&lock->cnts) = 0; -} -#endif - #include #endif /* _ASM_X86_QRWLOCK_H */ diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index 26d5a55a273610..e565e0e4d21698 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -45,7 +45,7 @@ static void __jump_label_transform(struct jump_entry *entry, const unsigned char default_nop[] = { STATIC_KEY_INIT_NOP }; const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5]; - if (type == JUMP_LABEL_ENABLE) { + if (type == JUMP_LABEL_JMP) { if (init) { /* * Jump label is enabled for the first time. diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 79055cf2c497e8..c8d52cb4cb6e8b 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -38,7 +38,7 @@ static int __read_mostly tsc_unstable; erroneous rdtsc usage on !cpu_has_tsc processors */ static int __read_mostly tsc_disabled = -1; -static struct static_key __use_tsc = STATIC_KEY_INIT; +static DEFINE_STATIC_KEY_FALSE(__use_tsc); int tsc_clocksource_reliable; @@ -274,7 +274,12 @@ static void set_cyc2ns_scale(unsigned long cpu_khz, int cpu) */ u64 native_sched_clock(void) { - u64 tsc_now; + if (static_branch_likely(&__use_tsc)) { + u64 tsc_now = rdtsc(); + + /* return the value in ns */ + return cycles_2_ns(tsc_now); + } /* * Fall back to jiffies if there's no TSC available: @@ -284,16 +289,9 @@ u64 native_sched_clock(void) * very important for it to be as fast as the platform * can achieve it. ) */ - if (!static_key_false(&__use_tsc)) { - /* No locking but a rare wrong value is not a big deal: */ - return (jiffies_64 - INITIAL_JIFFIES) * (1000000000 / HZ); - } - - /* read the Time Stamp Counter: */ - tsc_now = rdtsc(); - /* return the value in ns */ - return cycles_2_ns(tsc_now); + /* No locking but a rare wrong value is not a big deal: */ + return (jiffies_64 - INITIAL_JIFFIES) * (1000000000 / HZ); } /* @@ -1212,7 +1210,7 @@ void __init tsc_init(void) /* now allow native_sched_clock() to use rdtsc */ tsc_disabled = 0; - static_key_slow_inc(&__use_tsc); + static_branch_enable(&__use_tsc); if (!no_sched_irq_time) enable_sched_clock_irqtime(); diff --git a/arch/xtensa/configs/iss_defconfig b/arch/xtensa/configs/iss_defconfig index e4d193e7a30087..f3dfe0d921c275 100644 --- a/arch/xtensa/configs/iss_defconfig +++ b/arch/xtensa/configs/iss_defconfig @@ -616,7 +616,6 @@ CONFIG_SCHED_DEBUG=y # CONFIG_SLUB_DEBUG_ON is not set # CONFIG_SLUB_STATS is not set # CONFIG_DEBUG_RT_MUTEXES is not set -# CONFIG_RT_MUTEX_TESTER is not set # CONFIG_DEBUG_SPINLOCK is not set # CONFIG_DEBUG_MUTEXES is not set # CONFIG_DEBUG_SPINLOCK_SLEEP is not set diff --git a/arch/xtensa/include/asm/atomic.h b/arch/xtensa/include/asm/atomic.h index ebcd1f6fc8cb9f..93795d04730387 100644 --- a/arch/xtensa/include/asm/atomic.h +++ b/arch/xtensa/include/asm/atomic.h @@ -145,6 +145,10 @@ static inline int atomic_##op##_return(int i, atomic_t * v) \ ATOMIC_OPS(add) ATOMIC_OPS(sub) +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + #undef ATOMIC_OPS #undef ATOMIC_OP_RETURN #undef ATOMIC_OP @@ -250,75 +254,6 @@ static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u) return c; } - -static inline void atomic_clear_mask(unsigned int mask, atomic_t *v) -{ -#if XCHAL_HAVE_S32C1I - unsigned long tmp; - int result; - - __asm__ __volatile__( - "1: l32i %1, %3, 0\n" - " wsr %1, scompare1\n" - " and %0, %1, %2\n" - " s32c1i %0, %3, 0\n" - " bne %0, %1, 1b\n" - : "=&a" (result), "=&a" (tmp) - : "a" (~mask), "a" (v) - : "memory" - ); -#else - unsigned int all_f = -1; - unsigned int vval; - - __asm__ __volatile__( - " rsil a15,"__stringify(TOPLEVEL)"\n" - " l32i %0, %2, 0\n" - " xor %1, %4, %3\n" - " and %0, %0, %4\n" - " s32i %0, %2, 0\n" - " wsr a15, ps\n" - " rsync\n" - : "=&a" (vval), "=a" (mask) - : "a" (v), "a" (all_f), "1" (mask) - : "a15", "memory" - ); -#endif -} - -static inline void atomic_set_mask(unsigned int mask, atomic_t *v) -{ -#if XCHAL_HAVE_S32C1I - unsigned long tmp; - int result; - - __asm__ __volatile__( - "1: l32i %1, %3, 0\n" - " wsr %1, scompare1\n" - " or %0, %1, %2\n" - " s32c1i %0, %3, 0\n" - " bne %0, %1, 1b\n" - : "=&a" (result), "=&a" (tmp) - : "a" (mask), "a" (v) - : "memory" - ); -#else - unsigned int vval; - - __asm__ __volatile__( - " rsil a15,"__stringify(TOPLEVEL)"\n" - " l32i %0, %2, 0\n" - " or %0, %0, %1\n" - " s32i %0, %2, 0\n" - " wsr a15, ps\n" - " rsync\n" - : "=&a" (vval) - : "a" (mask), "a" (v) - : "a15", "memory" - ); -#endif -} - #endif /* __KERNEL__ */ #endif /* _XTENSA_ATOMIC_H */ diff --git a/block/bounce.c b/block/bounce.c index 2c310ea007ee36..0611aea1cfe9f0 100644 --- a/block/bounce.c +++ b/block/bounce.c @@ -177,26 +177,8 @@ static void bounce_end_io_read_isa(struct bio *bio) __bounce_end_io_read(bio, isa_page_pool); } -#ifdef CONFIG_NEED_BOUNCE_POOL -static int must_snapshot_stable_pages(struct request_queue *q, struct bio *bio) -{ - if (bio_data_dir(bio) != WRITE) - return 0; - - if (!bdi_cap_stable_pages_required(&q->backing_dev_info)) - return 0; - - return bio_flagged(bio, BIO_SNAP_STABLE); -} -#else -static int must_snapshot_stable_pages(struct request_queue *q, struct bio *bio) -{ - return 0; -} -#endif /* CONFIG_NEED_BOUNCE_POOL */ - static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig, - mempool_t *pool, int force) + mempool_t *pool) { struct bio *bio; int rw = bio_data_dir(*bio_orig); @@ -204,8 +186,6 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig, struct bvec_iter iter; unsigned i; - if (force) - goto bounce; bio_for_each_segment(from, *bio_orig, iter) if (page_to_pfn(from.bv_page) > queue_bounce_pfn(q)) goto bounce; @@ -217,7 +197,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig, bio_for_each_segment_all(to, bio, i) { struct page *page = to->bv_page; - if (page_to_pfn(page) <= queue_bounce_pfn(q) && !force) + if (page_to_pfn(page) <= queue_bounce_pfn(q)) continue; to->bv_page = mempool_alloc(pool, q->bounce_gfp); @@ -255,7 +235,6 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig, void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig) { - int must_bounce; mempool_t *pool; /* @@ -264,15 +243,13 @@ void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig) if (!bio_has_data(*bio_orig)) return; - must_bounce = must_snapshot_stable_pages(q, *bio_orig); - /* * for non-isa bounce case, just check if the bounce pfn is equal * to or bigger than the highest pfn in the system -- in that case, * don't waste time iterating over bio segments */ if (!(q->bounce_gfp & GFP_DMA)) { - if (queue_bounce_pfn(q) >= blk_max_pfn && !must_bounce) + if (queue_bounce_pfn(q) >= blk_max_pfn) return; pool = page_pool; } else { @@ -283,7 +260,7 @@ void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig) /* * slow path */ - __blk_queue_bounce(q, bio_orig, pool, must_bounce); + __blk_queue_bounce(q, bio_orig, pool); } EXPORT_SYMBOL(blk_queue_bounce); diff --git a/drivers/Kconfig b/drivers/Kconfig index 4e2e6aaf0b88a9..46b4a8e0f8598b 100644 --- a/drivers/Kconfig +++ b/drivers/Kconfig @@ -176,6 +176,8 @@ source "drivers/powercap/Kconfig" source "drivers/mcb/Kconfig" +source "drivers/perf/Kconfig" + source "drivers/ras/Kconfig" source "drivers/thunderbolt/Kconfig" diff --git a/drivers/Makefile b/drivers/Makefile index 4c270f5414f0b0..b250b36b54f266 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -11,7 +11,7 @@ obj-y += bus/ obj-$(CONFIG_GENERIC_PHY) += phy/ # GPIO must come after pinctrl as gpios may need to mux pins etc -obj-y += pinctrl/ +obj-$(CONFIG_PINCTRL) += pinctrl/ obj-y += gpio/ obj-y += pwm/ obj-$(CONFIG_PCI) += pci/ @@ -161,6 +161,7 @@ obj-$(CONFIG_NTB) += ntb/ obj-$(CONFIG_FMC) += fmc/ obj-$(CONFIG_POWERCAP) += powercap/ obj-$(CONFIG_MCB) += mcb/ +obj-$(CONFIG_PERF_EVENTS) += perf/ obj-$(CONFIG_RAS) += ras/ obj-$(CONFIG_THUNDERBOLT) += thunderbolt/ obj-$(CONFIG_CORESIGHT) += hwtracing/coresight/ diff --git a/drivers/ata/pata_rb532_cf.c b/drivers/ata/pata_rb532_cf.c index 6d08446b877c97..12fe0f3bb7e9b0 100644 --- a/drivers/ata/pata_rb532_cf.c +++ b/drivers/ata/pata_rb532_cf.c @@ -27,12 +27,11 @@ #include #include #include +#include #include #include -#include - #define DRV_NAME "pata-rb532-cf" #define DRV_VERSION "0.1.0" #define DRV_DESC "PATA driver for RouterBOARD 532 Compact Flash" diff --git a/drivers/bus/mips_cdmm.c b/drivers/bus/mips_cdmm.c index ab3bde16ecb444..1c543effe062f8 100644 --- a/drivers/bus/mips_cdmm.c +++ b/drivers/bus/mips_cdmm.c @@ -331,6 +331,18 @@ static phys_addr_t mips_cdmm_cur_base(void) << MIPS_CDMMBASE_ADDR_START; } +/** + * mips_cdmm_phys_base() - Choose a physical base address for CDMM region. + * + * Picking a suitable physical address at which to map the CDMM region is + * platform specific, so this weak function can be overridden by platform + * code to pick a suitable value if none is configured by the bootloader. + */ +phys_addr_t __weak mips_cdmm_phys_base(void) +{ + return 0; +} + /** * mips_cdmm_setup() - Ensure the CDMM bus is initialised and usable. * @bus: Pointer to bus information for current CPU. @@ -368,7 +380,7 @@ static int mips_cdmm_setup(struct mips_cdmm_bus *bus) if (!bus->phys) bus->phys = mips_cdmm_cur_base(); /* Otherwise, ask platform code for suggestions */ - if (!bus->phys && mips_cdmm_phys_base) + if (!bus->phys) bus->phys = mips_cdmm_phys_base(); /* Otherwise, copy what other CPUs have done */ if (!bus->phys) diff --git a/drivers/clk/rockchip/clk-rk3288.c b/drivers/clk/rockchip/clk-rk3288.c index 0df5bae9ddbf06..9040878e3e2b6c 100644 --- a/drivers/clk/rockchip/clk-rk3288.c +++ b/drivers/clk/rockchip/clk-rk3288.c @@ -780,6 +780,7 @@ static const char *const rk3288_critical_clocks[] __initconst = { "aclk_cpu", "aclk_peri", "hclk_peri", + "pclk_pd_pmu", }; #ifdef CONFIG_PM_SLEEP diff --git a/drivers/clocksource/Kconfig b/drivers/clocksource/Kconfig index c03f04d82c6a42..552c9b134cc52a 100644 --- a/drivers/clocksource/Kconfig +++ b/drivers/clocksource/Kconfig @@ -111,6 +111,10 @@ config CLKSRC_LPC32XX select CLKSRC_MMIO select CLKSRC_OF +config CLKSRC_PISTACHIO + bool + select CLKSRC_OF + config CLKSRC_STM32 bool "Clocksource for STM32 SoCs" if !ARCH_STM32 depends on OF && ARM && (ARCH_STM32 || COMPILE_TEST) diff --git a/drivers/clocksource/Makefile b/drivers/clocksource/Makefile index f228354961ca6f..066337e247379d 100644 --- a/drivers/clocksource/Makefile +++ b/drivers/clocksource/Makefile @@ -44,6 +44,7 @@ obj-$(CONFIG_FSL_FTM_TIMER) += fsl_ftm_timer.o obj-$(CONFIG_VF_PIT_TIMER) += vf_pit_timer.o obj-$(CONFIG_CLKSRC_QCOM) += qcom-timer.o obj-$(CONFIG_MTK_TIMER) += mtk_timer.o +obj-$(CONFIG_CLKSRC_PISTACHIO) += time-pistachio.o obj-$(CONFIG_ARM_ARCH_TIMER) += arm_arch_timer.o obj-$(CONFIG_ARM_GLOBAL_TIMER) += arm_global_timer.o diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c index c3810b61c815f7..02a1945e5093f7 100644 --- a/drivers/clocksource/mips-gic-timer.c +++ b/drivers/clocksource/mips-gic-timer.c @@ -72,6 +72,13 @@ static void gic_clockevent_cpu_exit(struct clock_event_device *cd) disable_percpu_irq(gic_timer_irq); } +static void gic_update_frequency(void *data) +{ + unsigned long rate = (unsigned long)data; + + clockevents_update_freq(this_cpu_ptr(&gic_clockevent_device), rate); +} + static int gic_cpu_notifier(struct notifier_block *nb, unsigned long action, void *data) { @@ -87,18 +94,40 @@ static int gic_cpu_notifier(struct notifier_block *nb, unsigned long action, return NOTIFY_OK; } +static int gic_clk_notifier(struct notifier_block *nb, unsigned long action, + void *data) +{ + struct clk_notifier_data *cnd = data; + + if (action == POST_RATE_CHANGE) + on_each_cpu(gic_update_frequency, (void *)cnd->new_rate, 1); + + return NOTIFY_OK; +} + + static struct notifier_block gic_cpu_nb = { .notifier_call = gic_cpu_notifier, }; +static struct notifier_block gic_clk_nb = { + .notifier_call = gic_clk_notifier, +}; + static int gic_clockevent_init(void) { + int ret; + if (!cpu_has_counter || !gic_frequency) return -ENXIO; - setup_percpu_irq(gic_timer_irq, &gic_compare_irqaction); + ret = setup_percpu_irq(gic_timer_irq, &gic_compare_irqaction); + if (ret < 0) + return ret; - register_cpu_notifier(&gic_cpu_nb); + ret = register_cpu_notifier(&gic_cpu_nb); + if (ret < 0) + pr_warn("GIC: Unable to register CPU notifier\n"); gic_clockevent_cpu_init(this_cpu_ptr(&gic_clockevent_device)); @@ -118,18 +147,17 @@ static struct clocksource gic_clocksource = { static void __init __gic_clocksource_init(void) { + int ret; + /* Set clocksource mask. */ gic_clocksource.mask = CLOCKSOURCE_MASK(gic_get_count_width()); /* Calculate a somewhat reasonable rating value. */ gic_clocksource.rating = 200 + gic_frequency / 10000000; - clocksource_register_hz(&gic_clocksource, gic_frequency); - - gic_clockevent_init(); - - /* And finally start the counter */ - gic_start_count(); + ret = clocksource_register_hz(&gic_clocksource, gic_frequency); + if (ret < 0) + pr_warn("GIC: Unable to register clocksource\n"); } void __init gic_clocksource_init(unsigned int frequency) @@ -139,11 +167,16 @@ void __init gic_clocksource_init(unsigned int frequency) GIC_LOCAL_TO_HWIRQ(GIC_LOCAL_INT_COMPARE); __gic_clocksource_init(); + gic_clockevent_init(); + + /* And finally start the counter */ + gic_start_count(); } static void __init gic_clocksource_of_init(struct device_node *node) { struct clk *clk; + int ret; if (WARN_ON(!gic_present || !node->parent || !of_device_is_compatible(node->parent, "mti,gic"))) @@ -151,8 +184,13 @@ static void __init gic_clocksource_of_init(struct device_node *node) clk = of_clk_get(node, 0); if (!IS_ERR(clk)) { + if (clk_prepare_enable(clk) < 0) { + pr_err("GIC failed to enable clock\n"); + clk_put(clk); + return; + } + gic_frequency = clk_get_rate(clk); - clk_put(clk); } else if (of_property_read_u32(node, "clock-frequency", &gic_frequency)) { pr_err("GIC frequency not specified.\n"); @@ -165,6 +203,15 @@ static void __init gic_clocksource_of_init(struct device_node *node) } __gic_clocksource_init(); + + ret = gic_clockevent_init(); + if (!ret && !IS_ERR(clk)) { + if (clk_notifier_register(clk, &gic_clk_nb) < 0) + pr_warn("GIC: Unable to register clock notifier\n"); + } + + /* And finally start the counter */ + gic_start_count(); } CLOCKSOURCE_OF_DECLARE(mips_gic_timer, "mti,gic-timer", gic_clocksource_of_init); diff --git a/drivers/clocksource/time-pistachio.c b/drivers/clocksource/time-pistachio.c new file mode 100644 index 00000000000000..18d4266c2986c6 --- /dev/null +++ b/drivers/clocksource/time-pistachio.c @@ -0,0 +1,217 @@ +/* + * Pistachio clocksource based on general-purpose timers + * + * Copyright (C) 2015 Imagination Technologies + * + * This file is subject to the terms and conditions of the GNU General Public + * License. See the file "COPYING" in the main directory of this archive + * for more details. + */ + +#define pr_fmt(fmt) "%s: " fmt, __func__ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* Top level reg */ +#define CR_TIMER_CTRL_CFG 0x00 +#define TIMER_ME_GLOBAL BIT(0) +#define CR_TIMER_REV 0x10 + +/* Timer specific registers */ +#define TIMER_CFG 0x20 +#define TIMER_ME_LOCAL BIT(0) +#define TIMER_RELOAD_VALUE 0x24 +#define TIMER_CURRENT_VALUE 0x28 +#define TIMER_CURRENT_OVERFLOW_VALUE 0x2C +#define TIMER_IRQ_STATUS 0x30 +#define TIMER_IRQ_CLEAR 0x34 +#define TIMER_IRQ_MASK 0x38 + +#define PERIP_TIMER_CONTROL 0x90 + +/* Timer specific configuration Values */ +#define RELOAD_VALUE 0xffffffff + +struct pistachio_clocksource { + void __iomem *base; + raw_spinlock_t lock; + struct clocksource cs; +}; + +static struct pistachio_clocksource pcs_gpt; + +#define to_pistachio_clocksource(cs) \ + container_of(cs, struct pistachio_clocksource, cs) + +static inline u32 gpt_readl(void __iomem *base, u32 offset, u32 gpt_id) +{ + return readl(base + 0x20 * gpt_id + offset); +} + +static inline void gpt_writel(void __iomem *base, u32 value, u32 offset, + u32 gpt_id) +{ + writel(value, base + 0x20 * gpt_id + offset); +} + +static cycle_t pistachio_clocksource_read_cycles(struct clocksource *cs) +{ + struct pistachio_clocksource *pcs = to_pistachio_clocksource(cs); + u32 counter, overflw; + unsigned long flags; + + /* + * The counter value is only refreshed after the overflow value is read. + * And they must be read in strict order, hence raw spin lock added. + */ + + raw_spin_lock_irqsave(&pcs->lock, flags); + overflw = gpt_readl(pcs->base, TIMER_CURRENT_OVERFLOW_VALUE, 0); + counter = gpt_readl(pcs->base, TIMER_CURRENT_VALUE, 0); + raw_spin_unlock_irqrestore(&pcs->lock, flags); + + return ~(cycle_t)counter; +} + +static u64 notrace pistachio_read_sched_clock(void) +{ + return pistachio_clocksource_read_cycles(&pcs_gpt.cs); +} + +static void pistachio_clksrc_set_mode(struct clocksource *cs, int timeridx, + int enable) +{ + struct pistachio_clocksource *pcs = to_pistachio_clocksource(cs); + u32 val; + + val = gpt_readl(pcs->base, TIMER_CFG, timeridx); + if (enable) + val |= TIMER_ME_LOCAL; + else + val &= ~TIMER_ME_LOCAL; + + gpt_writel(pcs->base, val, TIMER_CFG, timeridx); +} + +static void pistachio_clksrc_enable(struct clocksource *cs, int timeridx) +{ + struct pistachio_clocksource *pcs = to_pistachio_clocksource(cs); + + /* Disable GPT local before loading reload value */ + pistachio_clksrc_set_mode(cs, timeridx, false); + gpt_writel(pcs->base, RELOAD_VALUE, TIMER_RELOAD_VALUE, timeridx); + pistachio_clksrc_set_mode(cs, timeridx, true); +} + +static void pistachio_clksrc_disable(struct clocksource *cs, int timeridx) +{ + /* Disable GPT local */ + pistachio_clksrc_set_mode(cs, timeridx, false); +} + +static int pistachio_clocksource_enable(struct clocksource *cs) +{ + pistachio_clksrc_enable(cs, 0); + return 0; +} + +static void pistachio_clocksource_disable(struct clocksource *cs) +{ + pistachio_clksrc_disable(cs, 0); +} + +/* Desirable clock source for pistachio platform */ +static struct pistachio_clocksource pcs_gpt = { + .cs = { + .name = "gptimer", + .rating = 300, + .enable = pistachio_clocksource_enable, + .disable = pistachio_clocksource_disable, + .read = pistachio_clocksource_read_cycles, + .mask = CLOCKSOURCE_MASK(32), + .flags = CLOCK_SOURCE_IS_CONTINUOUS | + CLOCK_SOURCE_SUSPEND_NONSTOP, + }, +}; + +static void __init pistachio_clksrc_of_init(struct device_node *node) +{ + struct clk *sys_clk, *fast_clk; + struct regmap *periph_regs; + unsigned long rate; + int ret; + + pcs_gpt.base = of_iomap(node, 0); + if (!pcs_gpt.base) { + pr_err("cannot iomap\n"); + return; + } + + periph_regs = syscon_regmap_lookup_by_phandle(node, "img,cr-periph"); + if (IS_ERR(periph_regs)) { + pr_err("cannot get peripheral regmap (%lu)\n", + PTR_ERR(periph_regs)); + return; + } + + /* Switch to using the fast counter clock */ + ret = regmap_update_bits(periph_regs, PERIP_TIMER_CONTROL, + 0xf, 0x0); + if (ret) + return; + + sys_clk = of_clk_get_by_name(node, "sys"); + if (IS_ERR(sys_clk)) { + pr_err("clock get failed (%lu)\n", PTR_ERR(sys_clk)); + return; + } + + fast_clk = of_clk_get_by_name(node, "fast"); + if (IS_ERR(fast_clk)) { + pr_err("clock get failed (%lu)\n", PTR_ERR(fast_clk)); + return; + } + + ret = clk_prepare_enable(sys_clk); + if (ret < 0) { + pr_err("failed to enable clock (%d)\n", ret); + return; + } + + ret = clk_prepare_enable(fast_clk); + if (ret < 0) { + pr_err("failed to enable clock (%d)\n", ret); + clk_disable_unprepare(sys_clk); + return; + } + + rate = clk_get_rate(fast_clk); + + /* Disable irq's for clocksource usage */ + gpt_writel(&pcs_gpt.base, 0, TIMER_IRQ_MASK, 0); + gpt_writel(&pcs_gpt.base, 0, TIMER_IRQ_MASK, 1); + gpt_writel(&pcs_gpt.base, 0, TIMER_IRQ_MASK, 2); + gpt_writel(&pcs_gpt.base, 0, TIMER_IRQ_MASK, 3); + + /* Enable timer block */ + writel(TIMER_ME_GLOBAL, pcs_gpt.base); + + raw_spin_lock_init(&pcs_gpt.lock); + sched_clock_register(pistachio_read_sched_clock, 32, rate); + clocksource_register_hz(&pcs_gpt.cs, rate); +} +CLOCKSOURCE_OF_DECLARE(pistachio_gptimer, "img,pistachio-gptimer", + pistachio_clksrc_of_init); diff --git a/drivers/cpuidle/cpuidle-calxeda.c b/drivers/cpuidle/cpuidle-calxeda.c index c13feec89ea1d6..ea9728fde9b3c8 100644 --- a/drivers/cpuidle/cpuidle-calxeda.c +++ b/drivers/cpuidle/cpuidle-calxeda.c @@ -25,16 +25,21 @@ #include #include #include +#include + #include #include -#include + +#include + +#define CALXEDA_IDLE_PARAM \ + ((0 << PSCI_0_2_POWER_STATE_ID_SHIFT) | \ + (0 << PSCI_0_2_POWER_STATE_AFFL_SHIFT) | \ + (PSCI_POWER_STATE_TYPE_POWER_DOWN << PSCI_0_2_POWER_STATE_TYPE_SHIFT)) static int calxeda_idle_finish(unsigned long val) { - const struct psci_power_state ps = { - .type = PSCI_POWER_STATE_TYPE_POWER_DOWN, - }; - return psci_ops.cpu_suspend(ps, __pa(cpu_resume)); + return psci_ops.cpu_suspend(CALXEDA_IDLE_PARAM, __pa(cpu_resume)); } static int calxeda_pwrdown_idle(struct cpuidle_device *dev, diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index 99c69a3205c469..d8de6a8dd4de15 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -5,6 +5,9 @@ menu "Firmware Drivers" +config ARM_PSCI_FW + bool + config EDD tristate "BIOS Enhanced Disk Drive calls determine boot disk" depends on X86 diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile index 4a4b897f931433..000830fc6707e8 100644 --- a/drivers/firmware/Makefile +++ b/drivers/firmware/Makefile @@ -1,6 +1,7 @@ # # Makefile for the linux kernel. # +obj-$(CONFIG_ARM_PSCI_FW) += psci.o obj-$(CONFIG_DMI) += dmi_scan.o obj-$(CONFIG_DMI_SYSFS) += dmi-sysfs.o obj-$(CONFIG_EDD) += edd.o diff --git a/drivers/firmware/psci.c b/drivers/firmware/psci.c new file mode 100644 index 00000000000000..42700f09a8c584 --- /dev/null +++ b/drivers/firmware/psci.c @@ -0,0 +1,382 @@ +/* + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * Copyright (C) 2015 ARM Limited + */ + +#define pr_fmt(fmt) "psci: " fmt + +#include +#include +#include +#include +#include +#include +#include + +#include + +#include +#include +#include + +/* + * While a 64-bit OS can make calls with SMC32 calling conventions, for some + * calls it is necessary to use SMC64 to pass or return 64-bit values. For such + * calls PSCI_0_2_FN_NATIVE(x) will choose the appropriate (native-width) + * function ID. + */ +#ifdef CONFIG_64BIT +#define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN64_##name +#else +#define PSCI_0_2_FN_NATIVE(name) PSCI_0_2_FN_##name +#endif + +/* + * The CPU any Trusted OS is resident on. The trusted OS may reject CPU_OFF + * calls to its resident CPU, so we must avoid issuing those. We never migrate + * a Trusted OS even if it claims to be capable of migration -- doing so will + * require cooperation with a Trusted OS driver. + */ +static int resident_cpu = -1; + +bool psci_tos_resident_on(int cpu) +{ + return cpu == resident_cpu; +} + +struct psci_operations psci_ops; + +typedef unsigned long (psci_fn)(unsigned long, unsigned long, + unsigned long, unsigned long); +asmlinkage psci_fn __invoke_psci_fn_hvc; +asmlinkage psci_fn __invoke_psci_fn_smc; +static psci_fn *invoke_psci_fn; + +enum psci_function { + PSCI_FN_CPU_SUSPEND, + PSCI_FN_CPU_ON, + PSCI_FN_CPU_OFF, + PSCI_FN_MIGRATE, + PSCI_FN_MAX, +}; + +static u32 psci_function_id[PSCI_FN_MAX]; + +static int psci_to_linux_errno(int errno) +{ + switch (errno) { + case PSCI_RET_SUCCESS: + return 0; + case PSCI_RET_NOT_SUPPORTED: + return -EOPNOTSUPP; + case PSCI_RET_INVALID_PARAMS: + return -EINVAL; + case PSCI_RET_DENIED: + return -EPERM; + }; + + return -EINVAL; +} + +static u32 psci_get_version(void) +{ + return invoke_psci_fn(PSCI_0_2_FN_PSCI_VERSION, 0, 0, 0); +} + +static int psci_cpu_suspend(u32 state, unsigned long entry_point) +{ + int err; + u32 fn; + + fn = psci_function_id[PSCI_FN_CPU_SUSPEND]; + err = invoke_psci_fn(fn, state, entry_point, 0); + return psci_to_linux_errno(err); +} + +static int psci_cpu_off(u32 state) +{ + int err; + u32 fn; + + fn = psci_function_id[PSCI_FN_CPU_OFF]; + err = invoke_psci_fn(fn, state, 0, 0); + return psci_to_linux_errno(err); +} + +static int psci_cpu_on(unsigned long cpuid, unsigned long entry_point) +{ + int err; + u32 fn; + + fn = psci_function_id[PSCI_FN_CPU_ON]; + err = invoke_psci_fn(fn, cpuid, entry_point, 0); + return psci_to_linux_errno(err); +} + +static int psci_migrate(unsigned long cpuid) +{ + int err; + u32 fn; + + fn = psci_function_id[PSCI_FN_MIGRATE]; + err = invoke_psci_fn(fn, cpuid, 0, 0); + return psci_to_linux_errno(err); +} + +static int psci_affinity_info(unsigned long target_affinity, + unsigned long lowest_affinity_level) +{ + return invoke_psci_fn(PSCI_0_2_FN_NATIVE(AFFINITY_INFO), + target_affinity, lowest_affinity_level, 0); +} + +static int psci_migrate_info_type(void) +{ + return invoke_psci_fn(PSCI_0_2_FN_MIGRATE_INFO_TYPE, 0, 0, 0); +} + +static unsigned long psci_migrate_info_up_cpu(void) +{ + return invoke_psci_fn(PSCI_0_2_FN_NATIVE(MIGRATE_INFO_UP_CPU), + 0, 0, 0); +} + +static int get_set_conduit_method(struct device_node *np) +{ + const char *method; + + pr_info("probing for conduit method from DT.\n"); + + if (of_property_read_string(np, "method", &method)) { + pr_warn("missing \"method\" property\n"); + return -ENXIO; + } + + if (!strcmp("hvc", method)) { + invoke_psci_fn = __invoke_psci_fn_hvc; + } else if (!strcmp("smc", method)) { + invoke_psci_fn = __invoke_psci_fn_smc; + } else { + pr_warn("invalid \"method\" property: %s\n", method); + return -EINVAL; + } + return 0; +} + +static void psci_sys_reset(enum reboot_mode reboot_mode, const char *cmd) +{ + invoke_psci_fn(PSCI_0_2_FN_SYSTEM_RESET, 0, 0, 0); +} + +static void psci_sys_poweroff(void) +{ + invoke_psci_fn(PSCI_0_2_FN_SYSTEM_OFF, 0, 0, 0); +} + +/* + * Detect the presence of a resident Trusted OS which may cause CPU_OFF to + * return DENIED (which would be fatal). + */ +static void __init psci_init_migrate(void) +{ + unsigned long cpuid; + int type, cpu = -1; + + type = psci_ops.migrate_info_type(); + + if (type == PSCI_0_2_TOS_MP) { + pr_info("Trusted OS migration not required\n"); + return; + } + + if (type == PSCI_RET_NOT_SUPPORTED) { + pr_info("MIGRATE_INFO_TYPE not supported.\n"); + return; + } + + if (type != PSCI_0_2_TOS_UP_MIGRATE && + type != PSCI_0_2_TOS_UP_NO_MIGRATE) { + pr_err("MIGRATE_INFO_TYPE returned unknown type (%d)\n", type); + return; + } + + cpuid = psci_migrate_info_up_cpu(); + if (cpuid & ~MPIDR_HWID_BITMASK) { + pr_warn("MIGRATE_INFO_UP_CPU reported invalid physical ID (0x%lx)\n", + cpuid); + return; + } + + cpu = get_logical_index(cpuid); + resident_cpu = cpu >= 0 ? cpu : -1; + + pr_info("Trusted OS resident on physical CPU 0x%lx\n", cpuid); +} + +static void __init psci_0_2_set_functions(void) +{ + pr_info("Using standard PSCI v0.2 function IDs\n"); + psci_function_id[PSCI_FN_CPU_SUSPEND] = PSCI_0_2_FN_NATIVE(CPU_SUSPEND); + psci_ops.cpu_suspend = psci_cpu_suspend; + + psci_function_id[PSCI_FN_CPU_OFF] = PSCI_0_2_FN_CPU_OFF; + psci_ops.cpu_off = psci_cpu_off; + + psci_function_id[PSCI_FN_CPU_ON] = PSCI_0_2_FN_NATIVE(CPU_ON); + psci_ops.cpu_on = psci_cpu_on; + + psci_function_id[PSCI_FN_MIGRATE] = PSCI_0_2_FN_NATIVE(MIGRATE); + psci_ops.migrate = psci_migrate; + + psci_ops.affinity_info = psci_affinity_info; + + psci_ops.migrate_info_type = psci_migrate_info_type; + + arm_pm_restart = psci_sys_reset; + + pm_power_off = psci_sys_poweroff; +} + +/* + * Probe function for PSCI firmware versions >= 0.2 + */ +static int __init psci_probe(void) +{ + u32 ver = psci_get_version(); + + pr_info("PSCIv%d.%d detected in firmware.\n", + PSCI_VERSION_MAJOR(ver), + PSCI_VERSION_MINOR(ver)); + + if (PSCI_VERSION_MAJOR(ver) == 0 && PSCI_VERSION_MINOR(ver) < 2) { + pr_err("Conflicting PSCI version detected.\n"); + return -EINVAL; + } + + psci_0_2_set_functions(); + + psci_init_migrate(); + + return 0; +} + +typedef int (*psci_initcall_t)(const struct device_node *); + +/* + * PSCI init function for PSCI versions >=0.2 + * + * Probe based on PSCI PSCI_VERSION function + */ +static int __init psci_0_2_init(struct device_node *np) +{ + int err; + + err = get_set_conduit_method(np); + + if (err) + goto out_put_node; + /* + * Starting with v0.2, the PSCI specification introduced a call + * (PSCI_VERSION) that allows probing the firmware version, so + * that PSCI function IDs and version specific initialization + * can be carried out according to the specific version reported + * by firmware + */ + err = psci_probe(); + +out_put_node: + of_node_put(np); + return err; +} + +/* + * PSCI < v0.2 get PSCI Function IDs via DT. + */ +static int __init psci_0_1_init(struct device_node *np) +{ + u32 id; + int err; + + err = get_set_conduit_method(np); + + if (err) + goto out_put_node; + + pr_info("Using PSCI v0.1 Function IDs from DT\n"); + + if (!of_property_read_u32(np, "cpu_suspend", &id)) { + psci_function_id[PSCI_FN_CPU_SUSPEND] = id; + psci_ops.cpu_suspend = psci_cpu_suspend; + } + + if (!of_property_read_u32(np, "cpu_off", &id)) { + psci_function_id[PSCI_FN_CPU_OFF] = id; + psci_ops.cpu_off = psci_cpu_off; + } + + if (!of_property_read_u32(np, "cpu_on", &id)) { + psci_function_id[PSCI_FN_CPU_ON] = id; + psci_ops.cpu_on = psci_cpu_on; + } + + if (!of_property_read_u32(np, "migrate", &id)) { + psci_function_id[PSCI_FN_MIGRATE] = id; + psci_ops.migrate = psci_migrate; + } + +out_put_node: + of_node_put(np); + return err; +} + +static const struct of_device_id const psci_of_match[] __initconst = { + { .compatible = "arm,psci", .data = psci_0_1_init}, + { .compatible = "arm,psci-0.2", .data = psci_0_2_init}, + {}, +}; + +int __init psci_dt_init(void) +{ + struct device_node *np; + const struct of_device_id *matched_np; + psci_initcall_t init_fn; + + np = of_find_matching_node_and_match(NULL, psci_of_match, &matched_np); + + if (!np) + return -ENODEV; + + init_fn = (psci_initcall_t)matched_np->data; + return init_fn(np); +} + +#ifdef CONFIG_ACPI +/* + * We use PSCI 0.2+ when ACPI is deployed on ARM64 and it's + * explicitly clarified in SBBR + */ +int __init psci_acpi_init(void) +{ + if (!acpi_psci_present()) { + pr_info("is not implemented in ACPI.\n"); + return -EOPNOTSUPP; + } + + pr_info("probing for conduit method from ACPI.\n"); + + if (acpi_psci_use_hvc()) + invoke_psci_fn = __invoke_psci_fn_hvc; + else + invoke_psci_fn = __invoke_psci_fn_smc; + + return psci_probe(); +} +#endif diff --git a/drivers/firmware/qcom_scm-32.c b/drivers/firmware/qcom_scm-32.c index 1bd6f9c3433140..29e6850665eb34 100644 --- a/drivers/firmware/qcom_scm-32.c +++ b/drivers/firmware/qcom_scm-32.c @@ -24,7 +24,6 @@ #include #include -#include #include #include "qcom_scm.h" @@ -219,8 +218,7 @@ static int __qcom_scm_call(const struct qcom_scm_command *cmd) * Flush the command buffer so that the secure world sees * the correct data. */ - __cpuc_flush_dcache_area((void *)cmd, cmd->len); - outer_flush_range(cmd_addr, cmd_addr + cmd->len); + secure_flush_area(cmd, cmd->len); ret = smc(cmd_addr); if (ret < 0) diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig index 8f1fe739c985ef..b4fc9e4d24c6b1 100644 --- a/drivers/gpio/Kconfig +++ b/drivers/gpio/Kconfig @@ -113,7 +113,6 @@ config GPIO_74XX_MMIO config GPIO_ALTERA tristate "Altera GPIO" depends on OF_GPIO - select GPIO_GENERIC select GPIOLIB_IRQCHIP help Say Y or M here to build support for the Altera PIO device. @@ -131,6 +130,7 @@ config GPIO_BRCMSTB default y if ARCH_BRCMSTB depends on OF_GPIO && (ARCH_BRCMSTB || COMPILE_TEST) select GPIO_GENERIC + select GPIOLIB_IRQCHIP help Say yes here to enable GPIO support for Broadcom STB (BCM7XXX) SoCs. @@ -172,6 +172,7 @@ config GPIO_ETRAXFS depends on CRIS || COMPILE_TEST depends on OF select GPIO_GENERIC + select GPIOLIB_IRQCHIP help Say yes here to support the GPIO controller on Axis ETRAX FS SoCs. @@ -308,7 +309,6 @@ config GPIO_MVEBU def_bool y depends on PLAT_ORION depends on OF - select GPIO_GENERIC select GENERIC_IRQ_CHIP config GPIO_MXC @@ -1005,6 +1005,12 @@ config GPIO_MC33880 SPI driver for Freescale MC33880 high-side/low-side switch. This provides GPIO interface supporting inputs and outputs. +config GPIO_ZX + bool "ZTE ZX GPIO support" + select GPIOLIB_IRQCHIP + help + Say yes here to support the GPIO device on ZTE ZX SoCs. + endmenu menu "USB GPIO expanders" diff --git a/drivers/gpio/Makefile b/drivers/gpio/Makefile index f82cd678ce086e..f79a7c482a993c 100644 --- a/drivers/gpio/Makefile +++ b/drivers/gpio/Makefile @@ -20,6 +20,7 @@ obj-$(CONFIG_GPIO_ADP5588) += gpio-adp5588.o obj-$(CONFIG_GPIO_ALTERA) += gpio-altera.o obj-$(CONFIG_GPIO_AMD8111) += gpio-amd8111.o obj-$(CONFIG_GPIO_ARIZONA) += gpio-arizona.o +obj-$(CONFIG_ATH79) += gpio-ath79.o obj-$(CONFIG_GPIO_BCM_KONA) += gpio-bcm-kona.o obj-$(CONFIG_GPIO_BRCMSTB) += gpio-brcmstb.o obj-$(CONFIG_GPIO_BT8XX) += gpio-bt8xx.o @@ -116,3 +117,4 @@ obj-$(CONFIG_GPIO_XLP) += gpio-xlp.o obj-$(CONFIG_GPIO_XTENSA) += gpio-xtensa.o obj-$(CONFIG_GPIO_ZEVIO) += gpio-zevio.o obj-$(CONFIG_GPIO_ZYNQ) += gpio-zynq.o +obj-$(CONFIG_GPIO_ZX) += gpio-zx.o diff --git a/drivers/gpio/devres.c b/drivers/gpio/devres.c index 07ba82317eceb0..903fcf4d04a06e 100644 --- a/drivers/gpio/devres.c +++ b/drivers/gpio/devres.c @@ -59,13 +59,13 @@ static int devm_gpiod_match_array(struct device *dev, void *res, void *data) * automatically disposed on driver detach. See gpiod_get() for detailed * information about behavior and return values. */ -struct gpio_desc *__must_check __devm_gpiod_get(struct device *dev, +struct gpio_desc *__must_check devm_gpiod_get(struct device *dev, const char *con_id, enum gpiod_flags flags) { return devm_gpiod_get_index(dev, con_id, 0, flags); } -EXPORT_SYMBOL(__devm_gpiod_get); +EXPORT_SYMBOL(devm_gpiod_get); /** * devm_gpiod_get_optional - Resource-managed gpiod_get_optional() @@ -77,13 +77,13 @@ EXPORT_SYMBOL(__devm_gpiod_get); * are automatically disposed on driver detach. See gpiod_get_optional() for * detailed information about behavior and return values. */ -struct gpio_desc *__must_check __devm_gpiod_get_optional(struct device *dev, +struct gpio_desc *__must_check devm_gpiod_get_optional(struct device *dev, const char *con_id, enum gpiod_flags flags) { return devm_gpiod_get_index_optional(dev, con_id, 0, flags); } -EXPORT_SYMBOL(__devm_gpiod_get_optional); +EXPORT_SYMBOL(devm_gpiod_get_optional); /** * devm_gpiod_get_index - Resource-managed gpiod_get_index() @@ -96,7 +96,7 @@ EXPORT_SYMBOL(__devm_gpiod_get_optional); * automatically disposed on driver detach. See gpiod_get_index() for detailed * information about behavior and return values. */ -struct gpio_desc *__must_check __devm_gpiod_get_index(struct device *dev, +struct gpio_desc *__must_check devm_gpiod_get_index(struct device *dev, const char *con_id, unsigned int idx, enum gpiod_flags flags) @@ -120,7 +120,7 @@ struct gpio_desc *__must_check __devm_gpiod_get_index(struct device *dev, return desc; } -EXPORT_SYMBOL(__devm_gpiod_get_index); +EXPORT_SYMBOL(devm_gpiod_get_index); /** * devm_get_gpiod_from_child - get a GPIO descriptor from a device's child node @@ -182,10 +182,10 @@ EXPORT_SYMBOL(devm_get_gpiod_from_child); * gpiod_get_index_optional() for detailed information about behavior and * return values. */ -struct gpio_desc *__must_check __devm_gpiod_get_index_optional(struct device *dev, +struct gpio_desc *__must_check devm_gpiod_get_index_optional(struct device *dev, const char *con_id, unsigned int index, - enum gpiod_flags flags) + enum gpiod_flags flags) { struct gpio_desc *desc; @@ -197,7 +197,7 @@ struct gpio_desc *__must_check __devm_gpiod_get_index_optional(struct device *de return desc; } -EXPORT_SYMBOL(__devm_gpiod_get_index_optional); +EXPORT_SYMBOL(devm_gpiod_get_index_optional); /** * devm_gpiod_get_array - Resource-managed gpiod_get_array() diff --git a/drivers/gpio/gpio-74xx-mmio.c b/drivers/gpio/gpio-74xx-mmio.c index 0763655cca6ccc..6ed7c0fb3378ea 100644 --- a/drivers/gpio/gpio-74xx-mmio.c +++ b/drivers/gpio/gpio-74xx-mmio.c @@ -129,7 +129,7 @@ static int mmio_74xx_gpio_probe(struct platform_device *pdev) if (IS_ERR(dat)) return PTR_ERR(dat); - priv->flags = (unsigned)of_id->data; + priv->flags = (uintptr_t) of_id->data; err = bgpio_init(&priv->bgc, &pdev->dev, DIV_ROUND_UP(MMIO_74XX_BIT_CNT(priv->flags), 8), diff --git a/drivers/gpio/gpio-adp5588.c b/drivers/gpio/gpio-adp5588.c index d3fe6a6776daaa..984186ee58a099 100644 --- a/drivers/gpio/gpio-adp5588.c +++ b/drivers/gpio/gpio-adp5588.c @@ -305,15 +305,7 @@ static int adp5588_irq_setup(struct adp5588_gpio *dev) irq_set_chip_and_handler(irq, &adp5588_irq_chip, handle_level_irq); irq_set_nested_thread(irq, 1); -#ifdef CONFIG_ARM - /* - * ARM needs us to explicitly flag the IRQ as VALID, - * once we do so, it will also set the noprobe. - */ - set_irq_flags(irq, IRQF_VALID); -#else - irq_set_noprobe(irq); -#endif + irq_modify_status(irq, IRQ_NOREQUEST, IRQ_NOPROBE); } ret = request_threaded_irq(client->irq, diff --git a/drivers/gpio/gpio-altera.c b/drivers/gpio/gpio-altera.c index 0f3d336d6303e9..9b7e0b3db38721 100644 --- a/drivers/gpio/gpio-altera.c +++ b/drivers/gpio/gpio-altera.c @@ -338,9 +338,9 @@ static int altera_gpio_remove(struct platform_device *pdev) { struct altera_gpio_chip *altera_gc = platform_get_drvdata(pdev); - gpiochip_remove(&altera_gc->mmchip.gc); + of_mm_gpiochip_remove(&altera_gc->mmchip); - return -EIO; + return 0; } static const struct of_device_id altera_gpio_of_match[] = { diff --git a/arch/mips/ath79/gpio.c b/drivers/gpio/gpio-ath79.c similarity index 78% rename from arch/mips/ath79/gpio.c rename to drivers/gpio/gpio-ath79.c index f59ccb26520a94..03b995304ad682 100644 --- a/arch/mips/ath79/gpio.c +++ b/drivers/gpio/gpio-ath79.c @@ -24,8 +24,6 @@ #include #include -#include -#include "common.h" static void __iomem *ath79_gpio_base; static u32 ath79_gpio_count; @@ -139,47 +137,6 @@ static struct gpio_chip ath79_gpio_chip = { .base = 0, }; -static void __iomem *ath79_gpio_get_function_reg(void) -{ - u32 reg = 0; - - if (soc_is_ar71xx() || - soc_is_ar724x() || - soc_is_ar913x() || - soc_is_ar933x()) - reg = AR71XX_GPIO_REG_FUNC; - else if (soc_is_ar934x()) - reg = AR934X_GPIO_REG_FUNC; - else - BUG(); - - return ath79_gpio_base + reg; -} - -void ath79_gpio_function_setup(u32 set, u32 clear) -{ - void __iomem *reg = ath79_gpio_get_function_reg(); - unsigned long flags; - - spin_lock_irqsave(&ath79_gpio_lock, flags); - - __raw_writel((__raw_readl(reg) & ~clear) | set, reg); - /* flush write */ - __raw_readl(reg); - - spin_unlock_irqrestore(&ath79_gpio_lock, flags); -} - -void ath79_gpio_function_enable(u32 mask) -{ - ath79_gpio_function_setup(mask, 0); -} - -void ath79_gpio_function_disable(u32 mask) -{ - ath79_gpio_function_setup(0, mask); -} - static const struct of_device_id ath79_gpio_of_match[] = { { .compatible = "qca,ar7100-gpio" }, { .compatible = "qca,ar9340-gpio" }, @@ -245,35 +202,3 @@ static struct platform_driver ath79_gpio_driver = { }; module_platform_driver(ath79_gpio_driver); - -int gpio_get_value(unsigned gpio) -{ - if (gpio < ath79_gpio_count) - return __ath79_gpio_get_value(gpio); - - return __gpio_get_value(gpio); -} -EXPORT_SYMBOL(gpio_get_value); - -void gpio_set_value(unsigned gpio, int value) -{ - if (gpio < ath79_gpio_count) - __ath79_gpio_set_value(gpio, value); - else - __gpio_set_value(gpio, value); -} -EXPORT_SYMBOL(gpio_set_value); - -int gpio_to_irq(unsigned gpio) -{ - /* FIXME */ - return -EINVAL; -} -EXPORT_SYMBOL(gpio_to_irq); - -int irq_to_gpio(unsigned irq) -{ - /* FIXME */ - return -EINVAL; -} -EXPORT_SYMBOL(irq_to_gpio); diff --git a/drivers/gpio/gpio-bcm-kona.c b/drivers/gpio/gpio-bcm-kona.c index 40343fa92c7b9c..31b90ac15204b9 100644 --- a/drivers/gpio/gpio-bcm-kona.c +++ b/drivers/gpio/gpio-bcm-kona.c @@ -438,7 +438,7 @@ static void bcm_kona_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) void __iomem *reg_base; int bit, bank_id; unsigned long sta; - struct bcm_kona_gpio_bank *bank = irq_get_handler_data(irq); + struct bcm_kona_gpio_bank *bank = irq_desc_get_handler_data(desc); struct irq_chip *chip = irq_desc_get_chip(desc); chained_irq_enter(chip, desc); @@ -525,11 +525,7 @@ static int bcm_kona_gpio_irq_map(struct irq_domain *d, unsigned int irq, return ret; irq_set_lockdep_class(irq, &gpio_lock_class); irq_set_chip_and_handler(irq, &bcm_gpio_irq_chip, handle_simple_irq); -#ifdef CONFIG_ARM - set_irq_flags(irq, IRQF_VALID); -#else irq_set_noprobe(irq); -#endif return 0; } @@ -644,17 +640,6 @@ static int bcm_kona_gpio_probe(struct platform_device *pdev) dev_err(dev, "Couldn't add GPIO chip -- %d\n", ret); goto err_irq_domain; } - for (i = 0; i < chip->ngpio; i++) { - int irq = bcm_kona_gpio_to_irq(chip, i); - irq_set_lockdep_class(irq, &gpio_lock_class); - irq_set_chip_and_handler(irq, &bcm_gpio_irq_chip, - handle_simple_irq); -#ifdef CONFIG_ARM - set_irq_flags(irq, IRQF_VALID); -#else - irq_set_noprobe(irq); -#endif - } for (i = 0; i < kona_gpio->num_bank; i++) { bank = &kona_gpio->banks[i]; irq_set_chained_handler_and_data(bank->irq, diff --git a/drivers/gpio/gpio-brcmstb.c b/drivers/gpio/gpio-brcmstb.c index 4630a8133ea6b9..9ea86d2ac05462 100644 --- a/drivers/gpio/gpio-brcmstb.c +++ b/drivers/gpio/gpio-brcmstb.c @@ -17,6 +17,10 @@ #include #include #include +#include +#include +#include +#include #define GIO_BANK_SIZE 0x20 #define GIO_ODEN(bank) (((bank) * GIO_BANK_SIZE) + 0x00) @@ -34,14 +38,18 @@ struct brcmstb_gpio_bank { struct bgpio_chip bgc; struct brcmstb_gpio_priv *parent_priv; u32 width; + struct irq_chip irq_chip; }; struct brcmstb_gpio_priv { struct list_head bank_list; void __iomem *reg_base; - int num_banks; struct platform_device *pdev; + int parent_irq; int gpio_base; + bool can_wake; + int parent_wake_irq; + struct notifier_block reboot_notifier; }; #define MAX_GPIO_PER_BANK 32 @@ -63,6 +71,203 @@ brcmstb_gpio_gc_to_priv(struct gpio_chip *gc) return bank->parent_priv; } +static void brcmstb_gpio_set_imask(struct brcmstb_gpio_bank *bank, + unsigned int offset, bool enable) +{ + struct bgpio_chip *bgc = &bank->bgc; + struct brcmstb_gpio_priv *priv = bank->parent_priv; + u32 mask = bgc->pin2mask(bgc, offset); + u32 imask; + unsigned long flags; + + spin_lock_irqsave(&bgc->lock, flags); + imask = bgc->read_reg(priv->reg_base + GIO_MASK(bank->id)); + if (enable) + imask |= mask; + else + imask &= ~mask; + bgc->write_reg(priv->reg_base + GIO_MASK(bank->id), imask); + spin_unlock_irqrestore(&bgc->lock, flags); +} + +/* -------------------- IRQ chip functions -------------------- */ + +static void brcmstb_gpio_irq_mask(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct brcmstb_gpio_bank *bank = brcmstb_gpio_gc_to_bank(gc); + + brcmstb_gpio_set_imask(bank, d->hwirq, false); +} + +static void brcmstb_gpio_irq_unmask(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct brcmstb_gpio_bank *bank = brcmstb_gpio_gc_to_bank(gc); + + brcmstb_gpio_set_imask(bank, d->hwirq, true); +} + +static int brcmstb_gpio_irq_set_type(struct irq_data *d, unsigned int type) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct brcmstb_gpio_bank *bank = brcmstb_gpio_gc_to_bank(gc); + struct brcmstb_gpio_priv *priv = bank->parent_priv; + u32 mask = BIT(d->hwirq); + u32 edge_insensitive, iedge_insensitive; + u32 edge_config, iedge_config; + u32 level, ilevel; + unsigned long flags; + + switch (type) { + case IRQ_TYPE_LEVEL_LOW: + level = 0; + edge_config = 0; + edge_insensitive = 0; + break; + case IRQ_TYPE_LEVEL_HIGH: + level = mask; + edge_config = 0; + edge_insensitive = 0; + break; + case IRQ_TYPE_EDGE_FALLING: + level = 0; + edge_config = 0; + edge_insensitive = 0; + break; + case IRQ_TYPE_EDGE_RISING: + level = 0; + edge_config = mask; + edge_insensitive = 0; + break; + case IRQ_TYPE_EDGE_BOTH: + level = 0; + edge_config = 0; /* don't care, but want known value */ + edge_insensitive = mask; + break; + default: + return -EINVAL; + } + + spin_lock_irqsave(&bank->bgc.lock, flags); + + iedge_config = bank->bgc.read_reg(priv->reg_base + + GIO_EC(bank->id)) & ~mask; + iedge_insensitive = bank->bgc.read_reg(priv->reg_base + + GIO_EI(bank->id)) & ~mask; + ilevel = bank->bgc.read_reg(priv->reg_base + + GIO_LEVEL(bank->id)) & ~mask; + + bank->bgc.write_reg(priv->reg_base + GIO_EC(bank->id), + iedge_config | edge_config); + bank->bgc.write_reg(priv->reg_base + GIO_EI(bank->id), + iedge_insensitive | edge_insensitive); + bank->bgc.write_reg(priv->reg_base + GIO_LEVEL(bank->id), + ilevel | level); + + spin_unlock_irqrestore(&bank->bgc.lock, flags); + return 0; +} + +static int brcmstb_gpio_priv_set_wake(struct brcmstb_gpio_priv *priv, + unsigned int enable) +{ + int ret = 0; + + /* + * Only enable wake IRQ once for however many hwirqs can wake + * since they all use the same wake IRQ. Mask will be set + * up appropriately thanks to IRQCHIP_MASK_ON_SUSPEND flag. + */ + if (enable) + ret = enable_irq_wake(priv->parent_wake_irq); + else + ret = disable_irq_wake(priv->parent_wake_irq); + if (ret) + dev_err(&priv->pdev->dev, "failed to %s wake-up interrupt\n", + enable ? "enable" : "disable"); + return ret; +} + +static int brcmstb_gpio_irq_set_wake(struct irq_data *d, unsigned int enable) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct brcmstb_gpio_priv *priv = brcmstb_gpio_gc_to_priv(gc); + + return brcmstb_gpio_priv_set_wake(priv, enable); +} + +static irqreturn_t brcmstb_gpio_wake_irq_handler(int irq, void *data) +{ + struct brcmstb_gpio_priv *priv = data; + + if (!priv || irq != priv->parent_wake_irq) + return IRQ_NONE; + pm_wakeup_event(&priv->pdev->dev, 0); + return IRQ_HANDLED; +} + +static void brcmstb_gpio_irq_bank_handler(struct brcmstb_gpio_bank *bank) +{ + struct brcmstb_gpio_priv *priv = bank->parent_priv; + struct irq_domain *irq_domain = bank->bgc.gc.irqdomain; + void __iomem *reg_base = priv->reg_base; + unsigned long status; + unsigned long flags; + + spin_lock_irqsave(&bank->bgc.lock, flags); + while ((status = bank->bgc.read_reg(reg_base + GIO_STAT(bank->id)) & + bank->bgc.read_reg(reg_base + GIO_MASK(bank->id)))) { + int bit; + + for_each_set_bit(bit, &status, 32) { + u32 stat = bank->bgc.read_reg(reg_base + + GIO_STAT(bank->id)); + if (bit >= bank->width) + dev_warn(&priv->pdev->dev, + "IRQ for invalid GPIO (bank=%d, offset=%d)\n", + bank->id, bit); + bank->bgc.write_reg(reg_base + GIO_STAT(bank->id), + stat | BIT(bit)); + generic_handle_irq(irq_find_mapping(irq_domain, bit)); + } + } + spin_unlock_irqrestore(&bank->bgc.lock, flags); +} + +/* Each UPG GIO block has one IRQ for all banks */ +static void brcmstb_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) +{ + struct gpio_chip *gc = irq_desc_get_handler_data(desc); + struct brcmstb_gpio_priv *priv = brcmstb_gpio_gc_to_priv(gc); + struct irq_chip *chip = irq_desc_get_chip(desc); + struct list_head *pos; + + /* Interrupts weren't properly cleared during probe */ + BUG_ON(!priv || !chip); + + chained_irq_enter(chip, desc); + list_for_each(pos, &priv->bank_list) { + struct brcmstb_gpio_bank *bank = + list_entry(pos, struct brcmstb_gpio_bank, node); + brcmstb_gpio_irq_bank_handler(bank); + } + chained_irq_exit(chip, desc); +} + +static int brcmstb_gpio_reboot(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct brcmstb_gpio_priv *priv = + container_of(nb, struct brcmstb_gpio_priv, reboot_notifier); + + /* Enable GPIO for S5 cold boot */ + if (action == SYS_POWER_OFF) + brcmstb_gpio_priv_set_wake(priv, 1); + + return NOTIFY_DONE; +} + /* Make sure that the number of banks matches up between properties */ static int brcmstb_gpio_sanity_check_banks(struct device *dev, struct device_node *np, struct resource *res) @@ -100,7 +305,13 @@ static int brcmstb_gpio_remove(struct platform_device *pdev) bank = list_entry(pos, struct brcmstb_gpio_bank, node); ret = bgpio_remove(&bank->bgc); if (ret) - dev_err(&pdev->dev, "gpiochip_remove fail in cleanup"); + dev_err(&pdev->dev, "gpiochip_remove fail in cleanup\n"); + } + if (priv->reboot_notifier.notifier_call) { + ret = unregister_reboot_notifier(&priv->reboot_notifier); + if (ret) + dev_err(&pdev->dev, + "failed to unregister reboot notifier\n"); } return ret; } @@ -121,7 +332,7 @@ static int brcmstb_gpio_of_xlate(struct gpio_chip *gc, return -EINVAL; offset = gpiospec->args[0] - (gc->base - priv->gpio_base); - if (offset >= gc->ngpio) + if (offset >= gc->ngpio || offset < 0) return -EINVAL; if (unlikely(offset >= bank->width)) { @@ -136,6 +347,65 @@ static int brcmstb_gpio_of_xlate(struct gpio_chip *gc, return offset; } +/* Before calling, must have bank->parent_irq set and gpiochip registered */ +static int brcmstb_gpio_irq_setup(struct platform_device *pdev, + struct brcmstb_gpio_bank *bank) +{ + struct brcmstb_gpio_priv *priv = bank->parent_priv; + struct device *dev = &pdev->dev; + struct device_node *np = dev->of_node; + + bank->irq_chip.name = dev_name(dev); + bank->irq_chip.irq_mask = brcmstb_gpio_irq_mask; + bank->irq_chip.irq_unmask = brcmstb_gpio_irq_unmask; + bank->irq_chip.irq_set_type = brcmstb_gpio_irq_set_type; + + /* Ensures that all non-wakeup IRQs are disabled at suspend */ + bank->irq_chip.flags = IRQCHIP_MASK_ON_SUSPEND; + + if (IS_ENABLED(CONFIG_PM_SLEEP) && !priv->can_wake && + of_property_read_bool(np, "wakeup-source")) { + priv->parent_wake_irq = platform_get_irq(pdev, 1); + if (priv->parent_wake_irq < 0) { + dev_warn(dev, + "Couldn't get wake IRQ - GPIOs will not be able to wake from sleep"); + } else { + int err; + + /* + * Set wakeup capability before requesting wakeup + * interrupt, so we can process boot-time "wakeups" + * (e.g., from S5 cold boot) + */ + device_set_wakeup_capable(dev, true); + device_wakeup_enable(dev); + err = devm_request_irq(dev, priv->parent_wake_irq, + brcmstb_gpio_wake_irq_handler, 0, + "brcmstb-gpio-wake", priv); + + if (err < 0) { + dev_err(dev, "Couldn't request wake IRQ"); + return err; + } + + priv->reboot_notifier.notifier_call = + brcmstb_gpio_reboot; + register_reboot_notifier(&priv->reboot_notifier); + priv->can_wake = true; + } + } + + if (priv->can_wake) + bank->irq_chip.irq_set_wake = brcmstb_gpio_irq_set_wake; + + gpiochip_irqchip_add(&bank->bgc.gc, &bank->irq_chip, 0, + handle_simple_irq, IRQ_TYPE_NONE); + gpiochip_set_chained_irqchip(&bank->bgc.gc, &bank->irq_chip, + priv->parent_irq, brcmstb_gpio_irq_handler); + + return 0; +} + static int brcmstb_gpio_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -146,6 +416,7 @@ static int brcmstb_gpio_probe(struct platform_device *pdev) struct property *prop; const __be32 *p; u32 bank_width; + int num_banks = 0; int err; static int gpio_base; @@ -164,6 +435,16 @@ static int brcmstb_gpio_probe(struct platform_device *pdev) priv->reg_base = reg_base; priv->pdev = pdev; + if (of_property_read_bool(np, "interrupt-controller")) { + priv->parent_irq = platform_get_irq(pdev, 0); + if (priv->parent_irq <= 0) { + dev_err(dev, "Couldn't get IRQ"); + return -ENOENT; + } + } else { + priv->parent_irq = -ENOENT; + } + if (brcmstb_gpio_sanity_check_banks(dev, np, res)) return -EINVAL; @@ -180,7 +461,7 @@ static int brcmstb_gpio_probe(struct platform_device *pdev) } bank->parent_priv = priv; - bank->id = priv->num_banks; + bank->id = num_banks; if (bank_width <= 0 || bank_width > MAX_GPIO_PER_BANK) { dev_err(dev, "Invalid bank width %d\n", bank_width); goto fail; @@ -212,6 +493,12 @@ static int brcmstb_gpio_probe(struct platform_device *pdev) /* not all ngpio lines are valid, will use bank width later */ gc->ngpio = MAX_GPIO_PER_BANK; + /* + * Mask all interrupts by default, since wakeup interrupts may + * be retained from S5 cold boot + */ + bank->bgc.write_reg(reg_base + GIO_MASK(bank->id), 0); + err = gpiochip_add(gc); if (err) { dev_err(dev, "Could not add gpiochip for bank %d\n", @@ -219,17 +506,24 @@ static int brcmstb_gpio_probe(struct platform_device *pdev) goto fail; } gpio_base += gc->ngpio; + + if (priv->parent_irq > 0) { + err = brcmstb_gpio_irq_setup(pdev, bank); + if (err) + goto fail; + } + dev_dbg(dev, "bank=%d, base=%d, ngpio=%d, width=%d\n", bank->id, gc->base, gc->ngpio, bank->width); /* Everything looks good, so add bank to list */ list_add(&bank->node, &priv->bank_list); - priv->num_banks++; + num_banks++; } dev_info(dev, "Registered %d banks (GPIO(s): %d-%d)\n", - priv->num_banks, priv->gpio_base, gpio_base - 1); + num_banks, priv->gpio_base, gpio_base - 1); return 0; diff --git a/drivers/gpio/gpio-davinci.c b/drivers/gpio/gpio-davinci.c index c246ac3dda7ca2..94b0ab70972165 100644 --- a/drivers/gpio/gpio-davinci.c +++ b/drivers/gpio/gpio-davinci.c @@ -65,11 +65,11 @@ static struct davinci_gpio_regs __iomem *gpio2regs(unsigned gpio) return ptr; } -static inline struct davinci_gpio_regs __iomem *irq2regs(int irq) +static inline struct davinci_gpio_regs __iomem *irq2regs(struct irq_data *d) { struct davinci_gpio_regs __iomem *g; - g = (__force struct davinci_gpio_regs __iomem *)irq_get_chip_data(irq); + g = (__force struct davinci_gpio_regs __iomem *)irq_data_get_irq_chip_data(d); return g; } @@ -287,7 +287,7 @@ static int davinci_gpio_probe(struct platform_device *pdev) static void gpio_irq_disable(struct irq_data *d) { - struct davinci_gpio_regs __iomem *g = irq2regs(d->irq); + struct davinci_gpio_regs __iomem *g = irq2regs(d); u32 mask = (u32) irq_data_get_irq_handler_data(d); writel_relaxed(mask, &g->clr_falling); @@ -296,7 +296,7 @@ static void gpio_irq_disable(struct irq_data *d) static void gpio_irq_enable(struct irq_data *d) { - struct davinci_gpio_regs __iomem *g = irq2regs(d->irq); + struct davinci_gpio_regs __iomem *g = irq2regs(d); u32 mask = (u32) irq_data_get_irq_handler_data(d); unsigned status = irqd_get_trigger_type(d); @@ -327,8 +327,9 @@ static struct irq_chip gpio_irqchip = { }; static void -gpio_irq_handler(unsigned irq, struct irq_desc *desc) +gpio_irq_handler(unsigned __irq, struct irq_desc *desc) { + unsigned int irq = irq_desc_get_irq(desc); struct davinci_gpio_regs __iomem *g; u32 mask = 0xffff; struct davinci_gpio_controller *d; @@ -396,7 +397,7 @@ static int gpio_irq_type_unbanked(struct irq_data *data, unsigned trigger) struct davinci_gpio_regs __iomem *g; u32 mask; - d = (struct davinci_gpio_controller *)data->handler_data; + d = (struct davinci_gpio_controller *)irq_data_get_irq_handler_data(data); g = (struct davinci_gpio_regs __iomem *)d->regs; mask = __gpio_mask(data->irq - d->gpio_irq); @@ -422,7 +423,6 @@ davinci_gpio_irq_map(struct irq_domain *d, unsigned int irq, irq_set_irq_type(irq, IRQ_TYPE_NONE); irq_set_chip_data(irq, (__force void *)g); irq_set_handler_data(irq, (void *)__gpio_mask(hw)); - set_irq_flags(irq, IRQF_VALID); return 0; } @@ -545,7 +545,7 @@ static int davinci_gpio_irq_setup(struct platform_device *pdev) chips[0].chip.to_irq = gpio_to_irq_unbanked; chips[0].gpio_irq = bank_irq; chips[0].gpio_unbanked = pdata->gpio_unbanked; - binten = BIT(0); + binten = GENMASK(pdata->gpio_unbanked / 16, 0); /* AINTC handles mask/unmask; GPIO handles triggering */ irq = bank_irq; diff --git a/drivers/gpio/gpio-dwapb.c b/drivers/gpio/gpio-dwapb.c index 55fa9853a7f220..c5be4b9b8baf73 100644 --- a/drivers/gpio/gpio-dwapb.c +++ b/drivers/gpio/gpio-dwapb.c @@ -149,7 +149,7 @@ static u32 dwapb_do_irq(struct dwapb_gpio *gpio) static void dwapb_irq_handler(u32 irq, struct irq_desc *desc) { - struct dwapb_gpio *gpio = irq_get_handler_data(irq); + struct dwapb_gpio *gpio = irq_desc_get_handler_data(desc); struct irq_chip *chip = irq_desc_get_chip(desc); dwapb_do_irq(gpio); diff --git a/drivers/gpio/gpio-em.c b/drivers/gpio/gpio-em.c index fbf287307c4cc7..6bca1e125e1234 100644 --- a/drivers/gpio/gpio-em.c +++ b/drivers/gpio/gpio-em.c @@ -31,7 +31,6 @@ #include #include #include -#include struct em_gio_priv { void __iomem *base0; @@ -262,7 +261,6 @@ static int em_gio_irq_domain_map(struct irq_domain *h, unsigned int irq, irq_set_chip_data(irq, h->host_data); irq_set_chip_and_handler(irq, &p->irq_chip, handle_level_irq); - set_irq_flags(irq, IRQF_VALID); /* kill me now */ return 0; } @@ -273,13 +271,12 @@ static const struct irq_domain_ops em_gio_irq_domain_ops = { static int em_gio_probe(struct platform_device *pdev) { - struct gpio_em_config pdata_dt; - struct gpio_em_config *pdata = dev_get_platdata(&pdev->dev); struct em_gio_priv *p; struct resource *io[2], *irq[2]; struct gpio_chip *gpio_chip; struct irq_chip *irq_chip; const char *name = dev_name(&pdev->dev); + unsigned int ngpios; int ret; p = devm_kzalloc(&pdev->dev, sizeof(*p), GFP_KERNEL); @@ -319,18 +316,10 @@ static int em_gio_probe(struct platform_device *pdev) goto err0; } - if (!pdata) { - memset(&pdata_dt, 0, sizeof(pdata_dt)); - pdata = &pdata_dt; - - if (of_property_read_u32(pdev->dev.of_node, "ngpios", - &pdata->number_of_pins)) { - dev_err(&pdev->dev, "Missing ngpios OF property\n"); - ret = -EINVAL; - goto err0; - } - - pdata->gpio_base = -1; + if (of_property_read_u32(pdev->dev.of_node, "ngpios", &ngpios)) { + dev_err(&pdev->dev, "Missing ngpios OF property\n"); + ret = -EINVAL; + goto err0; } gpio_chip = &p->gpio_chip; @@ -345,8 +334,8 @@ static int em_gio_probe(struct platform_device *pdev) gpio_chip->label = name; gpio_chip->dev = &pdev->dev; gpio_chip->owner = THIS_MODULE; - gpio_chip->base = pdata->gpio_base; - gpio_chip->ngpio = pdata->number_of_pins; + gpio_chip->base = -1; + gpio_chip->ngpio = ngpios; irq_chip = &p->irq_chip; irq_chip->name = name; @@ -357,9 +346,7 @@ static int em_gio_probe(struct platform_device *pdev) irq_chip->irq_release_resources = em_gio_irq_relres; irq_chip->flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND; - p->irq_domain = irq_domain_add_simple(pdev->dev.of_node, - pdata->number_of_pins, - pdata->irq_base, + p->irq_domain = irq_domain_add_simple(pdev->dev.of_node, ngpios, 0, &em_gio_irq_domain_ops, p); if (!p->irq_domain) { ret = -ENXIO; @@ -387,12 +374,6 @@ static int em_gio_probe(struct platform_device *pdev) goto err1; } - if (pdata->pctl_name) { - ret = gpiochip_add_pin_range(gpio_chip, pdata->pctl_name, 0, - gpio_chip->base, gpio_chip->ngpio); - if (ret < 0) - dev_warn(&pdev->dev, "failed to add pin range\n"); - } return 0; err1: diff --git a/drivers/gpio/gpio-ep93xx.c b/drivers/gpio/gpio-ep93xx.c index 45684f36ddb15d..9d90366ea25999 100644 --- a/drivers/gpio/gpio-ep93xx.c +++ b/drivers/gpio/gpio-ep93xx.c @@ -100,13 +100,15 @@ static void ep93xx_gpio_ab_irq_handler(unsigned int irq, struct irq_desc *desc) } } -static void ep93xx_gpio_f_irq_handler(unsigned int irq, struct irq_desc *desc) +static void ep93xx_gpio_f_irq_handler(unsigned int __irq, + struct irq_desc *desc) { /* * map discontiguous hw irq range to continuous sw irq range: * * IRQ_EP93XX_GPIO{0..7}MUX -> gpio_to_irq(EP93XX_GPIO_LINE_F({0..7}) */ + unsigned int irq = irq_desc_get_irq(desc); int port_f_idx = ((irq + 1) & 7) ^ 4; /* {19..22,47..50} -> {0..7} */ int gpio_irq = gpio_to_irq(EP93XX_GPIO_LINE_F(0)) + port_f_idx; @@ -208,7 +210,7 @@ static int ep93xx_gpio_irq_type(struct irq_data *d, unsigned int type) return -EINVAL; } - __irq_set_handler_locked(d->irq, handler); + irq_set_handler_locked(d, handler); gpio_int_enabled[port] |= port_mask; @@ -234,7 +236,7 @@ static void ep93xx_gpio_init_irq(void) gpio_irq <= gpio_to_irq(EP93XX_GPIO_LINE_MAX_IRQ); ++gpio_irq) { irq_set_chip_and_handler(gpio_irq, &ep93xx_gpio_irq_chip, handle_level_irq); - set_irq_flags(gpio_irq, IRQF_VALID); + irq_clear_status_flags(gpio_irq, IRQ_NOREQUEST); } irq_set_chained_handler(IRQ_EP93XX_GPIO_AB, diff --git a/drivers/gpio/gpio-etraxfs.c b/drivers/gpio/gpio-etraxfs.c index 28071f4a56724b..2ffcd9fdd1f2f7 100644 --- a/drivers/gpio/gpio-etraxfs.c +++ b/drivers/gpio/gpio-etraxfs.c @@ -1,8 +1,10 @@ #include #include #include +#include #include #include +#include #include #include @@ -13,6 +15,7 @@ #define ETRAX_FS_rw_intr_mask 16 #define ETRAX_FS_rw_ack_intr 20 #define ETRAX_FS_r_intr 24 +#define ETRAX_FS_r_masked_intr 28 #define ETRAX_FS_rw_pb_dout 32 #define ETRAX_FS_r_pb_din 36 #define ETRAX_FS_rw_pb_oe 40 @@ -26,6 +29,48 @@ #define ETRAX_FS_r_pe_din 84 #define ETRAX_FS_rw_pe_oe 88 +#define ARTPEC3_r_pa_din 0 +#define ARTPEC3_rw_pa_dout 4 +#define ARTPEC3_rw_pa_oe 8 +#define ARTPEC3_r_pb_din 44 +#define ARTPEC3_rw_pb_dout 48 +#define ARTPEC3_rw_pb_oe 52 +#define ARTPEC3_r_pc_din 88 +#define ARTPEC3_rw_pc_dout 92 +#define ARTPEC3_rw_pc_oe 96 +#define ARTPEC3_r_pd_din 116 +#define ARTPEC3_rw_intr_cfg 120 +#define ARTPEC3_rw_intr_pins 124 +#define ARTPEC3_rw_intr_mask 128 +#define ARTPEC3_rw_ack_intr 132 +#define ARTPEC3_r_masked_intr 140 + +#define GIO_CFG_OFF 0 +#define GIO_CFG_HI 1 +#define GIO_CFG_LO 2 +#define GIO_CFG_SET 3 +#define GIO_CFG_POSEDGE 5 +#define GIO_CFG_NEGEDGE 6 +#define GIO_CFG_ANYEDGE 7 + +struct etraxfs_gpio_info; + +struct etraxfs_gpio_block { + spinlock_t lock; + u32 mask; + u32 cfg; + u32 pins; + unsigned int group[8]; + + void __iomem *regs; + const struct etraxfs_gpio_info *info; +}; + +struct etraxfs_gpio_chip { + struct bgpio_chip bgc; + struct etraxfs_gpio_block *block; +}; + struct etraxfs_gpio_port { const char *label; unsigned int oe; @@ -37,6 +82,12 @@ struct etraxfs_gpio_port { struct etraxfs_gpio_info { unsigned int num_ports; const struct etraxfs_gpio_port *ports; + + unsigned int rw_ack_intr; + unsigned int rw_intr_mask; + unsigned int rw_intr_cfg; + unsigned int rw_intr_pins; + unsigned int r_masked_intr; }; static const struct etraxfs_gpio_port etraxfs_gpio_etraxfs_ports[] = { @@ -80,8 +131,56 @@ static const struct etraxfs_gpio_port etraxfs_gpio_etraxfs_ports[] = { static const struct etraxfs_gpio_info etraxfs_gpio_etraxfs = { .num_ports = ARRAY_SIZE(etraxfs_gpio_etraxfs_ports), .ports = etraxfs_gpio_etraxfs_ports, + .rw_ack_intr = ETRAX_FS_rw_ack_intr, + .rw_intr_mask = ETRAX_FS_rw_intr_mask, + .rw_intr_cfg = ETRAX_FS_rw_intr_cfg, + .r_masked_intr = ETRAX_FS_r_masked_intr, +}; + +static const struct etraxfs_gpio_port etraxfs_gpio_artpec3_ports[] = { + { + .label = "A", + .ngpio = 32, + .oe = ARTPEC3_rw_pa_oe, + .dout = ARTPEC3_rw_pa_dout, + .din = ARTPEC3_r_pa_din, + }, + { + .label = "B", + .ngpio = 32, + .oe = ARTPEC3_rw_pb_oe, + .dout = ARTPEC3_rw_pb_dout, + .din = ARTPEC3_r_pb_din, + }, + { + .label = "C", + .ngpio = 16, + .oe = ARTPEC3_rw_pc_oe, + .dout = ARTPEC3_rw_pc_dout, + .din = ARTPEC3_r_pc_din, + }, + { + .label = "D", + .ngpio = 32, + .din = ARTPEC3_r_pd_din, + }, +}; + +static const struct etraxfs_gpio_info etraxfs_gpio_artpec3 = { + .num_ports = ARRAY_SIZE(etraxfs_gpio_artpec3_ports), + .ports = etraxfs_gpio_artpec3_ports, + .rw_ack_intr = ARTPEC3_rw_ack_intr, + .rw_intr_mask = ARTPEC3_rw_intr_mask, + .rw_intr_cfg = ARTPEC3_rw_intr_cfg, + .r_masked_intr = ARTPEC3_r_masked_intr, + .rw_intr_pins = ARTPEC3_rw_intr_pins, }; +static unsigned int etraxfs_gpio_chip_to_port(struct gpio_chip *gc) +{ + return gc->label[0] - 'A'; +} + static int etraxfs_gpio_of_xlate(struct gpio_chip *gc, const struct of_phandle_args *gpiospec, u32 *flags) @@ -90,7 +189,7 @@ static int etraxfs_gpio_of_xlate(struct gpio_chip *gc, * Port numbers are A to E, and the properties are integers, so we * specify them as 0xA - 0xE. */ - if (gc->label[0] - 'A' + 0xA != gpiospec->args[2]) + if (etraxfs_gpio_chip_to_port(gc) + 0xA != gpiospec->args[2]) return -EINVAL; return of_gpio_simple_xlate(gc, gpiospec, flags); @@ -101,24 +200,174 @@ static const struct of_device_id etraxfs_gpio_of_table[] = { .compatible = "axis,etraxfs-gio", .data = &etraxfs_gpio_etraxfs, }, + { + .compatible = "axis,artpec3-gio", + .data = &etraxfs_gpio_artpec3, + }, {}, }; +static unsigned int etraxfs_gpio_to_group_irq(unsigned int gpio) +{ + return gpio % 8; +} + +static unsigned int etraxfs_gpio_to_group_pin(struct etraxfs_gpio_chip *chip, + unsigned int gpio) +{ + return 4 * etraxfs_gpio_chip_to_port(&chip->bgc.gc) + gpio / 8; +} + +static void etraxfs_gpio_irq_ack(struct irq_data *d) +{ + struct etraxfs_gpio_chip *chip = irq_data_get_irq_chip_data(d); + struct etraxfs_gpio_block *block = chip->block; + unsigned int grpirq = etraxfs_gpio_to_group_irq(d->hwirq); + + writel(BIT(grpirq), block->regs + block->info->rw_ack_intr); +} + +static void etraxfs_gpio_irq_mask(struct irq_data *d) +{ + struct etraxfs_gpio_chip *chip = irq_data_get_irq_chip_data(d); + struct etraxfs_gpio_block *block = chip->block; + unsigned int grpirq = etraxfs_gpio_to_group_irq(d->hwirq); + + spin_lock(&block->lock); + block->mask &= ~BIT(grpirq); + writel(block->mask, block->regs + block->info->rw_intr_mask); + spin_unlock(&block->lock); +} + +static void etraxfs_gpio_irq_unmask(struct irq_data *d) +{ + struct etraxfs_gpio_chip *chip = irq_data_get_irq_chip_data(d); + struct etraxfs_gpio_block *block = chip->block; + unsigned int grpirq = etraxfs_gpio_to_group_irq(d->hwirq); + + spin_lock(&block->lock); + block->mask |= BIT(grpirq); + writel(block->mask, block->regs + block->info->rw_intr_mask); + spin_unlock(&block->lock); +} + +static int etraxfs_gpio_irq_set_type(struct irq_data *d, u32 type) +{ + struct etraxfs_gpio_chip *chip = irq_data_get_irq_chip_data(d); + struct etraxfs_gpio_block *block = chip->block; + unsigned int grpirq = etraxfs_gpio_to_group_irq(d->hwirq); + u32 cfg; + + switch (type) { + case IRQ_TYPE_EDGE_RISING: + cfg = GIO_CFG_POSEDGE; + break; + case IRQ_TYPE_EDGE_FALLING: + cfg = GIO_CFG_NEGEDGE; + break; + case IRQ_TYPE_EDGE_BOTH: + cfg = GIO_CFG_ANYEDGE; + break; + case IRQ_TYPE_LEVEL_LOW: + cfg = GIO_CFG_LO; + break; + case IRQ_TYPE_LEVEL_HIGH: + cfg = GIO_CFG_HI; + break; + default: + return -EINVAL; + } + + spin_lock(&block->lock); + block->cfg &= ~(0x7 << (grpirq * 3)); + block->cfg |= (cfg << (grpirq * 3)); + writel(block->cfg, block->regs + block->info->rw_intr_cfg); + spin_unlock(&block->lock); + + return 0; +} + +static int etraxfs_gpio_irq_request_resources(struct irq_data *d) +{ + struct etraxfs_gpio_chip *chip = irq_data_get_irq_chip_data(d); + struct etraxfs_gpio_block *block = chip->block; + unsigned int grpirq = etraxfs_gpio_to_group_irq(d->hwirq); + int ret = -EBUSY; + + spin_lock(&block->lock); + if (block->group[grpirq]) + goto out; + + ret = gpiochip_lock_as_irq(&chip->bgc.gc, d->hwirq); + if (ret) + goto out; + + block->group[grpirq] = d->irq; + if (block->info->rw_intr_pins) { + unsigned int pin = etraxfs_gpio_to_group_pin(chip, d->hwirq); + + block->pins &= ~(0xf << (grpirq * 4)); + block->pins |= (pin << (grpirq * 4)); + + writel(block->pins, block->regs + block->info->rw_intr_pins); + } + +out: + spin_unlock(&block->lock); + return ret; +} + +static void etraxfs_gpio_irq_release_resources(struct irq_data *d) +{ + struct etraxfs_gpio_chip *chip = irq_data_get_irq_chip_data(d); + struct etraxfs_gpio_block *block = chip->block; + unsigned int grpirq = etraxfs_gpio_to_group_irq(d->hwirq); + + spin_lock(&block->lock); + block->group[grpirq] = 0; + gpiochip_unlock_as_irq(&chip->bgc.gc, d->hwirq); + spin_unlock(&block->lock); +} + +static struct irq_chip etraxfs_gpio_irq_chip = { + .name = "gpio-etraxfs", + .irq_ack = etraxfs_gpio_irq_ack, + .irq_mask = etraxfs_gpio_irq_mask, + .irq_unmask = etraxfs_gpio_irq_unmask, + .irq_set_type = etraxfs_gpio_irq_set_type, + .irq_request_resources = etraxfs_gpio_irq_request_resources, + .irq_release_resources = etraxfs_gpio_irq_release_resources, +}; + +static irqreturn_t etraxfs_gpio_interrupt(int irq, void *dev_id) +{ + struct etraxfs_gpio_block *block = dev_id; + unsigned long intr = readl(block->regs + block->info->r_masked_intr); + int bit; + + for_each_set_bit(bit, &intr, 8) + generic_handle_irq(block->group[bit]); + + return IRQ_RETVAL(intr & 0xff); +} + static int etraxfs_gpio_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; const struct etraxfs_gpio_info *info; const struct of_device_id *match; - struct bgpio_chip *chips; - struct resource *res; + struct etraxfs_gpio_block *block; + struct etraxfs_gpio_chip *chips; + struct resource *res, *irq; + bool allportsirq = false; void __iomem *regs; int ret; int i; res = platform_get_resource(pdev, IORESOURCE_MEM, 0); regs = devm_ioremap_resource(dev, res); - if (!regs) - return -ENOMEM; + if (IS_ERR(regs)) + return PTR_ERR(regs); match = of_match_node(etraxfs_gpio_of_table, dev->of_node); if (!match) @@ -130,19 +379,57 @@ static int etraxfs_gpio_probe(struct platform_device *pdev) if (!chips) return -ENOMEM; + irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0); + if (!irq) + return -EINVAL; + + block = devm_kzalloc(dev, sizeof(*block), GFP_KERNEL); + if (!block) + return -ENOMEM; + + spin_lock_init(&block->lock); + + block->regs = regs; + block->info = info; + + writel(0, block->regs + info->rw_intr_mask); + writel(0, block->regs + info->rw_intr_cfg); + if (info->rw_intr_pins) { + allportsirq = true; + writel(0, block->regs + info->rw_intr_pins); + } + + ret = devm_request_irq(dev, irq->start, etraxfs_gpio_interrupt, + IRQF_SHARED, dev_name(dev), block); + if (ret) { + dev_err(dev, "Unable to request irq %d\n", ret); + return ret; + } + for (i = 0; i < info->num_ports; i++) { - struct bgpio_chip *bgc = &chips[i]; + struct etraxfs_gpio_chip *chip = &chips[i]; + struct bgpio_chip *bgc = &chip->bgc; const struct etraxfs_gpio_port *port = &info->ports[i]; + unsigned long flags = BGPIOF_READ_OUTPUT_REG_SET; + void __iomem *dat = regs + port->din; + void __iomem *set = regs + port->dout; + void __iomem *dirout = regs + port->oe; + + chip->block = block; + + if (dirout == set) { + dirout = set = NULL; + flags = BGPIOF_NO_OUTPUT; + } ret = bgpio_init(bgc, dev, 4, - regs + port->din, /* dat */ - regs + port->dout, /* set */ - NULL, /* clr */ - regs + port->oe, /* dirout */ - NULL, /* dirin */ - BGPIOF_UNREADABLE_REG_SET); - if (ret) - return ret; + dat, set, NULL, dirout, NULL, + flags); + if (ret) { + dev_err(dev, "Unable to init port %s\n", + port->label); + continue; + } bgc->gc.ngpio = port->ngpio; bgc->gc.label = port->label; @@ -152,9 +439,21 @@ static int etraxfs_gpio_probe(struct platform_device *pdev) bgc->gc.of_xlate = etraxfs_gpio_of_xlate; ret = gpiochip_add(&bgc->gc); - if (ret) + if (ret) { dev_err(dev, "Unable to register port %s\n", bgc->gc.label); + continue; + } + + if (i > 0 && !allportsirq) + continue; + + ret = gpiochip_irqchip_add(&bgc->gc, &etraxfs_gpio_irq_chip, 0, + handle_level_irq, IRQ_TYPE_NONE); + if (ret) { + dev_err(dev, "Unable to add irqchip to port %s\n", + bgc->gc.label); + } } return 0; diff --git a/drivers/gpio/gpio-generic.c b/drivers/gpio/gpio-generic.c index 9bda3727fac12d..a3f07537fe6250 100644 --- a/drivers/gpio/gpio-generic.c +++ b/drivers/gpio/gpio-generic.c @@ -153,6 +153,10 @@ static int bgpio_get(struct gpio_chip *gc, unsigned int gpio) return !!(bgc->read_reg(bgc->reg_dat) & bgc->pin2mask(bgc, gpio)); } +static void bgpio_set_none(struct gpio_chip *gc, unsigned int gpio, int val) +{ +} + static void bgpio_set(struct gpio_chip *gc, unsigned int gpio, int val) { struct bgpio_chip *bgc = to_bgpio_chip(gc); @@ -279,6 +283,12 @@ static int bgpio_simple_dir_in(struct gpio_chip *gc, unsigned int gpio) return 0; } +static int bgpio_dir_out_err(struct gpio_chip *gc, unsigned int gpio, + int val) +{ + return -EINVAL; +} + static int bgpio_simple_dir_out(struct gpio_chip *gc, unsigned int gpio, int val) { @@ -302,6 +312,14 @@ static int bgpio_dir_in(struct gpio_chip *gc, unsigned int gpio) return 0; } +static int bgpio_get_dir(struct gpio_chip *gc, unsigned int gpio) +{ + struct bgpio_chip *bgc = to_bgpio_chip(gc); + + return (bgc->read_reg(bgc->reg_dir) & bgc->pin2mask(bgc, gpio)) ? + GPIOF_DIR_OUT : GPIOF_DIR_IN; +} + static int bgpio_dir_out(struct gpio_chip *gc, unsigned int gpio, int val) { struct bgpio_chip *bgc = to_bgpio_chip(gc); @@ -351,6 +369,14 @@ static int bgpio_dir_out_inv(struct gpio_chip *gc, unsigned int gpio, int val) return 0; } +static int bgpio_get_dir_inv(struct gpio_chip *gc, unsigned int gpio) +{ + struct bgpio_chip *bgc = to_bgpio_chip(gc); + + return (bgc->read_reg(bgc->reg_dir) & bgc->pin2mask(bgc, gpio)) ? + GPIOF_DIR_IN : GPIOF_DIR_OUT; +} + static int bgpio_setup_accessors(struct device *dev, struct bgpio_chip *bgc, bool bit_be, @@ -444,6 +470,9 @@ static int bgpio_setup_io(struct bgpio_chip *bgc, bgc->reg_set = set; bgc->gc.set = bgpio_set_set; bgc->gc.set_multiple = bgpio_set_multiple_set; + } else if (flags & BGPIOF_NO_OUTPUT) { + bgc->gc.set = bgpio_set_none; + bgc->gc.set_multiple = NULL; } else { bgc->gc.set = bgpio_set; bgc->gc.set_multiple = bgpio_set_multiple; @@ -460,7 +489,8 @@ static int bgpio_setup_io(struct bgpio_chip *bgc, static int bgpio_setup_direction(struct bgpio_chip *bgc, void __iomem *dirout, - void __iomem *dirin) + void __iomem *dirin, + unsigned long flags) { if (dirout && dirin) { return -EINVAL; @@ -468,12 +498,17 @@ static int bgpio_setup_direction(struct bgpio_chip *bgc, bgc->reg_dir = dirout; bgc->gc.direction_output = bgpio_dir_out; bgc->gc.direction_input = bgpio_dir_in; + bgc->gc.get_direction = bgpio_get_dir; } else if (dirin) { bgc->reg_dir = dirin; bgc->gc.direction_output = bgpio_dir_out_inv; bgc->gc.direction_input = bgpio_dir_in_inv; + bgc->gc.get_direction = bgpio_get_dir_inv; } else { - bgc->gc.direction_output = bgpio_simple_dir_out; + if (flags & BGPIOF_NO_OUTPUT) + bgc->gc.direction_output = bgpio_dir_out_err; + else + bgc->gc.direction_output = bgpio_simple_dir_out; bgc->gc.direction_input = bgpio_simple_dir_in; } @@ -525,7 +560,7 @@ int bgpio_init(struct bgpio_chip *bgc, struct device *dev, if (ret) return ret; - ret = bgpio_setup_direction(bgc, dirout, dirin); + ret = bgpio_setup_direction(bgc, dirout, dirin, flags); if (ret) return ret; diff --git a/drivers/gpio/gpio-grgpio.c b/drivers/gpio/gpio-grgpio.c index 0a8f7617e72e99..801423fe8143d2 100644 --- a/drivers/gpio/gpio-grgpio.c +++ b/drivers/gpio/gpio-grgpio.c @@ -104,17 +104,12 @@ static void grgpio_set_imask(struct grgpio_priv *priv, unsigned int offset, { struct bgpio_chip *bgc = &priv->bgc; unsigned long mask = bgc->pin2mask(bgc, offset); - unsigned long flags; - - spin_lock_irqsave(&bgc->lock, flags); if (val) priv->imask |= mask; else priv->imask &= ~mask; bgc->write_reg(priv->regs + GRGPIO_IMASK, priv->imask); - - spin_unlock_irqrestore(&bgc->lock, flags); } static int grgpio_to_irq(struct gpio_chip *gc, unsigned offset) @@ -180,16 +175,26 @@ static void grgpio_irq_mask(struct irq_data *d) { struct grgpio_priv *priv = irq_data_get_irq_chip_data(d); int offset = d->hwirq; + unsigned long flags; + + spin_lock_irqsave(&priv->bgc.lock, flags); grgpio_set_imask(priv, offset, 0); + + spin_unlock_irqrestore(&priv->bgc.lock, flags); } static void grgpio_irq_unmask(struct irq_data *d) { struct grgpio_priv *priv = irq_data_get_irq_chip_data(d); int offset = d->hwirq; + unsigned long flags; + + spin_lock_irqsave(&priv->bgc.lock, flags); grgpio_set_imask(priv, offset, 1); + + spin_unlock_irqrestore(&priv->bgc.lock, flags); } static struct irq_chip grgpio_irq_chip = { @@ -281,12 +286,7 @@ static int grgpio_irq_map(struct irq_domain *d, unsigned int irq, irq_set_chip_data(irq, priv); irq_set_chip_and_handler(irq, &grgpio_irq_chip, handle_simple_irq); - irq_clear_status_flags(irq, IRQ_NOREQUEST); -#ifdef CONFIG_ARM - set_irq_flags(irq, IRQF_VALID); -#else irq_set_noprobe(irq); -#endif return ret; } @@ -301,9 +301,6 @@ static void grgpio_irq_unmap(struct irq_domain *d, unsigned int irq) int ngpio = priv->bgc.gc.ngpio; int i; -#ifdef CONFIG_ARM - set_irq_flags(irq, 0); -#endif irq_set_chip_and_handler(irq, NULL, NULL); irq_set_chip_data(irq, NULL); diff --git a/drivers/gpio/gpio-max732x.c b/drivers/gpio/gpio-max732x.c index 7d3c90e9da71a3..8c5252c6c32776 100644 --- a/drivers/gpio/gpio-max732x.c +++ b/drivers/gpio/gpio-max732x.c @@ -685,9 +685,14 @@ static int max732x_probe(struct i2c_client *client, mutex_init(&chip->lock); - max732x_readb(chip, is_group_a(chip, 0), &chip->reg_out[0]); - if (nr_port > 8) - max732x_readb(chip, is_group_a(chip, 8), &chip->reg_out[1]); + ret = max732x_readb(chip, is_group_a(chip, 0), &chip->reg_out[0]); + if (ret) + goto out_failed; + if (nr_port > 8) { + ret = max732x_readb(chip, is_group_a(chip, 8), &chip->reg_out[1]); + if (ret) + goto out_failed; + } ret = gpiochip_add(&chip->gpio_chip); if (ret) diff --git a/drivers/gpio/gpio-mcp23s08.c b/drivers/gpio/gpio-mcp23s08.c index 2fc7ff852d1676..73db7ecd7ffd9c 100644 --- a/drivers/gpio/gpio-mcp23s08.c +++ b/drivers/gpio/gpio-mcp23s08.c @@ -507,11 +507,7 @@ static int mcp23s08_irq_setup(struct mcp23s08 *mcp) irq_set_chip_data(irq, mcp); irq_set_chip(irq, &mcp23s08_irq_chip); irq_set_nested_thread(irq, true); -#ifdef CONFIG_ARM - set_irq_flags(irq, IRQF_VALID); -#else irq_set_noprobe(irq); -#endif } return 0; } diff --git a/drivers/gpio/gpio-mpc8xxx.c b/drivers/gpio/gpio-mpc8xxx.c index 20aa66f34f6e04..8ef7a12de98375 100644 --- a/drivers/gpio/gpio-mpc8xxx.c +++ b/drivers/gpio/gpio-mpc8xxx.c @@ -32,7 +32,7 @@ struct mpc8xxx_gpio_chip { struct of_mm_gpio_chip mm_gc; - spinlock_t lock; + raw_spinlock_t lock; /* * shadowed data register to be able to clear/set output pins in @@ -95,7 +95,7 @@ static void mpc8xxx_gpio_set(struct gpio_chip *gc, unsigned int gpio, int val) struct mpc8xxx_gpio_chip *mpc8xxx_gc = to_mpc8xxx_gpio_chip(mm); unsigned long flags; - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); if (val) mpc8xxx_gc->data |= mpc8xxx_gpio2mask(gpio); @@ -104,7 +104,7 @@ static void mpc8xxx_gpio_set(struct gpio_chip *gc, unsigned int gpio, int val) out_be32(mm->regs + GPIO_DAT, mpc8xxx_gc->data); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); } static void mpc8xxx_gpio_set_multiple(struct gpio_chip *gc, @@ -115,7 +115,7 @@ static void mpc8xxx_gpio_set_multiple(struct gpio_chip *gc, unsigned long flags; int i; - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); for (i = 0; i < gc->ngpio; i++) { if (*mask == 0) @@ -130,7 +130,7 @@ static void mpc8xxx_gpio_set_multiple(struct gpio_chip *gc, out_be32(mm->regs + GPIO_DAT, mpc8xxx_gc->data); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); } static int mpc8xxx_gpio_dir_in(struct gpio_chip *gc, unsigned int gpio) @@ -139,11 +139,11 @@ static int mpc8xxx_gpio_dir_in(struct gpio_chip *gc, unsigned int gpio) struct mpc8xxx_gpio_chip *mpc8xxx_gc = to_mpc8xxx_gpio_chip(mm); unsigned long flags; - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); clrbits32(mm->regs + GPIO_DIR, mpc8xxx_gpio2mask(gpio)); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); return 0; } @@ -156,11 +156,11 @@ static int mpc8xxx_gpio_dir_out(struct gpio_chip *gc, unsigned int gpio, int val mpc8xxx_gpio_set(gc, gpio, val); - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); setbits32(mm->regs + GPIO_DIR, mpc8xxx_gpio2mask(gpio)); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); return 0; } @@ -174,6 +174,15 @@ static int mpc5121_gpio_dir_out(struct gpio_chip *gc, unsigned int gpio, int val return mpc8xxx_gpio_dir_out(gc, gpio, val); } +static int mpc5125_gpio_dir_out(struct gpio_chip *gc, unsigned int gpio, int val) +{ + /* GPIO 0..3 are input only on MPC5125 */ + if (gpio <= 3) + return -EINVAL; + + return mpc8xxx_gpio_dir_out(gc, gpio, val); +} + static int mpc8xxx_gpio_to_irq(struct gpio_chip *gc, unsigned offset) { struct of_mm_gpio_chip *mm = to_of_mm_gpio_chip(gc); @@ -206,11 +215,11 @@ static void mpc8xxx_irq_unmask(struct irq_data *d) struct of_mm_gpio_chip *mm = &mpc8xxx_gc->mm_gc; unsigned long flags; - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); setbits32(mm->regs + GPIO_IMR, mpc8xxx_gpio2mask(irqd_to_hwirq(d))); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); } static void mpc8xxx_irq_mask(struct irq_data *d) @@ -219,11 +228,11 @@ static void mpc8xxx_irq_mask(struct irq_data *d) struct of_mm_gpio_chip *mm = &mpc8xxx_gc->mm_gc; unsigned long flags; - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); clrbits32(mm->regs + GPIO_IMR, mpc8xxx_gpio2mask(irqd_to_hwirq(d))); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); } static void mpc8xxx_irq_ack(struct irq_data *d) @@ -242,17 +251,17 @@ static int mpc8xxx_irq_set_type(struct irq_data *d, unsigned int flow_type) switch (flow_type) { case IRQ_TYPE_EDGE_FALLING: - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); setbits32(mm->regs + GPIO_ICR, mpc8xxx_gpio2mask(irqd_to_hwirq(d))); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); break; case IRQ_TYPE_EDGE_BOTH: - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); clrbits32(mm->regs + GPIO_ICR, mpc8xxx_gpio2mask(irqd_to_hwirq(d))); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); break; default: @@ -282,22 +291,22 @@ static int mpc512x_irq_set_type(struct irq_data *d, unsigned int flow_type) switch (flow_type) { case IRQ_TYPE_EDGE_FALLING: case IRQ_TYPE_LEVEL_LOW: - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); clrsetbits_be32(reg, 3 << shift, 2 << shift); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); break; case IRQ_TYPE_EDGE_RISING: case IRQ_TYPE_LEVEL_HIGH: - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); clrsetbits_be32(reg, 3 << shift, 1 << shift); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); break; case IRQ_TYPE_EDGE_BOTH: - spin_lock_irqsave(&mpc8xxx_gc->lock, flags); + raw_spin_lock_irqsave(&mpc8xxx_gc->lock, flags); clrbits32(reg, 3 << shift); - spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); + raw_spin_unlock_irqrestore(&mpc8xxx_gc->lock, flags); break; default: @@ -312,17 +321,13 @@ static struct irq_chip mpc8xxx_irq_chip = { .irq_unmask = mpc8xxx_irq_unmask, .irq_mask = mpc8xxx_irq_mask, .irq_ack = mpc8xxx_irq_ack, + /* this might get overwritten in mpc8xxx_probe() */ .irq_set_type = mpc8xxx_irq_set_type, }; static int mpc8xxx_gpio_irq_map(struct irq_domain *h, unsigned int irq, irq_hw_number_t hwirq) { - struct mpc8xxx_gpio_chip *mpc8xxx_gc = h->host_data; - - if (mpc8xxx_gc->of_dev_id_data) - mpc8xxx_irq_chip.irq_set_type = mpc8xxx_gc->of_dev_id_data; - irq_set_chip_data(irq, h->host_data); irq_set_chip_and_handler(irq, &mpc8xxx_irq_chip, handle_level_irq); @@ -334,11 +339,38 @@ static const struct irq_domain_ops mpc8xxx_gpio_irq_ops = { .xlate = irq_domain_xlate_twocell, }; -static struct of_device_id mpc8xxx_gpio_ids[] = { +struct mpc8xxx_gpio_devtype { + int (*gpio_dir_out)(struct gpio_chip *, unsigned int, int); + int (*gpio_get)(struct gpio_chip *, unsigned int); + int (*irq_set_type)(struct irq_data *, unsigned int); +}; + +static const struct mpc8xxx_gpio_devtype mpc512x_gpio_devtype = { + .gpio_dir_out = mpc5121_gpio_dir_out, + .irq_set_type = mpc512x_irq_set_type, +}; + +static const struct mpc8xxx_gpio_devtype mpc5125_gpio_devtype = { + .gpio_dir_out = mpc5125_gpio_dir_out, + .irq_set_type = mpc512x_irq_set_type, +}; + +static const struct mpc8xxx_gpio_devtype mpc8572_gpio_devtype = { + .gpio_get = mpc8572_gpio_get, +}; + +static const struct mpc8xxx_gpio_devtype mpc8xxx_gpio_devtype_default = { + .gpio_dir_out = mpc8xxx_gpio_dir_out, + .gpio_get = mpc8xxx_gpio_get, + .irq_set_type = mpc8xxx_irq_set_type, +}; + +static const struct of_device_id mpc8xxx_gpio_ids[] = { { .compatible = "fsl,mpc8349-gpio", }, - { .compatible = "fsl,mpc8572-gpio", }, + { .compatible = "fsl,mpc8572-gpio", .data = &mpc8572_gpio_devtype, }, { .compatible = "fsl,mpc8610-gpio", }, - { .compatible = "fsl,mpc5121-gpio", .data = mpc512x_irq_set_type, }, + { .compatible = "fsl,mpc5121-gpio", .data = &mpc512x_gpio_devtype, }, + { .compatible = "fsl,mpc5125-gpio", .data = &mpc5125_gpio_devtype, }, { .compatible = "fsl,pq3-gpio", }, { .compatible = "fsl,qoriq-gpio", }, {} @@ -351,6 +383,8 @@ static int mpc8xxx_probe(struct platform_device *pdev) struct of_mm_gpio_chip *mm_gc; struct gpio_chip *gc; const struct of_device_id *id; + const struct mpc8xxx_gpio_devtype *devtype = + of_device_get_match_data(&pdev->dev); int ret; mpc8xxx_gc = devm_kzalloc(&pdev->dev, sizeof(*mpc8xxx_gc), GFP_KERNEL); @@ -359,7 +393,7 @@ static int mpc8xxx_probe(struct platform_device *pdev) platform_set_drvdata(pdev, mpc8xxx_gc); - spin_lock_init(&mpc8xxx_gc->lock); + raw_spin_lock_init(&mpc8xxx_gc->lock); mm_gc = &mpc8xxx_gc->mm_gc; gc = &mm_gc->gc; @@ -367,10 +401,18 @@ static int mpc8xxx_probe(struct platform_device *pdev) mm_gc->save_regs = mpc8xxx_gpio_save_regs; gc->ngpio = MPC8XXX_GPIO_PINS; gc->direction_input = mpc8xxx_gpio_dir_in; - gc->direction_output = of_device_is_compatible(np, "fsl,mpc5121-gpio") ? - mpc5121_gpio_dir_out : mpc8xxx_gpio_dir_out; - gc->get = of_device_is_compatible(np, "fsl,mpc8572-gpio") ? - mpc8572_gpio_get : mpc8xxx_gpio_get; + + if (!devtype) + devtype = &mpc8xxx_gpio_devtype_default; + + /* + * It's assumed that only a single type of gpio controller is available + * on the current machine, so overwriting global data is fine. + */ + mpc8xxx_irq_chip.irq_set_type = devtype->irq_set_type; + + gc->direction_output = devtype->gpio_dir_out ?: mpc8xxx_gpio_dir_out; + gc->get = devtype->gpio_get ?: mpc8xxx_gpio_get; gc->set = mpc8xxx_gpio_set; gc->set_multiple = mpc8xxx_gpio_set_multiple; gc->to_irq = mpc8xxx_gpio_to_irq; @@ -396,8 +438,8 @@ static int mpc8xxx_probe(struct platform_device *pdev) out_be32(mm_gc->regs + GPIO_IER, 0xffffffff); out_be32(mm_gc->regs + GPIO_IMR, 0); - irq_set_handler_data(mpc8xxx_gc->irqn, mpc8xxx_gc); - irq_set_chained_handler(mpc8xxx_gc->irqn, mpc8xxx_gpio_irq_cascade); + irq_set_chained_handler_and_data(mpc8xxx_gc->irqn, + mpc8xxx_gpio_irq_cascade, mpc8xxx_gc); return 0; } @@ -407,8 +449,7 @@ static int mpc8xxx_remove(struct platform_device *pdev) struct mpc8xxx_gpio_chip *mpc8xxx_gc = platform_get_drvdata(pdev); if (mpc8xxx_gc->irq) { - irq_set_handler_data(mpc8xxx_gc->irqn, NULL); - irq_set_chained_handler(mpc8xxx_gc->irqn, NULL); + irq_set_chained_handler_and_data(mpc8xxx_gc->irqn, NULL, NULL); irq_domain_remove(mpc8xxx_gc->irq); } diff --git a/drivers/gpio/gpio-msm-v2.c b/drivers/gpio/gpio-msm-v2.c index 52ff18229fdc87..d2012cfb557181 100644 --- a/drivers/gpio/gpio-msm-v2.c +++ b/drivers/gpio/gpio-msm-v2.c @@ -187,14 +187,6 @@ static int msm_gpio_to_irq(struct gpio_chip *chip, unsigned offset) return irq_create_mapping(domain, offset); } -static inline int msm_irq_to_gpio(struct gpio_chip *chip, unsigned irq) -{ - struct irq_data *irq_data = irq_get_irq_data(irq); - - return irq_data->hwirq; -} - - /* For dual-edge interrupts in software, since the hardware has no * such support: * @@ -238,7 +230,7 @@ static void msm_gpio_update_dual_edge_pos(unsigned gpio) static void msm_gpio_irq_ack(struct irq_data *d) { - int gpio = msm_irq_to_gpio(&msm_gpio.gpio_chip, d->irq); + int gpio = d->hwirq; writel(BIT(INTR_STATUS), GPIO_INTR_STATUS(gpio)); if (test_bit(gpio, msm_gpio.dual_edge_irqs)) @@ -247,8 +239,8 @@ static void msm_gpio_irq_ack(struct irq_data *d) static void msm_gpio_irq_mask(struct irq_data *d) { - int gpio = msm_irq_to_gpio(&msm_gpio.gpio_chip, d->irq); unsigned long irq_flags; + int gpio = d->hwirq; spin_lock_irqsave(&tlmm_lock, irq_flags); writel(TARGET_PROC_NONE, GPIO_INTR_CFG_SU(gpio)); @@ -259,8 +251,8 @@ static void msm_gpio_irq_mask(struct irq_data *d) static void msm_gpio_irq_unmask(struct irq_data *d) { - int gpio = msm_irq_to_gpio(&msm_gpio.gpio_chip, d->irq); unsigned long irq_flags; + int gpio = d->hwirq; spin_lock_irqsave(&tlmm_lock, irq_flags); __set_bit(gpio, msm_gpio.enabled_irqs); @@ -271,8 +263,8 @@ static void msm_gpio_irq_unmask(struct irq_data *d) static int msm_gpio_irq_set_type(struct irq_data *d, unsigned int flow_type) { - int gpio = msm_irq_to_gpio(&msm_gpio.gpio_chip, d->irq); unsigned long irq_flags; + int gpio = d->hwirq; uint32_t bits; spin_lock_irqsave(&tlmm_lock, irq_flags); @@ -281,14 +273,14 @@ static int msm_gpio_irq_set_type(struct irq_data *d, unsigned int flow_type) if (flow_type & IRQ_TYPE_EDGE_BOTH) { bits |= BIT(INTR_DECT_CTL); - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); if ((flow_type & IRQ_TYPE_EDGE_BOTH) == IRQ_TYPE_EDGE_BOTH) __set_bit(gpio, msm_gpio.dual_edge_irqs); else __clear_bit(gpio, msm_gpio.dual_edge_irqs); } else { bits &= ~BIT(INTR_DECT_CTL); - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); __clear_bit(gpio, msm_gpio.dual_edge_irqs); } @@ -331,7 +323,7 @@ static void msm_summary_irq_handler(unsigned int irq, struct irq_desc *desc) static int msm_gpio_irq_set_wake(struct irq_data *d, unsigned int on) { - int gpio = msm_irq_to_gpio(&msm_gpio.gpio_chip, d->irq); + int gpio = d->hwirq; if (on) { if (bitmap_empty(msm_gpio.wake_irqs, MAX_NR_GPIO)) @@ -363,7 +355,6 @@ static int msm_gpio_irq_domain_map(struct irq_domain *d, unsigned int irq, irq_set_lockdep_class(irq, &msm_gpio_lock_class); irq_set_chip_and_handler(irq, &msm_gpio_irq_chip, handle_level_irq); - set_irq_flags(irq, IRQF_VALID); return 0; } diff --git a/drivers/gpio/gpio-mvebu.c b/drivers/gpio/gpio-mvebu.c index 1a54205860f590..b396bf3bf29464 100644 --- a/drivers/gpio/gpio-mvebu.c +++ b/drivers/gpio/gpio-mvebu.c @@ -458,9 +458,9 @@ static int mvebu_gpio_irq_set_type(struct irq_data *d, unsigned int type) return 0; } -static void mvebu_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) +static void mvebu_gpio_irq_handler(unsigned int __irq, struct irq_desc *desc) { - struct mvebu_gpio_chip *mvchip = irq_get_handler_data(irq); + struct mvebu_gpio_chip *mvchip = irq_desc_get_handler_data(desc); struct irq_chip *chip = irq_desc_get_chip(desc); u32 cause, type; int i; @@ -787,8 +787,8 @@ static int mvebu_gpio_probe(struct platform_device *pdev) if (irq < 0) continue; - irq_set_handler_data(irq, mvchip); - irq_set_chained_handler(irq, mvebu_gpio_irq_handler); + irq_set_chained_handler_and_data(irq, mvebu_gpio_irq_handler, + mvchip); } mvchip->irqbase = irq_alloc_descs(-1, 0, ngpios, -1); diff --git a/drivers/gpio/gpio-mxc.c b/drivers/gpio/gpio-mxc.c index ec1eb1b7250ff1..b752b560126e71 100644 --- a/drivers/gpio/gpio-mxc.c +++ b/drivers/gpio/gpio-mxc.c @@ -275,8 +275,8 @@ static void mxc_gpio_irq_handler(struct mxc_gpio_port *port, u32 irq_stat) static void mx3_gpio_irq_handler(u32 irq, struct irq_desc *desc) { u32 irq_stat; - struct mxc_gpio_port *port = irq_get_handler_data(irq); - struct irq_chip *chip = irq_get_chip(irq); + struct mxc_gpio_port *port = irq_desc_get_handler_data(desc); + struct irq_chip *chip = irq_desc_get_chip(desc); chained_irq_enter(chip, desc); @@ -292,7 +292,7 @@ static void mx2_gpio_irq_handler(u32 irq, struct irq_desc *desc) { u32 irq_msk, irq_stat; struct mxc_gpio_port *port; - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); chained_irq_enter(chip, desc); @@ -339,7 +339,7 @@ static int gpio_set_wake_irq(struct irq_data *d, u32 enable) return 0; } -static void __init mxc_gpio_init_gc(struct mxc_gpio_port *port, int irq_base) +static void mxc_gpio_init_gc(struct mxc_gpio_port *port, int irq_base) { struct irq_chip_generic *gc; struct irq_chip_type *ct; @@ -354,6 +354,7 @@ static void __init mxc_gpio_init_gc(struct mxc_gpio_port *port, int irq_base) ct->chip.irq_unmask = irq_gc_mask_set_bit; ct->chip.irq_set_type = gpio_set_irq_type; ct->chip.irq_set_wake = gpio_set_wake_irq; + ct->chip.flags = IRQCHIP_MASK_ON_SUSPEND; ct->regs.ack = GPIO_ISR; ct->regs.mask = GPIO_IMR; diff --git a/drivers/gpio/gpio-mxs.c b/drivers/gpio/gpio-mxs.c index 551d15d7c369cc..b7f383eb18d91e 100644 --- a/drivers/gpio/gpio-mxs.c +++ b/drivers/gpio/gpio-mxs.c @@ -157,7 +157,7 @@ static void mxs_flip_edge(struct mxs_gpio_port *port, u32 gpio) static void mxs_gpio_irq_handler(u32 irq, struct irq_desc *desc) { u32 irq_stat; - struct mxs_gpio_port *port = irq_get_handler_data(irq); + struct mxs_gpio_port *port = irq_desc_get_handler_data(desc); desc->irq_data.chip->irq_ack(&desc->irq_data); diff --git a/drivers/gpio/gpio-omap.c b/drivers/gpio/gpio-omap.c index 61a731ff9a076f..2ae0d47e955443 100644 --- a/drivers/gpio/gpio-omap.c +++ b/drivers/gpio/gpio-omap.c @@ -29,6 +29,7 @@ #include #define OFF_MODE 1 +#define OMAP4_GPIO_DEBOUNCINGTIME_MASK 0xFF static LIST_HEAD(omap_gpio_list); @@ -57,7 +58,7 @@ struct gpio_bank { u32 saved_datain; u32 level_mask; u32 toggle_mask; - spinlock_t lock; + raw_spinlock_t lock; struct gpio_chip chip; struct clk *dbck; u32 mod_usage; @@ -175,7 +176,7 @@ static inline void omap_gpio_rmw(void __iomem *base, u32 reg, u32 mask, bool set static inline void omap_gpio_dbck_enable(struct gpio_bank *bank) { if (bank->dbck_enable_mask && !bank->dbck_enabled) { - clk_prepare_enable(bank->dbck); + clk_enable(bank->dbck); bank->dbck_enabled = true; writel_relaxed(bank->dbck_enable_mask, @@ -193,7 +194,7 @@ static inline void omap_gpio_dbck_disable(struct gpio_bank *bank) */ writel_relaxed(0, bank->base + bank->regs->debounce_en); - clk_disable_unprepare(bank->dbck); + clk_disable(bank->dbck); bank->dbck_enabled = false; } } @@ -204,8 +205,9 @@ static inline void omap_gpio_dbck_disable(struct gpio_bank *bank) * @offset: the gpio number on this @bank * @debounce: debounce time to use * - * OMAP's debounce time is in 31us steps so we need - * to convert and round up to the closest unit. + * OMAP's debounce time is in 31us steps + * = (GPIO_DEBOUNCINGTIME[7:0].DEBOUNCETIME + 1) x 31 + * so we need to convert and round up to the closest unit. */ static void omap2_set_gpio_debounce(struct gpio_bank *bank, unsigned offset, unsigned debounce) @@ -213,34 +215,33 @@ static void omap2_set_gpio_debounce(struct gpio_bank *bank, unsigned offset, void __iomem *reg; u32 val; u32 l; + bool enable = !!debounce; if (!bank->dbck_flag) return; - if (debounce < 32) - debounce = 0x01; - else if (debounce > 7936) - debounce = 0xff; - else - debounce = (debounce / 0x1f) - 1; + if (enable) { + debounce = DIV_ROUND_UP(debounce, 31) - 1; + debounce &= OMAP4_GPIO_DEBOUNCINGTIME_MASK; + } l = BIT(offset); - clk_prepare_enable(bank->dbck); + clk_enable(bank->dbck); reg = bank->base + bank->regs->debounce; writel_relaxed(debounce, reg); reg = bank->base + bank->regs->debounce_en; val = readl_relaxed(reg); - if (debounce) + if (enable) val |= l; else val &= ~l; bank->dbck_enable_mask = val; writel_relaxed(val, reg); - clk_disable_unprepare(bank->dbck); + clk_disable(bank->dbck); /* * Enable debounce clock per module. * This call is mandatory because in omap_gpio_request() when @@ -285,7 +286,7 @@ static void omap_clear_gpio_debounce(struct gpio_bank *bank, unsigned offset) bank->context.debounce = 0; writel_relaxed(bank->context.debounce, bank->base + bank->regs->debounce); - clk_disable_unprepare(bank->dbck); + clk_disable(bank->dbck); bank->dbck_enabled = false; } } @@ -498,24 +499,24 @@ static int omap_gpio_irq_type(struct irq_data *d, unsigned type) if (!BANK_USED(bank)) pm_runtime_get_sync(bank->dev); - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); retval = omap_set_gpio_triggering(bank, offset, type); if (retval) { - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); goto error; } omap_gpio_init_irq(bank, offset); if (!omap_gpio_is_input(bank, offset)) { - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); retval = -EINVAL; goto error; } - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); if (type & (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_LEVEL_HIGH)) - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); else if (type & (IRQ_TYPE_EDGE_FALLING | IRQ_TYPE_EDGE_RISING)) - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); return 0; @@ -636,14 +637,14 @@ static int omap_set_gpio_wakeup(struct gpio_bank *bank, unsigned offset, return -EINVAL; } - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); if (enable) bank->context.wake_en |= gpio_bit; else bank->context.wake_en &= ~gpio_bit; writel_relaxed(bank->context.wake_en, bank->base + bank->regs->wkup_en); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } @@ -669,10 +670,10 @@ static int omap_gpio_request(struct gpio_chip *chip, unsigned offset) if (!BANK_USED(bank)) pm_runtime_get_sync(bank->dev); - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); omap_enable_gpio_module(bank, offset); bank->mod_usage |= BIT(offset); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } @@ -682,14 +683,14 @@ static void omap_gpio_free(struct gpio_chip *chip, unsigned offset) struct gpio_bank *bank = container_of(chip, struct gpio_bank, chip); unsigned long flags; - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); bank->mod_usage &= ~(BIT(offset)); if (!LINE_USED(bank->irq_usage, offset)) { omap_set_gpio_direction(bank, offset, 1); omap_clear_gpio_debounce(bank, offset); } omap_disable_gpio_module(bank, offset); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); /* * If this is the last gpio to be freed in the bank, @@ -716,7 +717,8 @@ static void omap_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) struct gpio_bank *bank; int unmasked = 0; struct irq_chip *irqchip = irq_desc_get_chip(desc); - struct gpio_chip *chip = irq_get_handler_data(irq); + struct gpio_chip *chip = irq_desc_get_handler_data(desc); + unsigned long lock_flags; chained_irq_enter(irqchip, desc); @@ -731,6 +733,8 @@ static void omap_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) u32 isr_saved, level_mask = 0; u32 enabled; + raw_spin_lock_irqsave(&bank->lock, lock_flags); + enabled = omap_get_gpio_irqbank_mask(bank); isr_saved = isr = readl_relaxed(isr_reg) & enabled; @@ -744,6 +748,8 @@ static void omap_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) omap_clear_gpio_irqbank(bank, isr_saved & ~level_mask); omap_enable_gpio_irqbank(bank, isr_saved & ~level_mask); + raw_spin_unlock_irqrestore(&bank->lock, lock_flags); + /* if there is only edge sensitive GPIO pin interrupts configured, we could unmask GPIO bank interrupt immediately */ if (!level_mask && !unmasked) { @@ -758,6 +764,7 @@ static void omap_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) bit = __ffs(isr); isr &= ~(BIT(bit)); + raw_spin_lock_irqsave(&bank->lock, lock_flags); /* * Some chips can't respond to both rising and falling * at the same time. If this irq was requested with @@ -768,6 +775,8 @@ static void omap_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) if (bank->toggle_mask & (BIT(bit))) omap_toggle_gpio_edge_triggering(bank, bit); + raw_spin_unlock_irqrestore(&bank->lock, lock_flags); + generic_handle_irq(irq_find_mapping(bank->chip.irqdomain, bit)); } @@ -791,7 +800,7 @@ static unsigned int omap_gpio_irq_startup(struct irq_data *d) if (!BANK_USED(bank)) pm_runtime_get_sync(bank->dev); - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); if (!LINE_USED(bank->mod_usage, offset)) omap_set_gpio_direction(bank, offset, 1); @@ -800,12 +809,12 @@ static unsigned int omap_gpio_irq_startup(struct irq_data *d) omap_enable_gpio_module(bank, offset); bank->irq_usage |= BIT(offset); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); omap_gpio_unmask_irq(d); return 0; err: - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); if (!BANK_USED(bank)) pm_runtime_put(bank->dev); return -EINVAL; @@ -817,7 +826,7 @@ static void omap_gpio_irq_shutdown(struct irq_data *d) unsigned long flags; unsigned offset = d->hwirq; - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); bank->irq_usage &= ~(BIT(offset)); omap_set_gpio_irqenable(bank, offset, 0); omap_clear_gpio_irqstatus(bank, offset); @@ -825,7 +834,7 @@ static void omap_gpio_irq_shutdown(struct irq_data *d) if (!LINE_USED(bank->mod_usage, offset)) omap_clear_gpio_debounce(bank, offset); omap_disable_gpio_module(bank, offset); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); /* * If this is the last IRQ to be freed in the bank, @@ -849,10 +858,10 @@ static void omap_gpio_mask_irq(struct irq_data *d) unsigned offset = d->hwirq; unsigned long flags; - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); omap_set_gpio_irqenable(bank, offset, 0); omap_set_gpio_triggering(bank, offset, IRQ_TYPE_NONE); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); } static void omap_gpio_unmask_irq(struct irq_data *d) @@ -862,7 +871,7 @@ static void omap_gpio_unmask_irq(struct irq_data *d) u32 trigger = irqd_get_trigger_type(d); unsigned long flags; - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); if (trigger) omap_set_gpio_triggering(bank, offset, trigger); @@ -874,7 +883,7 @@ static void omap_gpio_unmask_irq(struct irq_data *d) } omap_set_gpio_irqenable(bank, offset, 1); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); } /*---------------------------------------------------------------------*/ @@ -887,9 +896,9 @@ static int omap_mpuio_suspend_noirq(struct device *dev) OMAP_MPUIO_GPIO_MASKIT / bank->stride; unsigned long flags; - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); writel_relaxed(0xffff & ~bank->context.wake_en, mask_reg); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } @@ -902,9 +911,9 @@ static int omap_mpuio_resume_noirq(struct device *dev) OMAP_MPUIO_GPIO_MASKIT / bank->stride; unsigned long flags; - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); writel_relaxed(bank->context.wake_en, mask_reg); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } @@ -950,9 +959,9 @@ static int omap_gpio_get_direction(struct gpio_chip *chip, unsigned offset) bank = container_of(chip, struct gpio_bank, chip); reg = bank->base + bank->regs->direction; - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); dir = !!(readl_relaxed(reg) & BIT(offset)); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return dir; } @@ -962,9 +971,9 @@ static int omap_gpio_input(struct gpio_chip *chip, unsigned offset) unsigned long flags; bank = container_of(chip, struct gpio_bank, chip); - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); omap_set_gpio_direction(bank, offset, 1); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } @@ -986,10 +995,10 @@ static int omap_gpio_output(struct gpio_chip *chip, unsigned offset, int value) unsigned long flags; bank = container_of(chip, struct gpio_bank, chip); - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); bank->set_dataout(bank, offset, value); omap_set_gpio_direction(bank, offset, 0); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } @@ -1001,9 +1010,9 @@ static int omap_gpio_debounce(struct gpio_chip *chip, unsigned offset, bank = container_of(chip, struct gpio_bank, chip); - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); omap2_set_gpio_debounce(bank, offset, debounce); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } @@ -1014,9 +1023,9 @@ static void omap_gpio_set(struct gpio_chip *chip, unsigned offset, int value) unsigned long flags; bank = container_of(chip, struct gpio_bank, chip); - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); bank->set_dataout(bank, offset, value); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); } /*---------------------------------------------------------------------*/ @@ -1061,10 +1070,6 @@ static void omap_gpio_mod_init(struct gpio_bank *bank) /* Initialize interface clk ungated, module enabled */ if (bank->regs->ctrl) writel_relaxed(0, base + bank->regs->ctrl); - - bank->dbck = clk_get(bank->dev, "dbclk"); - if (IS_ERR(bank->dbck)) - dev_err(bank->dev, "Could not get gpio dbck\n"); } static int omap_gpio_chip_init(struct gpio_bank *bank, struct irq_chip *irqc) @@ -1178,13 +1183,16 @@ static int omap_gpio_probe(struct platform_device *pdev) irqc->irq_set_wake = omap_gpio_wake_enable, irqc->name = dev_name(&pdev->dev); - res = platform_get_resource(pdev, IORESOURCE_IRQ, 0); - if (unlikely(!res)) { - dev_err(dev, "Invalid IRQ resource\n"); - return -ENODEV; + bank->irq = platform_get_irq(pdev, 0); + if (bank->irq <= 0) { + if (!bank->irq) + bank->irq = -ENXIO; + if (bank->irq != -EPROBE_DEFER) + dev_err(dev, + "can't get irq resource ret=%d\n", bank->irq); + return bank->irq; } - bank->irq = res->start; bank->dev = dev; bank->chip.dev = dev; bank->chip.owner = THIS_MODULE; @@ -1213,16 +1221,26 @@ static int omap_gpio_probe(struct platform_device *pdev) else bank->set_dataout = omap_set_gpio_dataout_mask; - spin_lock_init(&bank->lock); + raw_spin_lock_init(&bank->lock); /* Static mapping, never released */ res = platform_get_resource(pdev, IORESOURCE_MEM, 0); bank->base = devm_ioremap_resource(dev, res); if (IS_ERR(bank->base)) { - irq_domain_remove(bank->chip.irqdomain); return PTR_ERR(bank->base); } + if (bank->dbck_flag) { + bank->dbck = devm_clk_get(bank->dev, "dbclk"); + if (IS_ERR(bank->dbck)) { + dev_err(bank->dev, + "Could not get gpio dbck. Disable debounce\n"); + bank->dbck_flag = false; + } else { + clk_prepare(bank->dbck); + } + } + platform_set_drvdata(pdev, bank); pm_runtime_enable(bank->dev); @@ -1254,6 +1272,8 @@ static int omap_gpio_remove(struct platform_device *pdev) list_del(&bank->node); gpiochip_remove(&bank->chip); pm_runtime_disable(bank->dev); + if (bank->dbck_flag) + clk_unprepare(bank->dbck); return 0; } @@ -1271,7 +1291,7 @@ static int omap_gpio_runtime_suspend(struct device *dev) unsigned long flags; u32 wake_low, wake_hi; - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); /* * Only edges can generate a wakeup event to the PRCM. @@ -1324,7 +1344,7 @@ static int omap_gpio_runtime_suspend(struct device *dev) bank->get_context_loss_count(bank->dev); omap_gpio_dbck_disable(bank); - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } @@ -1339,7 +1359,7 @@ static int omap_gpio_runtime_resume(struct device *dev) unsigned long flags; int c; - spin_lock_irqsave(&bank->lock, flags); + raw_spin_lock_irqsave(&bank->lock, flags); /* * On the first resume during the probe, the context has not @@ -1375,14 +1395,14 @@ static int omap_gpio_runtime_resume(struct device *dev) if (c != bank->context_loss_count) { omap_gpio_restore_context(bank); } else { - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } } } if (!bank->workaround_enabled) { - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } @@ -1437,7 +1457,7 @@ static int omap_gpio_runtime_resume(struct device *dev) } bank->workaround_enabled = false; - spin_unlock_irqrestore(&bank->lock, flags); + raw_spin_unlock_irqrestore(&bank->lock, flags); return 0; } diff --git a/drivers/gpio/gpio-pcf857x.c b/drivers/gpio/gpio-pcf857x.c index 404f3c61ef9b19..1d4d9bc8b69db5 100644 --- a/drivers/gpio/gpio-pcf857x.c +++ b/drivers/gpio/gpio-pcf857x.c @@ -88,7 +88,6 @@ struct pcf857x { struct gpio_chip chip; struct i2c_client *client; struct mutex lock; /* protect 'out' */ - spinlock_t slock; /* protect irq demux */ unsigned out; /* software latch */ unsigned status; /* current status */ unsigned int irq_parent; @@ -185,23 +184,21 @@ static void pcf857x_set(struct gpio_chip *chip, unsigned offset, int value) static irqreturn_t pcf857x_irq(int irq, void *data) { struct pcf857x *gpio = data; - unsigned long change, i, status, flags; + unsigned long change, i, status; status = gpio->read(gpio->client); - spin_lock_irqsave(&gpio->slock, flags); - /* * call the interrupt handler iff gpio is used as * interrupt source, just to avoid bad irqs */ - + mutex_lock(&gpio->lock); change = (gpio->status ^ status) & gpio->irq_enabled; - for_each_set_bit(i, &change, gpio->chip.ngpio) - handle_nested_irq(irq_find_mapping(gpio->chip.irqdomain, i)); gpio->status = status; + mutex_unlock(&gpio->lock); - spin_unlock_irqrestore(&gpio->slock, flags); + for_each_set_bit(i, &change, gpio->chip.ngpio) + handle_nested_irq(irq_find_mapping(gpio->chip.irqdomain, i)); return IRQ_HANDLED; } @@ -293,7 +290,6 @@ static int pcf857x_probe(struct i2c_client *client, return -ENOMEM; mutex_init(&gpio->lock); - spin_lock_init(&gpio->slock); gpio->chip.base = pdata ? pdata->gpio_base : -1; gpio->chip.can_sleep = true; diff --git a/drivers/gpio/gpio-pch.c b/drivers/gpio/gpio-pch.c index 2d9a950ca2d46d..34ed176df15a9b 100644 --- a/drivers/gpio/gpio-pch.c +++ b/drivers/gpio/gpio-pch.c @@ -281,9 +281,9 @@ static int pch_irq_type(struct irq_data *d, unsigned int type) /* And the handler */ if (type & (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_LEVEL_HIGH)) - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); else if (type & (IRQ_TYPE_EDGE_FALLING | IRQ_TYPE_EDGE_RISING)) - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); unlock: spin_unlock_irqrestore(&chip->spinlock, flags); diff --git a/drivers/gpio/gpio-pxa.c b/drivers/gpio/gpio-pxa.c index cdbbcf0faf9da4..55a11de3d5b741 100644 --- a/drivers/gpio/gpio-pxa.c +++ b/drivers/gpio/gpio-pxa.c @@ -524,7 +524,7 @@ static int pxa_irq_domain_map(struct irq_domain *d, unsigned int irq, { irq_set_chip_and_handler(irq, &pxa_muxed_gpio_chip, handle_edge_irq); - set_irq_flags(irq, IRQF_VALID | IRQF_PROBE); + irq_set_noprobe(irq); return 0; } @@ -643,20 +643,20 @@ static int pxa_gpio_probe(struct platform_device *pdev) irq = gpio_to_irq(0); irq_set_chip_and_handler(irq, &pxa_muxed_gpio_chip, handle_edge_irq); - set_irq_flags(irq, IRQF_VALID | IRQF_PROBE); + irq_clear_status_flags(irq, IRQ_NOREQUEST | IRQ_NOPROBE); } if (irq1 > 0) { irq = gpio_to_irq(1); irq_set_chip_and_handler(irq, &pxa_muxed_gpio_chip, handle_edge_irq); - set_irq_flags(irq, IRQF_VALID | IRQF_PROBE); + irq_clear_status_flags(irq, IRQ_NOREQUEST | IRQ_NOPROBE); } for (irq = gpio_to_irq(gpio_offset); irq <= gpio_to_irq(pxa_last_gpio); irq++) { irq_set_chip_and_handler(irq, &pxa_muxed_gpio_chip, handle_edge_irq); - set_irq_flags(irq, IRQF_VALID | IRQF_PROBE); + irq_clear_status_flags(irq, IRQ_NOREQUEST | IRQ_NOPROBE); } } diff --git a/drivers/gpio/gpio-rcar.c b/drivers/gpio/gpio-rcar.c index 1e14a6c74ed139..2a81224446145d 100644 --- a/drivers/gpio/gpio-rcar.c +++ b/drivers/gpio/gpio-rcar.c @@ -251,17 +251,32 @@ static void gpio_rcar_config_general_input_output_mode(struct gpio_chip *chip, static int gpio_rcar_request(struct gpio_chip *chip, unsigned offset) { - return pinctrl_request_gpio(chip->base + offset); + struct gpio_rcar_priv *p = gpio_to_priv(chip); + int error; + + error = pm_runtime_get_sync(&p->pdev->dev); + if (error < 0) + return error; + + error = pinctrl_request_gpio(chip->base + offset); + if (error) + pm_runtime_put(&p->pdev->dev); + + return error; } static void gpio_rcar_free(struct gpio_chip *chip, unsigned offset) { + struct gpio_rcar_priv *p = gpio_to_priv(chip); + pinctrl_free_gpio(chip->base + offset); /* Set the GPIO as an input to ensure that the next GPIO request won't * drive the GPIO pin as an output. */ gpio_rcar_config_general_input_output_mode(chip, offset, false); + + pm_runtime_put(&p->pdev->dev); } static int gpio_rcar_direction_input(struct gpio_chip *chip, unsigned offset) @@ -326,6 +341,10 @@ static const struct of_device_id gpio_rcar_of_table[] = { }, { .compatible = "renesas,gpio-r8a7794", .data = &gpio_rcar_info_gen2, + }, { + .compatible = "renesas,gpio-r8a7795", + /* Gen3 GPIO is identical to Gen2. */ + .data = &gpio_rcar_info_gen2, }, { .compatible = "renesas,gpio-rcar", .data = &gpio_rcar_info_gen1, @@ -405,7 +424,6 @@ static int gpio_rcar_probe(struct platform_device *pdev) } pm_runtime_enable(dev); - pm_runtime_get_sync(dev); io = platform_get_resource(pdev, IORESOURCE_MEM, 0); irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0); @@ -487,7 +505,6 @@ static int gpio_rcar_probe(struct platform_device *pdev) err1: gpiochip_remove(gpio_chip); err0: - pm_runtime_put(dev); pm_runtime_disable(dev); return ret; } @@ -498,7 +515,6 @@ static int gpio_rcar_remove(struct platform_device *pdev) gpiochip_remove(&p->gpio_chip); - pm_runtime_put(&pdev->dev); pm_runtime_disable(&pdev->dev); return 0; } diff --git a/drivers/gpio/gpio-sa1100.c b/drivers/gpio/gpio-sa1100.c index 3fa22dade243e1..67bd2f5d89e8f3 100644 --- a/drivers/gpio/gpio-sa1100.c +++ b/drivers/gpio/gpio-sa1100.c @@ -155,7 +155,7 @@ static int sa1100_gpio_irqdomain_map(struct irq_domain *d, { irq_set_chip_and_handler(irq, &sa1100_gpio_irq_chip, handle_edge_irq); - set_irq_flags(irq, IRQF_VALID | IRQF_PROBE); + irq_set_noprobe(irq); return 0; } @@ -173,9 +173,9 @@ static struct irq_domain *sa1100_gpio_irqdomain; * and call the handler. */ static void -sa1100_gpio_handler(unsigned int irq, struct irq_desc *desc) +sa1100_gpio_handler(unsigned int __irq, struct irq_desc *desc) { - unsigned int mask; + unsigned int irq, mask; mask = GEDR; do { diff --git a/drivers/gpio/gpio-sta2x11.c b/drivers/gpio/gpio-sta2x11.c index 18579ac65b2b04..55e47828ddfcb6 100644 --- a/drivers/gpio/gpio-sta2x11.c +++ b/drivers/gpio/gpio-sta2x11.c @@ -346,7 +346,7 @@ static void gsta_alloc_irq_chip(struct gsta_gpio *chip) i = chip->irq_base + j; irq_set_chip_and_handler(i, &ct->chip, ct->handler); irq_set_chip_data(i, gc); - irq_modify_status(i, IRQ_NOREQUEST | IRQ_NOPROBE, 0); + irq_clear_status_flags(i, IRQ_NOREQUEST | IRQ_NOPROBE); } gc->irq_cnt = i - gc->irq_base; } diff --git a/drivers/gpio/gpio-tc3589x.c b/drivers/gpio/gpio-tc3589x.c index 31b244cffabb2f..d1d585ddb9ab78 100644 --- a/drivers/gpio/gpio-tc3589x.c +++ b/drivers/gpio/gpio-tc3589x.c @@ -102,7 +102,7 @@ static struct gpio_chip template_chip = { static int tc3589x_gpio_irq_set_type(struct irq_data *d, unsigned int type) { struct gpio_chip *gc = irq_data_get_irq_chip_data(d); - struct tc3589x_gpio *tc3589x_gpio = container_of(gc, struct tc3589x_gpio, chip); + struct tc3589x_gpio *tc3589x_gpio = to_tc3589x_gpio(gc); int offset = d->hwirq; int regoffset = offset / 8; int mask = 1 << (offset % 8); @@ -130,7 +130,7 @@ static int tc3589x_gpio_irq_set_type(struct irq_data *d, unsigned int type) static void tc3589x_gpio_irq_lock(struct irq_data *d) { struct gpio_chip *gc = irq_data_get_irq_chip_data(d); - struct tc3589x_gpio *tc3589x_gpio = container_of(gc, struct tc3589x_gpio, chip); + struct tc3589x_gpio *tc3589x_gpio = to_tc3589x_gpio(gc); mutex_lock(&tc3589x_gpio->irq_lock); } @@ -138,7 +138,7 @@ static void tc3589x_gpio_irq_lock(struct irq_data *d) static void tc3589x_gpio_irq_sync_unlock(struct irq_data *d) { struct gpio_chip *gc = irq_data_get_irq_chip_data(d); - struct tc3589x_gpio *tc3589x_gpio = container_of(gc, struct tc3589x_gpio, chip); + struct tc3589x_gpio *tc3589x_gpio = to_tc3589x_gpio(gc); struct tc3589x *tc3589x = tc3589x_gpio->tc3589x; static const u8 regmap[] = { [REG_IBE] = TC3589x_GPIOIBE0, @@ -167,7 +167,7 @@ static void tc3589x_gpio_irq_sync_unlock(struct irq_data *d) static void tc3589x_gpio_irq_mask(struct irq_data *d) { struct gpio_chip *gc = irq_data_get_irq_chip_data(d); - struct tc3589x_gpio *tc3589x_gpio = container_of(gc, struct tc3589x_gpio, chip); + struct tc3589x_gpio *tc3589x_gpio = to_tc3589x_gpio(gc); int offset = d->hwirq; int regoffset = offset / 8; int mask = 1 << (offset % 8); @@ -178,7 +178,7 @@ static void tc3589x_gpio_irq_mask(struct irq_data *d) static void tc3589x_gpio_irq_unmask(struct irq_data *d) { struct gpio_chip *gc = irq_data_get_irq_chip_data(d); - struct tc3589x_gpio *tc3589x_gpio = container_of(gc, struct tc3589x_gpio, chip); + struct tc3589x_gpio *tc3589x_gpio = to_tc3589x_gpio(gc); int offset = d->hwirq; int regoffset = offset / 8; int mask = 1 << (offset % 8); diff --git a/drivers/gpio/gpio-tegra.c b/drivers/gpio/gpio-tegra.c index 9b25c90f725c21..9b14aafb576da8 100644 --- a/drivers/gpio/gpio-tegra.c +++ b/drivers/gpio/gpio-tegra.c @@ -252,9 +252,9 @@ static int tegra_gpio_irq_set_type(struct irq_data *d, unsigned int type) tegra_gpio_enable(gpio); if (type & (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_LEVEL_HIGH)) - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); else if (type & (IRQ_TYPE_EDGE_FALLING | IRQ_TYPE_EDGE_RISING)) - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); return 0; } @@ -268,16 +268,14 @@ static void tegra_gpio_irq_shutdown(struct irq_data *d) static void tegra_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) { - struct tegra_gpio_bank *bank; int port; int pin; int unmasked = 0; struct irq_chip *chip = irq_desc_get_chip(desc); + struct tegra_gpio_bank *bank = irq_desc_get_handler_data(desc); chained_irq_enter(chip, desc); - bank = irq_get_handler_data(irq); - for (port = 0; port < 4; port++) { int gpio = tegra_gpio_compose(bank->bank, port, 0); unsigned long sta = tegra_gpio_readl(GPIO_INT_STA(gpio)) & @@ -509,7 +507,6 @@ static int tegra_gpio_probe(struct platform_device *pdev) irq_set_chip_data(irq, bank); irq_set_chip_and_handler(irq, &tegra_gpio_irq_chip, handle_simple_irq); - set_irq_flags(irq, IRQF_VALID); } for (i = 0; i < tegra_gpio_bank_count; i++) { diff --git a/drivers/gpio/gpio-timberdale.c b/drivers/gpio/gpio-timberdale.c index e8f97e03c9bb6a..5a492054589fa4 100644 --- a/drivers/gpio/gpio-timberdale.c +++ b/drivers/gpio/gpio-timberdale.c @@ -194,11 +194,12 @@ static int timbgpio_irq_type(struct irq_data *d, unsigned trigger) static void timbgpio_irq(unsigned int irq, struct irq_desc *desc) { - struct timbgpio *tgpio = irq_get_handler_data(irq); + struct timbgpio *tgpio = irq_desc_get_handler_data(desc); + struct irq_data *data = irq_desc_get_irq_data(desc); unsigned long ipr; int offset; - desc->irq_data.chip->irq_ack(irq_get_irq_data(irq)); + data->chip->irq_ack(data); ipr = ioread32(tgpio->membase + TGPIO_IPR); iowrite32(ipr, tgpio->membase + TGPIO_ICR); @@ -294,13 +295,10 @@ static int timbgpio_probe(struct platform_device *pdev) irq_set_chip_and_handler(tgpio->irq_base + i, &timbgpio_irqchip, handle_simple_irq); irq_set_chip_data(tgpio->irq_base + i, tgpio); -#ifdef CONFIG_ARM - set_irq_flags(tgpio->irq_base + i, IRQF_VALID | IRQF_PROBE); -#endif + irq_clear_status_flags(tgpio->irq_base + i, IRQ_NOREQUEST | IRQ_NOPROBE); } - irq_set_handler_data(irq, tgpio); - irq_set_chained_handler(irq, timbgpio_irq); + irq_set_chained_handler_and_data(irq, timbgpio_irq, tgpio); return 0; } diff --git a/drivers/gpio/gpio-tz1090.c b/drivers/gpio/gpio-tz1090.c index 445660adc89891..bbac92ae4c328a 100644 --- a/drivers/gpio/gpio-tz1090.c +++ b/drivers/gpio/gpio-tz1090.c @@ -510,8 +510,8 @@ static int tz1090_gpio_bank_probe(struct tz1090_gpio_bank_info *info) gc->chip_types[1].chip.flags = IRQCHIP_MASK_ON_SUSPEND; /* Setup chained handler for this GPIO bank */ - irq_set_handler_data(bank->irq, bank); - irq_set_chained_handler(bank->irq, tz1090_gpio_irq_handler); + irq_set_chained_handler_and_data(bank->irq, tz1090_gpio_irq_handler, + bank); return 0; } diff --git a/drivers/gpio/gpio-vf610.c b/drivers/gpio/gpio-vf610.c index 7bd9f209ffa81a..3d5714d4f405dd 100644 --- a/drivers/gpio/gpio-vf610.c +++ b/drivers/gpio/gpio-vf610.c @@ -60,6 +60,8 @@ struct vf610_gpio_port { #define PORT_INT_EITHER_EDGE 0xb #define PORT_INT_LOGIC_ONE 0xc +static struct irq_chip vf610_gpio_irq_chip; + static const struct of_device_id vf610_gpio_dt_ids[] = { { .compatible = "fsl,vf610-gpio" }, { /* sentinel */ } @@ -120,7 +122,7 @@ static int vf610_gpio_direction_output(struct gpio_chip *chip, unsigned gpio, static void vf610_gpio_irq_handler(u32 irq, struct irq_desc *desc) { - struct vf610_gpio_port *port = irq_get_handler_data(irq); + struct vf610_gpio_port *port = irq_desc_get_handler_data(desc); struct irq_chip *chip = irq_desc_get_chip(desc); int pin; unsigned long irq_isfr; @@ -173,6 +175,11 @@ static int vf610_gpio_irq_set_type(struct irq_data *d, u32 type) port->irqc[d->hwirq] = irqc; + if (type & IRQ_TYPE_LEVEL_MASK) + __irq_set_handler_locked(d->irq, handle_level_irq); + else + __irq_set_handler_locked(d->irq, handle_edge_irq); + return 0; } @@ -263,7 +270,7 @@ static int vf610_gpio_probe(struct platform_device *pdev) vf610_gpio_writel(~0, port->base + PORT_ISFR); ret = gpiochip_irqchip_add(gc, &vf610_gpio_irq_chip, 0, - handle_simple_irq, IRQ_TYPE_NONE); + handle_edge_irq, IRQ_TYPE_NONE); if (ret) { dev_err(dev, "failed to add irqchip\n"); gpiochip_remove(gc); diff --git a/drivers/gpio/gpio-xlp.c b/drivers/gpio/gpio-xlp.c index 9bdab7203d6515..e02499a15e7226 100644 --- a/drivers/gpio/gpio-xlp.c +++ b/drivers/gpio/gpio-xlp.c @@ -387,7 +387,7 @@ static int xlp_gpio_probe(struct platform_device *pdev) irq_base = irq_alloc_descs(-1, XLP_GPIO_IRQ_BASE, gc->ngpio, 0); if (irq_base < 0) { dev_err(&pdev->dev, "Failed to allocate IRQ numbers\n"); - return err; + return -ENODEV; } err = gpiochip_add(gc); diff --git a/drivers/gpio/gpio-zx.c b/drivers/gpio/gpio-zx.c new file mode 100644 index 00000000000000..12ee1969298cc1 --- /dev/null +++ b/drivers/gpio/gpio-zx.c @@ -0,0 +1,324 @@ +/* + * Copyright (C) 2015 Linaro Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define ZX_GPIO_DIR 0x00 +#define ZX_GPIO_IVE 0x04 +#define ZX_GPIO_IV 0x08 +#define ZX_GPIO_IEP 0x0C +#define ZX_GPIO_IEN 0x10 +#define ZX_GPIO_DI 0x14 +#define ZX_GPIO_DO1 0x18 +#define ZX_GPIO_DO0 0x1C +#define ZX_GPIO_DO 0x20 + +#define ZX_GPIO_IM 0x28 +#define ZX_GPIO_IE 0x2C + +#define ZX_GPIO_MIS 0x30 +#define ZX_GPIO_IC 0x34 + +#define ZX_GPIO_NR 16 + +struct zx_gpio { + spinlock_t lock; + + void __iomem *base; + struct gpio_chip gc; + bool uses_pinctrl; +}; + +static inline struct zx_gpio *to_zx(struct gpio_chip *gc) +{ + return container_of(gc, struct zx_gpio, gc); +} + +static int zx_gpio_request(struct gpio_chip *gc, unsigned offset) +{ + struct zx_gpio *chip = to_zx(gc); + int gpio = gc->base + offset; + + if (chip->uses_pinctrl) + return pinctrl_request_gpio(gpio); + return 0; +} + +static void zx_gpio_free(struct gpio_chip *gc, unsigned offset) +{ + struct zx_gpio *chip = to_zx(gc); + int gpio = gc->base + offset; + + if (chip->uses_pinctrl) + pinctrl_free_gpio(gpio); +} + +static int zx_direction_input(struct gpio_chip *gc, unsigned offset) +{ + struct zx_gpio *chip = to_zx(gc); + unsigned long flags; + u16 gpiodir; + + if (offset >= gc->ngpio) + return -EINVAL; + + spin_lock_irqsave(&chip->lock, flags); + gpiodir = readw_relaxed(chip->base + ZX_GPIO_DIR); + gpiodir &= ~BIT(offset); + writew_relaxed(gpiodir, chip->base + ZX_GPIO_DIR); + spin_unlock_irqrestore(&chip->lock, flags); + + return 0; +} + +static int zx_direction_output(struct gpio_chip *gc, unsigned offset, + int value) +{ + struct zx_gpio *chip = to_zx(gc); + unsigned long flags; + u16 gpiodir; + + if (offset >= gc->ngpio) + return -EINVAL; + + spin_lock_irqsave(&chip->lock, flags); + gpiodir = readw_relaxed(chip->base + ZX_GPIO_DIR); + gpiodir |= BIT(offset); + writew_relaxed(gpiodir, chip->base + ZX_GPIO_DIR); + + if (value) + writew_relaxed(BIT(offset), chip->base + ZX_GPIO_DO1); + else + writew_relaxed(BIT(offset), chip->base + ZX_GPIO_DO0); + spin_unlock_irqrestore(&chip->lock, flags); + + return 0; +} + +static int zx_get_value(struct gpio_chip *gc, unsigned offset) +{ + struct zx_gpio *chip = to_zx(gc); + + return !!(readw_relaxed(chip->base + ZX_GPIO_DI) & BIT(offset)); +} + +static void zx_set_value(struct gpio_chip *gc, unsigned offset, int value) +{ + struct zx_gpio *chip = to_zx(gc); + + if (value) + writew_relaxed(BIT(offset), chip->base + ZX_GPIO_DO1); + else + writew_relaxed(BIT(offset), chip->base + ZX_GPIO_DO0); +} + +static int zx_irq_type(struct irq_data *d, unsigned trigger) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct zx_gpio *chip = to_zx(gc); + int offset = irqd_to_hwirq(d); + unsigned long flags; + u16 gpiois, gpioi_epos, gpioi_eneg, gpioiev; + u16 bit = BIT(offset); + + if (offset < 0 || offset >= ZX_GPIO_NR) + return -EINVAL; + + spin_lock_irqsave(&chip->lock, flags); + + gpioiev = readw_relaxed(chip->base + ZX_GPIO_IV); + gpiois = readw_relaxed(chip->base + ZX_GPIO_IVE); + gpioi_epos = readw_relaxed(chip->base + ZX_GPIO_IEP); + gpioi_eneg = readw_relaxed(chip->base + ZX_GPIO_IEN); + + if (trigger & (IRQ_TYPE_LEVEL_HIGH | IRQ_TYPE_LEVEL_LOW)) { + gpiois |= bit; + if (trigger & IRQ_TYPE_LEVEL_HIGH) + gpioiev |= bit; + else + gpioiev &= ~bit; + } else + gpiois &= ~bit; + + if ((trigger & IRQ_TYPE_EDGE_BOTH) == IRQ_TYPE_EDGE_BOTH) { + gpioi_epos |= bit; + gpioi_eneg |= bit; + } else { + if (trigger & IRQ_TYPE_EDGE_RISING) { + gpioi_epos |= bit; + gpioi_eneg &= ~bit; + } else if (trigger & IRQ_TYPE_EDGE_FALLING) { + gpioi_eneg |= bit; + gpioi_epos &= ~bit; + } + } + + writew_relaxed(gpiois, chip->base + ZX_GPIO_IVE); + writew_relaxed(gpioi_epos, chip->base + ZX_GPIO_IEP); + writew_relaxed(gpioi_eneg, chip->base + ZX_GPIO_IEN); + writew_relaxed(gpioiev, chip->base + ZX_GPIO_IV); + spin_unlock_irqrestore(&chip->lock, flags); + + return 0; +} + +static void zx_irq_handler(unsigned irq, struct irq_desc *desc) +{ + unsigned long pending; + int offset; + struct gpio_chip *gc = irq_desc_get_handler_data(desc); + struct zx_gpio *chip = to_zx(gc); + struct irq_chip *irqchip = irq_desc_get_chip(desc); + + chained_irq_enter(irqchip, desc); + + pending = readw_relaxed(chip->base + ZX_GPIO_MIS); + writew_relaxed(pending, chip->base + ZX_GPIO_IC); + if (pending) { + for_each_set_bit(offset, &pending, ZX_GPIO_NR) + generic_handle_irq(irq_find_mapping(gc->irqdomain, + offset)); + } + + chained_irq_exit(irqchip, desc); +} + +static void zx_irq_mask(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct zx_gpio *chip = to_zx(gc); + u16 mask = BIT(irqd_to_hwirq(d) % ZX_GPIO_NR); + u16 gpioie; + + spin_lock(&chip->lock); + gpioie = readw_relaxed(chip->base + ZX_GPIO_IM) | mask; + writew_relaxed(gpioie, chip->base + ZX_GPIO_IM); + gpioie = readw_relaxed(chip->base + ZX_GPIO_IE) & ~mask; + writew_relaxed(gpioie, chip->base + ZX_GPIO_IE); + spin_unlock(&chip->lock); +} + +static void zx_irq_unmask(struct irq_data *d) +{ + struct gpio_chip *gc = irq_data_get_irq_chip_data(d); + struct zx_gpio *chip = to_zx(gc); + u16 mask = BIT(irqd_to_hwirq(d) % ZX_GPIO_NR); + u16 gpioie; + + spin_lock(&chip->lock); + gpioie = readw_relaxed(chip->base + ZX_GPIO_IM) & ~mask; + writew_relaxed(gpioie, chip->base + ZX_GPIO_IM); + gpioie = readw_relaxed(chip->base + ZX_GPIO_IE) | mask; + writew_relaxed(gpioie, chip->base + ZX_GPIO_IE); + spin_unlock(&chip->lock); +} + +static struct irq_chip zx_irqchip = { + .name = "zx-gpio", + .irq_mask = zx_irq_mask, + .irq_unmask = zx_irq_unmask, + .irq_set_type = zx_irq_type, +}; + +static int zx_gpio_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct zx_gpio *chip; + struct resource *res; + int irq, id, ret; + + chip = devm_kzalloc(dev, sizeof(*chip), GFP_KERNEL); + if (!chip) + return -ENOMEM; + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + chip->base = devm_ioremap_resource(dev, res); + if (IS_ERR(chip->base)) + return PTR_ERR(chip->base); + + spin_lock_init(&chip->lock); + if (of_property_read_bool(dev->of_node, "gpio-ranges")) + chip->uses_pinctrl = true; + + id = of_alias_get_id(dev->of_node, "gpio"); + chip->gc.request = zx_gpio_request; + chip->gc.free = zx_gpio_free; + chip->gc.direction_input = zx_direction_input; + chip->gc.direction_output = zx_direction_output; + chip->gc.get = zx_get_value; + chip->gc.set = zx_set_value; + chip->gc.base = ZX_GPIO_NR * id; + chip->gc.ngpio = ZX_GPIO_NR; + chip->gc.label = dev_name(dev); + chip->gc.dev = dev; + chip->gc.owner = THIS_MODULE; + + ret = gpiochip_add(&chip->gc); + if (ret) + return ret; + + /* + * irq_chip support + */ + writew_relaxed(0xffff, chip->base + ZX_GPIO_IM); + writew_relaxed(0, chip->base + ZX_GPIO_IE); + irq = platform_get_irq(pdev, 0); + if (irq < 0) { + dev_err(dev, "invalid IRQ\n"); + gpiochip_remove(&chip->gc); + return -ENODEV; + } + + ret = gpiochip_irqchip_add(&chip->gc, &zx_irqchip, + 0, handle_simple_irq, + IRQ_TYPE_NONE); + if (ret) { + dev_err(dev, "could not add irqchip\n"); + gpiochip_remove(&chip->gc); + return ret; + } + gpiochip_set_chained_irqchip(&chip->gc, &zx_irqchip, + irq, zx_irq_handler); + + platform_set_drvdata(pdev, chip); + dev_info(dev, "ZX GPIO chip registered\n"); + + return 0; +} + +static const struct of_device_id zx_gpio_match[] = { + { + .compatible = "zte,zx296702-gpio", + }, + { }, +}; +MODULE_DEVICE_TABLE(of, zx_gpio_match); + +static struct platform_driver zx_gpio_driver = { + .probe = zx_gpio_probe, + .driver = { + .name = "zx_gpio", + .of_match_table = of_match_ptr(zx_gpio_match), + }, +}; + +module_platform_driver(zx_gpio_driver) + +MODULE_AUTHOR("Jun Nie "); +MODULE_DESCRIPTION("ZTE ZX296702 GPIO driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/gpio/gpio-zynq.c b/drivers/gpio/gpio-zynq.c index a7888238983607..27348e7cb70590 100644 --- a/drivers/gpio/gpio-zynq.c +++ b/drivers/gpio/gpio-zynq.c @@ -441,10 +441,10 @@ static int zynq_gpio_set_irq_type(struct irq_data *irq_data, unsigned int type) gpio->base_addr + ZYNQ_GPIO_INTANY_OFFSET(bank_num)); if (type & IRQ_TYPE_LEVEL_MASK) { - __irq_set_chip_handler_name_locked(irq_data->irq, + irq_set_chip_handler_name_locked(irq_data, &zynq_gpio_level_irqchip, handle_fasteoi_irq, NULL); } else { - __irq_set_chip_handler_name_locked(irq_data->irq, + irq_set_chip_handler_name_locked(irq_data, &zynq_gpio_edge_irqchip, handle_level_irq, NULL); } @@ -518,7 +518,7 @@ static void zynq_gpio_irqhandler(unsigned int irq, struct irq_desc *desc) { u32 int_sts, int_enb; unsigned int bank_num; - struct zynq_gpio *gpio = irq_get_handler_data(irq); + struct zynq_gpio *gpio = irq_desc_get_handler_data(desc); struct irq_chip *irqchip = irq_desc_get_chip(desc); chained_irq_enter(irqchip, desc); @@ -782,6 +782,12 @@ static int __init zynq_gpio_init(void) } postcore_initcall(zynq_gpio_init); +static void __exit zynq_gpio_exit(void) +{ + platform_driver_unregister(&zynq_gpio_driver); +} +module_exit(zynq_gpio_exit); + MODULE_AUTHOR("Xilinx Inc."); MODULE_DESCRIPTION("Zynq GPIO driver"); MODULE_LICENSE("GPL"); diff --git a/drivers/gpio/gpiolib-of.c b/drivers/gpio/gpiolib-of.c index 9a0ec48a47375d..fa6e3c8823d614 100644 --- a/drivers/gpio/gpiolib-of.c +++ b/drivers/gpio/gpiolib-of.c @@ -136,7 +136,6 @@ static struct gpio_desc *of_get_gpio_hog(struct device_node *np, { struct device_node *chip_np; enum of_gpio_flags xlate_flags; - struct gpio_desc *desc; struct gg_data gg_data = { .flags = &xlate_flags, }; @@ -193,9 +192,7 @@ static struct gpio_desc *of_get_gpio_hog(struct device_node *np, if (name && of_property_read_string(np, "line-name", name)) *name = np->name; - desc = gg_data.out_gpio; - - return desc; + return gg_data.out_gpio; } /** @@ -338,7 +335,7 @@ void of_mm_gpiochip_remove(struct of_mm_gpio_chip *mm_gc) EXPORT_SYMBOL(of_mm_gpiochip_remove); #ifdef CONFIG_PINCTRL -static void of_gpiochip_add_pin_range(struct gpio_chip *chip) +static int of_gpiochip_add_pin_range(struct gpio_chip *chip) { struct device_node *np = chip->of_node; struct of_phandle_args pinspec; @@ -349,7 +346,7 @@ static void of_gpiochip_add_pin_range(struct gpio_chip *chip) struct property *group_names; if (!np) - return; + return 0; group_names = of_find_property(np, group_names_propname, NULL); @@ -361,11 +358,11 @@ static void of_gpiochip_add_pin_range(struct gpio_chip *chip) pctldev = of_pinctrl_get(pinspec.np); if (!pctldev) - break; + return -EPROBE_DEFER; if (pinspec.args[2]) { if (group_names) { - ret = of_property_read_string_index(np, + of_property_read_string_index(np, group_names_propname, index, &name); if (strlen(name)) { @@ -381,7 +378,7 @@ static void of_gpiochip_add_pin_range(struct gpio_chip *chip) pinspec.args[1], pinspec.args[2]); if (ret) - break; + return ret; } else { /* npins == 0: special range */ if (pinspec.args[1]) { @@ -411,32 +408,41 @@ static void of_gpiochip_add_pin_range(struct gpio_chip *chip) ret = gpiochip_add_pingroup_range(chip, pctldev, pinspec.args[0], name); if (ret) - break; + return ret; } } + + return 0; } #else -static void of_gpiochip_add_pin_range(struct gpio_chip *chip) {} +static int of_gpiochip_add_pin_range(struct gpio_chip *chip) { return 0; } #endif -void of_gpiochip_add(struct gpio_chip *chip) +int of_gpiochip_add(struct gpio_chip *chip) { + int status; + if ((!chip->of_node) && (chip->dev)) chip->of_node = chip->dev->of_node; if (!chip->of_node) - return; + return 0; if (!chip->of_xlate) { chip->of_gpio_n_cells = 2; chip->of_xlate = of_gpio_simple_xlate; } - of_gpiochip_add_pin_range(chip); + status = of_gpiochip_add_pin_range(chip); + if (status) + return status; + of_node_get(chip->of_node); of_gpiochip_scan_hogs(chip); + + return 0; } void of_gpiochip_remove(struct gpio_chip *chip) diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c index bf4bd1d120c38a..b562dd36c4af19 100644 --- a/drivers/gpio/gpiolib.c +++ b/drivers/gpio/gpiolib.c @@ -190,7 +190,7 @@ EXPORT_SYMBOL_GPL(gpiod_get_direction); */ static int gpiochip_add_to_list(struct gpio_chip *chip) { - struct list_head *pos = &gpio_chips; + struct list_head *pos; struct gpio_chip *_chip; int err = 0; @@ -287,7 +287,13 @@ int gpiochip_add(struct gpio_chip *chip) INIT_LIST_HEAD(&chip->pin_ranges); #endif - of_gpiochip_add(chip); + if (!chip->owner && chip->dev && chip->dev->driver) + chip->owner = chip->dev->driver->owner; + + status = of_gpiochip_add(chip); + if (status) + goto err_remove_chip; + acpi_gpiochip_add(chip); status = gpiochip_sysfs_register(chip); @@ -443,8 +449,8 @@ void gpiochip_set_chained_irqchip(struct gpio_chip *gpiochip, * The parent irqchip is already using the chip_data for this * irqchip, so our callbacks simply use the handler_data. */ - irq_set_handler_data(parent_irq, gpiochip); - irq_set_chained_handler(parent_irq, parent_handler); + irq_set_chained_handler_and_data(parent_irq, parent_handler, + gpiochip); gpiochip->irq_parent = parent_irq; } @@ -456,12 +462,6 @@ void gpiochip_set_chained_irqchip(struct gpio_chip *gpiochip, } EXPORT_SYMBOL_GPL(gpiochip_set_chained_irqchip); -/* - * This lock class tells lockdep that GPIO irqs are in a different - * category than their parents, so it won't report false recursion. - */ -static struct lock_class_key gpiochip_irq_lock_class; - /** * gpiochip_irq_map() - maps an IRQ into a GPIO irqchip * @d: the irqdomain used by this irqchip @@ -478,16 +478,17 @@ static int gpiochip_irq_map(struct irq_domain *d, unsigned int irq, struct gpio_chip *chip = d->host_data; irq_set_chip_data(irq, chip); - irq_set_lockdep_class(irq, &gpiochip_irq_lock_class); + /* + * This lock class tells lockdep that GPIO irqs are in a different + * category than their parents, so it won't report false recursion. + */ + irq_set_lockdep_class(irq, chip->lock_key); irq_set_chip_and_handler(irq, chip->irqchip, chip->irq_handler); /* Chips that can sleep need nested thread handlers */ if (chip->can_sleep && !chip->irq_not_threaded) irq_set_nested_thread(irq, 1); -#ifdef CONFIG_ARM - set_irq_flags(irq, IRQF_VALID); -#else irq_set_noprobe(irq); -#endif + /* * No set-up of the hardware will happen if IRQ_TYPE_NONE * is passed as default type. @@ -502,9 +503,6 @@ static void gpiochip_irq_unmap(struct irq_domain *d, unsigned int irq) { struct gpio_chip *chip = d->host_data; -#ifdef CONFIG_ARM - set_irq_flags(irq, 0); -#endif if (chip->can_sleep) irq_set_nested_thread(irq, 0); irq_set_chip_and_handler(irq, NULL, NULL); @@ -522,10 +520,14 @@ static int gpiochip_irq_reqres(struct irq_data *d) { struct gpio_chip *chip = irq_data_get_irq_chip_data(d); + if (!try_module_get(chip->owner)) + return -ENODEV; + if (gpiochip_lock_as_irq(chip, d->hwirq)) { chip_err(chip, "unable to lock HW IRQ %lu for IRQ\n", d->hwirq); + module_put(chip->owner); return -EINVAL; } return 0; @@ -536,6 +538,7 @@ static void gpiochip_irq_relres(struct irq_data *d) struct gpio_chip *chip = irq_data_get_irq_chip_data(d); gpiochip_unlock_as_irq(chip, d->hwirq); + module_put(chip->owner); } static int gpiochip_to_irq(struct gpio_chip *chip, unsigned offset) @@ -584,6 +587,7 @@ static void gpiochip_irqchip_remove(struct gpio_chip *gpiochip) * @handler: the irq handler to use (often a predefined irq core function) * @type: the default type for IRQs on this irqchip, pass IRQ_TYPE_NONE * to have the core avoid setting up any default type in the hardware. + * @lock_key: lockdep class * * This function closely associates a certain irqchip with a certain * gpiochip, providing an irq domain to translate the local IRQs to @@ -599,11 +603,12 @@ static void gpiochip_irqchip_remove(struct gpio_chip *gpiochip) * the pins on the gpiochip can generate a unique IRQ. Everything else * need to be open coded. */ -int gpiochip_irqchip_add(struct gpio_chip *gpiochip, - struct irq_chip *irqchip, - unsigned int first_irq, - irq_flow_handler_t handler, - unsigned int type) +int _gpiochip_irqchip_add(struct gpio_chip *gpiochip, + struct irq_chip *irqchip, + unsigned int first_irq, + irq_flow_handler_t handler, + unsigned int type, + struct lock_class_key *lock_key) { struct device_node *of_node; unsigned int offset; @@ -629,6 +634,7 @@ int gpiochip_irqchip_add(struct gpio_chip *gpiochip, gpiochip->irq_handler = handler; gpiochip->irq_default_type = type; gpiochip->to_irq = gpiochip_to_irq; + gpiochip->lock_key = lock_key; gpiochip->irqdomain = irq_domain_add_simple(of_node, gpiochip->ngpio, first_irq, &gpiochip_domain_ops, gpiochip); @@ -636,8 +642,16 @@ int gpiochip_irqchip_add(struct gpio_chip *gpiochip, gpiochip->irqchip = NULL; return -EINVAL; } - irqchip->irq_request_resources = gpiochip_irq_reqres; - irqchip->irq_release_resources = gpiochip_irq_relres; + + /* + * It is possible for a driver to override this, but only if the + * alternative functions are both implemented. + */ + if (!irqchip->irq_request_resources && + !irqchip->irq_release_resources) { + irqchip->irq_request_resources = gpiochip_irq_reqres; + irqchip->irq_release_resources = gpiochip_irq_relres; + } /* * Prepare the mapping since the irqchip shall be orthogonal to @@ -658,7 +672,7 @@ int gpiochip_irqchip_add(struct gpio_chip *gpiochip, return 0; } -EXPORT_SYMBOL_GPL(gpiochip_irqchip_add); +EXPORT_SYMBOL_GPL(_gpiochip_irqchip_add); #else /* CONFIG_GPIOLIB_IRQCHIP */ @@ -671,7 +685,7 @@ static void gpiochip_irqchip_remove(struct gpio_chip *gpiochip) {} /** * gpiochip_add_pingroup_range() - add a range for GPIO <-> pin mapping * @chip: the gpiochip to add the range for - * @pinctrl: the dev_name() of the pin controller to map to + * @pctldev: the pin controller to map to * @gpio_offset: the start offset in the current gpio_chip number space * @pin_group: name of the pin group inside the pin controller */ @@ -1894,12 +1908,12 @@ EXPORT_SYMBOL_GPL(gpiod_count); * dev, -ENOENT if no GPIO has been assigned to the requested function, or * another IS_ERR() code if an error occurred while trying to acquire the GPIO. */ -struct gpio_desc *__must_check __gpiod_get(struct device *dev, const char *con_id, +struct gpio_desc *__must_check gpiod_get(struct device *dev, const char *con_id, enum gpiod_flags flags) { return gpiod_get_index(dev, con_id, 0, flags); } -EXPORT_SYMBOL_GPL(__gpiod_get); +EXPORT_SYMBOL_GPL(gpiod_get); /** * gpiod_get_optional - obtain an optional GPIO for a given GPIO function @@ -1911,13 +1925,13 @@ EXPORT_SYMBOL_GPL(__gpiod_get); * the requested function it will return NULL. This is convenient for drivers * that need to handle optional GPIOs. */ -struct gpio_desc *__must_check __gpiod_get_optional(struct device *dev, +struct gpio_desc *__must_check gpiod_get_optional(struct device *dev, const char *con_id, enum gpiod_flags flags) { return gpiod_get_index_optional(dev, con_id, 0, flags); } -EXPORT_SYMBOL_GPL(__gpiod_get_optional); +EXPORT_SYMBOL_GPL(gpiod_get_optional); /** @@ -1974,7 +1988,7 @@ static int gpiod_configure_flags(struct gpio_desc *desc, const char *con_id, * requested function and/or index, or another IS_ERR() code if an error * occurred while trying to acquire the GPIO. */ -struct gpio_desc *__must_check __gpiod_get_index(struct device *dev, +struct gpio_desc *__must_check gpiod_get_index(struct device *dev, const char *con_id, unsigned int idx, enum gpiod_flags flags) @@ -2023,7 +2037,7 @@ struct gpio_desc *__must_check __gpiod_get_index(struct device *dev, return desc; } -EXPORT_SYMBOL_GPL(__gpiod_get_index); +EXPORT_SYMBOL_GPL(gpiod_get_index); /** * fwnode_get_named_gpiod - obtain a GPIO from firmware node @@ -2092,7 +2106,7 @@ EXPORT_SYMBOL_GPL(fwnode_get_named_gpiod); * specified index was assigned to the requested function it will return NULL. * This is convenient for drivers that need to handle optional GPIOs. */ -struct gpio_desc *__must_check __gpiod_get_index_optional(struct device *dev, +struct gpio_desc *__must_check gpiod_get_index_optional(struct device *dev, const char *con_id, unsigned int index, enum gpiod_flags flags) @@ -2107,7 +2121,7 @@ struct gpio_desc *__must_check __gpiod_get_index_optional(struct device *dev, return desc; } -EXPORT_SYMBOL_GPL(__gpiod_get_index_optional); +EXPORT_SYMBOL_GPL(gpiod_get_index_optional); /** * gpiod_hog - Hog the specified GPIO desc given the provided flags diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index 884b4f9b81c4ab..8917c98ff121c6 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -748,7 +748,7 @@ static int i915_drm_resume(struct drm_device *dev) mutex_lock(&dev->struct_mutex); if (i915_gem_init_hw(dev)) { DRM_ERROR("failed to re-initialize GPU, declaring wedged!\n"); - atomic_set_mask(I915_WEDGED, &dev_priv->gpu_error.reset_counter); + atomic_or(I915_WEDGED, &dev_priv->gpu_error.reset_counter); } mutex_unlock(&dev->struct_mutex); diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 52b446b27b4d08..7a918d1c12ba3f 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -5091,7 +5091,7 @@ int i915_gem_init(struct drm_device *dev) * for all other failure, such as an allocation failure, bail. */ DRM_ERROR("Failed to initialize GPU, declaring it wedged\n"); - atomic_set_mask(I915_WEDGED, &dev_priv->gpu_error.reset_counter); + atomic_or(I915_WEDGED, &dev_priv->gpu_error.reset_counter); ret = 0; } diff --git a/drivers/gpu/drm/i915/i915_irq.c b/drivers/gpu/drm/i915/i915_irq.c index 984e2fe6688c4c..449a95c6c2a1d7 100644 --- a/drivers/gpu/drm/i915/i915_irq.c +++ b/drivers/gpu/drm/i915/i915_irq.c @@ -2446,7 +2446,7 @@ static void i915_reset_and_wakeup(struct drm_device *dev) kobject_uevent_env(&dev->primary->kdev->kobj, KOBJ_CHANGE, reset_done_event); } else { - atomic_set_mask(I915_WEDGED, &error->reset_counter); + atomic_or(I915_WEDGED, &error->reset_counter); } /* @@ -2574,7 +2574,7 @@ void i915_handle_error(struct drm_device *dev, bool wedged, i915_report_and_clear_eir(dev); if (wedged) { - atomic_set_mask(I915_RESET_IN_PROGRESS_FLAG, + atomic_or(I915_RESET_IN_PROGRESS_FLAG, &dev_priv->gpu_error.reset_counter); /* diff --git a/drivers/gpu/drm/msm/edp/edp_ctrl.c b/drivers/gpu/drm/msm/edp/edp_ctrl.c index 7991069dd492ad..81200e9be3821c 100644 --- a/drivers/gpu/drm/msm/edp/edp_ctrl.c +++ b/drivers/gpu/drm/msm/edp/edp_ctrl.c @@ -373,7 +373,7 @@ static int edp_gpio_config(struct edp_ctrl *ctrl) struct device *dev = &ctrl->pdev->dev; int ret; - ctrl->panel_hpd_gpio = devm_gpiod_get(dev, "panel-hpd"); + ctrl->panel_hpd_gpio = devm_gpiod_get(dev, "panel-hpd", GPIOD_IN); if (IS_ERR(ctrl->panel_hpd_gpio)) { ret = PTR_ERR(ctrl->panel_hpd_gpio); ctrl->panel_hpd_gpio = NULL; @@ -381,13 +381,7 @@ static int edp_gpio_config(struct edp_ctrl *ctrl) return ret; } - ret = gpiod_direction_input(ctrl->panel_hpd_gpio); - if (ret) { - pr_err("%s: Set direction for hpd failed, %d\n", __func__, ret); - return ret; - } - - ctrl->panel_en_gpio = devm_gpiod_get(dev, "panel-en"); + ctrl->panel_en_gpio = devm_gpiod_get(dev, "panel-en", GPIOD_OUT_LOW); if (IS_ERR(ctrl->panel_en_gpio)) { ret = PTR_ERR(ctrl->panel_en_gpio); ctrl->panel_en_gpio = NULL; @@ -395,13 +389,6 @@ static int edp_gpio_config(struct edp_ctrl *ctrl) return ret; } - ret = gpiod_direction_output(ctrl->panel_en_gpio, 0); - if (ret) { - pr_err("%s: Set direction for panel_en failed, %d\n", - __func__, ret); - return ret; - } - DBG("gpio on"); return 0; diff --git a/drivers/gpu/drm/tilcdc/tilcdc_panel.c b/drivers/gpu/drm/tilcdc/tilcdc_panel.c index 7a0315855e9013..0af8bed7ce1ee7 100644 --- a/drivers/gpu/drm/tilcdc/tilcdc_panel.c +++ b/drivers/gpu/drm/tilcdc/tilcdc_panel.c @@ -375,25 +375,17 @@ static int panel_probe(struct platform_device *pdev) dev_info(&pdev->dev, "found backlight\n"); } - panel_mod->enable_gpio = devm_gpiod_get(&pdev->dev, "enable"); + panel_mod->enable_gpio = devm_gpiod_get_optional(&pdev->dev, "enable", + GPIOD_OUT_LOW); if (IS_ERR(panel_mod->enable_gpio)) { ret = PTR_ERR(panel_mod->enable_gpio); - if (ret != -ENOENT) { - dev_err(&pdev->dev, "failed to request enable GPIO\n"); - goto fail_backlight; - } - - /* Optional GPIO is not here, continue silently. */ - panel_mod->enable_gpio = NULL; - } else { - ret = gpiod_direction_output(panel_mod->enable_gpio, 0); - if (ret < 0) { - dev_err(&pdev->dev, "failed to setup GPIO\n"); - goto fail_backlight; - } - dev_info(&pdev->dev, "found enable GPIO\n"); + dev_err(&pdev->dev, "failed to request enable GPIO\n"); + goto fail_backlight; } + if (panel_mod->enable_gpio) + dev_info(&pdev->dev, "found enable GPIO\n"); + mod = &panel_mod->base; pdev->dev.platform_data = mod; diff --git a/drivers/iio/light/stk3310.c b/drivers/iio/light/stk3310.c index f101bb5bddc7d7..993eb201148e08 100644 --- a/drivers/iio/light/stk3310.c +++ b/drivers/iio/light/stk3310.c @@ -469,16 +469,12 @@ static int stk3310_gpio_probe(struct i2c_client *client) dev = &client->dev; /* gpio interrupt pin */ - gpio = devm_gpiod_get_index(dev, STK3310_GPIO, 0); + gpio = devm_gpiod_get_index(dev, STK3310_GPIO, 0, GPIOD_IN); if (IS_ERR(gpio)) { dev_err(dev, "acpi gpio get index failed\n"); return PTR_ERR(gpio); } - ret = gpiod_direction_input(gpio); - if (ret) - return ret; - ret = gpiod_to_irq(gpio); dev_dbg(dev, "GPIO resource, no:%d irq:%d\n", desc_to_gpio(gpio), ret); diff --git a/drivers/iio/magnetometer/bmc150_magn.c b/drivers/iio/magnetometer/bmc150_magn.c index e330205f819475..d8e614ca069fe7 100644 --- a/drivers/iio/magnetometer/bmc150_magn.c +++ b/drivers/iio/magnetometer/bmc150_magn.c @@ -845,16 +845,12 @@ static int bmc150_magn_gpio_probe(struct i2c_client *client) dev = &client->dev; /* data ready GPIO interrupt pin */ - gpio = devm_gpiod_get_index(dev, BMC150_MAGN_GPIO_INT, 0); + gpio = devm_gpiod_get_index(dev, BMC150_MAGN_GPIO_INT, 0, GPIOD_IN); if (IS_ERR(gpio)) { dev_err(dev, "ACPI GPIO get index failed\n"); return PTR_ERR(gpio); } - ret = gpiod_direction_input(gpio); - if (ret) - return ret; - ret = gpiod_to_irq(gpio); dev_dbg(dev, "GPIO resource, no:%d irq:%d\n", desc_to_gpio(gpio), ret); diff --git a/drivers/input/misc/rb532_button.c b/drivers/input/misc/rb532_button.c index e956e81cd4e6de..62c5814c796bcf 100644 --- a/drivers/input/misc/rb532_button.c +++ b/drivers/input/misc/rb532_button.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index dae59145788340..1764bcf8ee6bed 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -41,20 +41,46 @@ static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller; static void __gic_irq_dispatch(void); -static inline unsigned int gic_read(unsigned int reg) +static inline u32 gic_read32(unsigned int reg) { return __raw_readl(gic_base + reg); } -static inline void gic_write(unsigned int reg, unsigned int val) +static inline u64 gic_read64(unsigned int reg) { - __raw_writel(val, gic_base + reg); + return __raw_readq(gic_base + reg); } -static inline void gic_update_bits(unsigned int reg, unsigned int mask, - unsigned int val) +static inline unsigned long gic_read(unsigned int reg) { - unsigned int regval; + if (!mips_cm_is64) + return gic_read32(reg); + else + return gic_read64(reg); +} + +static inline void gic_write32(unsigned int reg, u32 val) +{ + return __raw_writel(val, gic_base + reg); +} + +static inline void gic_write64(unsigned int reg, u64 val) +{ + return __raw_writeq(val, gic_base + reg); +} + +static inline void gic_write(unsigned int reg, unsigned long val) +{ + if (!mips_cm_is64) + return gic_write32(reg, (u32)val); + else + return gic_write64(reg, (u64)val); +} + +static inline void gic_update_bits(unsigned int reg, unsigned long mask, + unsigned long val) +{ + unsigned long regval; regval = gic_read(reg); regval &= ~mask; @@ -65,40 +91,40 @@ static inline void gic_update_bits(unsigned int reg, unsigned int mask, static inline void gic_reset_mask(unsigned int intr) { gic_write(GIC_REG(SHARED, GIC_SH_RMASK) + GIC_INTR_OFS(intr), - 1 << GIC_INTR_BIT(intr)); + 1ul << GIC_INTR_BIT(intr)); } static inline void gic_set_mask(unsigned int intr) { gic_write(GIC_REG(SHARED, GIC_SH_SMASK) + GIC_INTR_OFS(intr), - 1 << GIC_INTR_BIT(intr)); + 1ul << GIC_INTR_BIT(intr)); } static inline void gic_set_polarity(unsigned int intr, unsigned int pol) { gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_POLARITY) + - GIC_INTR_OFS(intr), 1 << GIC_INTR_BIT(intr), - pol << GIC_INTR_BIT(intr)); + GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr), + (unsigned long)pol << GIC_INTR_BIT(intr)); } static inline void gic_set_trigger(unsigned int intr, unsigned int trig) { gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_TRIGGER) + - GIC_INTR_OFS(intr), 1 << GIC_INTR_BIT(intr), - trig << GIC_INTR_BIT(intr)); + GIC_INTR_OFS(intr), 1ul << GIC_INTR_BIT(intr), + (unsigned long)trig << GIC_INTR_BIT(intr)); } static inline void gic_set_dual_edge(unsigned int intr, unsigned int dual) { gic_update_bits(GIC_REG(SHARED, GIC_SH_SET_DUAL) + GIC_INTR_OFS(intr), - 1 << GIC_INTR_BIT(intr), - dual << GIC_INTR_BIT(intr)); + 1ul << GIC_INTR_BIT(intr), + (unsigned long)dual << GIC_INTR_BIT(intr)); } static inline void gic_map_to_pin(unsigned int intr, unsigned int pin) { - gic_write(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_PIN_BASE) + - GIC_SH_MAP_TO_PIN(intr), GIC_MAP_TO_PIN_MSK | pin); + gic_write32(GIC_REG(SHARED, GIC_SH_INTR_MAP_TO_PIN_BASE) + + GIC_SH_MAP_TO_PIN(intr), GIC_MAP_TO_PIN_MSK | pin); } static inline void gic_map_to_vpe(unsigned int intr, unsigned int vpe) @@ -113,10 +139,13 @@ cycle_t gic_read_count(void) { unsigned int hi, hi2, lo; + if (mips_cm_is64) + return (cycle_t)gic_read(GIC_REG(SHARED, GIC_SH_COUNTER)); + do { - hi = gic_read(GIC_REG(SHARED, GIC_SH_COUNTER_63_32)); - lo = gic_read(GIC_REG(SHARED, GIC_SH_COUNTER_31_00)); - hi2 = gic_read(GIC_REG(SHARED, GIC_SH_COUNTER_63_32)); + hi = gic_read32(GIC_REG(SHARED, GIC_SH_COUNTER_63_32)); + lo = gic_read32(GIC_REG(SHARED, GIC_SH_COUNTER_31_00)); + hi2 = gic_read32(GIC_REG(SHARED, GIC_SH_COUNTER_63_32)); } while (hi2 != hi); return (((cycle_t) hi) << 32) + lo; @@ -135,10 +164,14 @@ unsigned int gic_get_count_width(void) void gic_write_compare(cycle_t cnt) { - gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE_HI), - (int)(cnt >> 32)); - gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE_LO), - (int)(cnt & 0xffffffff)); + if (mips_cm_is64) { + gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE), cnt); + } else { + gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE_HI), + (int)(cnt >> 32)); + gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE_LO), + (int)(cnt & 0xffffffff)); + } } void gic_write_cpu_compare(cycle_t cnt, int cpu) @@ -148,10 +181,15 @@ void gic_write_cpu_compare(cycle_t cnt, int cpu) local_irq_save(flags); gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), cpu); - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE_HI), - (int)(cnt >> 32)); - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE_LO), - (int)(cnt & 0xffffffff)); + + if (mips_cm_is64) { + gic_write(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE), cnt); + } else { + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE_HI), + (int)(cnt >> 32)); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE_LO), + (int)(cnt & 0xffffffff)); + } local_irq_restore(flags); } @@ -160,8 +198,11 @@ cycle_t gic_read_compare(void) { unsigned int hi, lo; - hi = gic_read(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE_HI)); - lo = gic_read(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE_LO)); + if (mips_cm_is64) + return (cycle_t)gic_read(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE)); + + hi = gic_read32(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE_HI)); + lo = gic_read32(GIC_REG(VPE_LOCAL, GIC_VPE_COMPARE_LO)); return (((cycle_t) hi) << 32) + lo; } @@ -196,7 +237,7 @@ static bool gic_local_irq_is_routable(int intr) if (cpu_has_veic) return true; - vpe_ctl = gic_read(GIC_REG(VPE_LOCAL, GIC_VPE_CTL)); + vpe_ctl = gic_read32(GIC_REG(VPE_LOCAL, GIC_VPE_CTL)); switch (intr) { case GIC_LOCAL_INT_TIMER: return vpe_ctl & GIC_VPE_CTL_TIMER_RTBL_MSK; @@ -262,7 +303,7 @@ int gic_get_c0_fdc_int(void) static void gic_handle_shared_int(bool chained) { - unsigned int i, intr, virq; + unsigned int i, intr, virq, gic_reg_step = mips_cm_is64 ? 8 : 4; unsigned long *pcpu_mask; unsigned long pending_reg, intrmask_reg; DECLARE_BITMAP(pending, GIC_MAX_INTRS); @@ -277,8 +318,8 @@ static void gic_handle_shared_int(bool chained) for (i = 0; i < BITS_TO_LONGS(gic_shared_intrs); i++) { pending[i] = gic_read(pending_reg); intrmask[i] = gic_read(intrmask_reg); - pending_reg += 0x4; - intrmask_reg += 0x4; + pending_reg += gic_reg_step; + intrmask_reg += gic_reg_step; } bitmap_and(pending, pending, intrmask, gic_shared_intrs); @@ -425,8 +466,8 @@ static void gic_handle_local_int(bool chained) unsigned long pending, masked; unsigned int intr, virq; - pending = gic_read(GIC_REG(VPE_LOCAL, GIC_VPE_PEND)); - masked = gic_read(GIC_REG(VPE_LOCAL, GIC_VPE_MASK)); + pending = gic_read32(GIC_REG(VPE_LOCAL, GIC_VPE_PEND)); + masked = gic_read32(GIC_REG(VPE_LOCAL, GIC_VPE_MASK)); bitmap_and(&pending, &pending, &masked, GIC_NUM_LOCAL_INTRS); @@ -449,14 +490,14 @@ static void gic_mask_local_irq(struct irq_data *d) { int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); - gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_RMASK), 1 << intr); + gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_RMASK), 1 << intr); } static void gic_unmask_local_irq(struct irq_data *d) { int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); - gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_SMASK), 1 << intr); + gic_write32(GIC_REG(VPE_LOCAL, GIC_VPE_SMASK), 1 << intr); } static struct irq_chip gic_local_irq_controller = { @@ -474,7 +515,7 @@ static void gic_mask_local_irq_all_vpes(struct irq_data *d) spin_lock_irqsave(&gic_lock, flags); for (i = 0; i < gic_vpes; i++) { gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), i); - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), 1 << intr); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), 1 << intr); } spin_unlock_irqrestore(&gic_lock, flags); } @@ -488,7 +529,7 @@ static void gic_unmask_local_irq_all_vpes(struct irq_data *d) spin_lock_irqsave(&gic_lock, flags); for (i = 0; i < gic_vpes; i++) { gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), i); - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), 1 << intr); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), 1 << intr); } spin_unlock_irqrestore(&gic_lock, flags); } @@ -608,7 +649,7 @@ static void __init gic_basic_init(void) for (j = 0; j < GIC_NUM_LOCAL_INTRS; j++) { if (!gic_local_irq_is_routable(j)) continue; - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), 1 << j); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), 1 << j); } } } @@ -653,27 +694,32 @@ static int gic_local_irq_domain_map(struct irq_domain *d, unsigned int virq, switch (intr) { case GIC_LOCAL_INT_WD: - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_WD_MAP), val); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_WD_MAP), val); break; case GIC_LOCAL_INT_COMPARE: - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE_MAP), val); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE_MAP), + val); break; case GIC_LOCAL_INT_TIMER: /* CONFIG_MIPS_CMP workaround (see __gic_init) */ val = GIC_MAP_TO_PIN_MSK | timer_cpu_pin; - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_TIMER_MAP), val); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_TIMER_MAP), + val); break; case GIC_LOCAL_INT_PERFCTR: - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_PERFCTR_MAP), val); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_PERFCTR_MAP), + val); break; case GIC_LOCAL_INT_SWINT0: - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_SWINT0_MAP), val); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SWINT0_MAP), + val); break; case GIC_LOCAL_INT_SWINT1: - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_SWINT1_MAP), val); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SWINT1_MAP), + val); break; case GIC_LOCAL_INT_FDC: - gic_write(GIC_REG(VPE_OTHER, GIC_VPE_FDC_MAP), val); + gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_FDC_MAP), val); break; default: pr_err("Invalid local IRQ %d\n", intr); @@ -778,7 +824,7 @@ static void __init __gic_init(unsigned long gic_base_addr, */ if (IS_ENABLED(CONFIG_MIPS_CMP) && gic_local_irq_is_routable(GIC_LOCAL_INT_TIMER)) { - timer_cpu_pin = gic_read(GIC_REG(VPE_LOCAL, + timer_cpu_pin = gic_read32(GIC_REG(VPE_LOCAL, GIC_VPE_TIMER_MAP)) & GIC_MAP_MSK; irq_set_chained_handler(MIPS_CPU_IRQ_BASE + diff --git a/drivers/leds/Kconfig b/drivers/leds/Kconfig index 23408bd68fdc0c..bea24bd4d519e4 100644 --- a/drivers/leds/Kconfig +++ b/drivers/leds/Kconfig @@ -565,6 +565,17 @@ config LEDS_BLINKM This option enables support for the BlinkM RGB LED connected through I2C. Say Y to enable support for the BlinkM LED. +config LEDS_POWERNV + tristate "LED support for PowerNV Platform" + depends on LEDS_CLASS + depends on PPC_POWERNV + depends on OF + help + This option enables support for the system LEDs present on + PowerNV platforms. Say 'y' to enable this support in kernel. + To compile this driver as a module, choose 'm' here: the module + will be called leds-powernv. + config LEDS_SYSCON bool "LED support for LEDs on system controllers" depends on LEDS_CLASS=y diff --git a/drivers/leds/Makefile b/drivers/leds/Makefile index 8d6a24a2f51376..6a943d16ecab03 100644 --- a/drivers/leds/Makefile +++ b/drivers/leds/Makefile @@ -65,6 +65,7 @@ obj-$(CONFIG_LEDS_VERSATILE) += leds-versatile.o obj-$(CONFIG_LEDS_MENF21BMC) += leds-menf21bmc.o obj-$(CONFIG_LEDS_PM8941_WLED) += leds-pm8941-wled.o obj-$(CONFIG_LEDS_KTD2692) += leds-ktd2692.o +obj-$(CONFIG_LEDS_POWERNV) += leds-powernv.o # LED SPI Drivers obj-$(CONFIG_LEDS_DAC124S085) += leds-dac124s085.o diff --git a/drivers/leds/leds-powernv.c b/drivers/leds/leds-powernv.c new file mode 100644 index 00000000000000..2c5c5b12ab6475 --- /dev/null +++ b/drivers/leds/leds-powernv.c @@ -0,0 +1,345 @@ +/* + * PowerNV LED Driver + * + * Copyright IBM Corp. 2015 + * + * Author: Vasant Hegde + * Author: Anshuman Khandual + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + */ + +#include +#include +#include +#include +#include +#include + +#include + +/* Map LED type to description. */ +struct led_type_map { + const int type; + const char *desc; +}; +static const struct led_type_map led_type_map[] = { + {OPAL_SLOT_LED_TYPE_ID, "identify"}, + {OPAL_SLOT_LED_TYPE_FAULT, "fault"}, + {OPAL_SLOT_LED_TYPE_ATTN, "attention"}, + {-1, NULL}, +}; + +struct powernv_led_common { + /* + * By default unload path resets all the LEDs. But on PowerNV + * platform we want to retain LED state across reboot as these + * are controlled by firmware. Also service processor can modify + * the LEDs independent of OS. Hence avoid resetting LEDs in + * unload path. + */ + bool led_disabled; + + /* Max supported LED type */ + __be64 max_led_type; + + /* glabal lock */ + struct mutex lock; +}; + +/* PowerNV LED data */ +struct powernv_led_data { + struct led_classdev cdev; + char *loc_code; /* LED location code */ + int led_type; /* OPAL_SLOT_LED_TYPE_* */ + + struct powernv_led_common *common; +}; + + +/* Returns OPAL_SLOT_LED_TYPE_* for given led type string */ +static int powernv_get_led_type(const char *led_type_desc) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(led_type_map); i++) + if (!strcmp(led_type_map[i].desc, led_type_desc)) + return led_type_map[i].type; + + return -1; +} + +/* + * This commits the state change of the requested LED through an OPAL call. + * This function is called from work queue task context when ever it gets + * scheduled. This function can sleep at opal_async_wait_response call. + */ +static void powernv_led_set(struct powernv_led_data *powernv_led, + enum led_brightness value) +{ + int rc, token; + u64 led_mask, led_value = 0; + __be64 max_type; + struct opal_msg msg; + struct device *dev = powernv_led->cdev.dev; + struct powernv_led_common *powernv_led_common = powernv_led->common; + + /* Prepare for the OPAL call */ + max_type = powernv_led_common->max_led_type; + led_mask = OPAL_SLOT_LED_STATE_ON << powernv_led->led_type; + if (value) + led_value = led_mask; + + /* OPAL async call */ + token = opal_async_get_token_interruptible(); + if (token < 0) { + if (token != -ERESTARTSYS) + dev_err(dev, "%s: Couldn't get OPAL async token\n", + __func__); + return; + } + + rc = opal_leds_set_ind(token, powernv_led->loc_code, + led_mask, led_value, &max_type); + if (rc != OPAL_ASYNC_COMPLETION) { + dev_err(dev, "%s: OPAL set LED call failed for %s [rc=%d]\n", + __func__, powernv_led->loc_code, rc); + goto out_token; + } + + rc = opal_async_wait_response(token, &msg); + if (rc) { + dev_err(dev, + "%s: Failed to wait for the async response [rc=%d]\n", + __func__, rc); + goto out_token; + } + + rc = be64_to_cpu(msg.params[1]); + if (rc != OPAL_SUCCESS) + dev_err(dev, "%s : OAPL async call returned failed [rc=%d]\n", + __func__, rc); + +out_token: + opal_async_release_token(token); +} + +/* + * This function fetches the LED state for a given LED type for + * mentioned LED classdev structure. + */ +static enum led_brightness powernv_led_get(struct powernv_led_data *powernv_led) +{ + int rc; + __be64 mask, value, max_type; + u64 led_mask, led_value; + struct device *dev = powernv_led->cdev.dev; + struct powernv_led_common *powernv_led_common = powernv_led->common; + + /* Fetch all LED status */ + mask = cpu_to_be64(0); + value = cpu_to_be64(0); + max_type = powernv_led_common->max_led_type; + + rc = opal_leds_get_ind(powernv_led->loc_code, + &mask, &value, &max_type); + if (rc != OPAL_SUCCESS && rc != OPAL_PARTIAL) { + dev_err(dev, "%s: OPAL get led call failed [rc=%d]\n", + __func__, rc); + return LED_OFF; + } + + led_mask = be64_to_cpu(mask); + led_value = be64_to_cpu(value); + + /* LED status available */ + if (!((led_mask >> powernv_led->led_type) & OPAL_SLOT_LED_STATE_ON)) { + dev_err(dev, "%s: LED status not available for %s\n", + __func__, powernv_led->cdev.name); + return LED_OFF; + } + + /* LED status value */ + if ((led_value >> powernv_led->led_type) & OPAL_SLOT_LED_STATE_ON) + return LED_FULL; + + return LED_OFF; +} + +/* + * LED classdev 'brightness_get' function. This schedules work + * to update LED state. + */ +static void powernv_brightness_set(struct led_classdev *led_cdev, + enum led_brightness value) +{ + struct powernv_led_data *powernv_led = + container_of(led_cdev, struct powernv_led_data, cdev); + struct powernv_led_common *powernv_led_common = powernv_led->common; + + /* Do not modify LED in unload path */ + if (powernv_led_common->led_disabled) + return; + + mutex_lock(&powernv_led_common->lock); + powernv_led_set(powernv_led, value); + mutex_unlock(&powernv_led_common->lock); +} + +/* LED classdev 'brightness_get' function */ +static enum led_brightness powernv_brightness_get(struct led_classdev *led_cdev) +{ + struct powernv_led_data *powernv_led = + container_of(led_cdev, struct powernv_led_data, cdev); + + return powernv_led_get(powernv_led); +} + +/* + * This function registers classdev structure for any given type of LED on + * a given child LED device node. + */ +static int powernv_led_create(struct device *dev, + struct powernv_led_data *powernv_led, + const char *led_type_desc) +{ + int rc; + + /* Make sure LED type is supported */ + powernv_led->led_type = powernv_get_led_type(led_type_desc); + if (powernv_led->led_type == -1) { + dev_warn(dev, "%s: No support for led type : %s\n", + __func__, led_type_desc); + return -EINVAL; + } + + /* Create the name for classdev */ + powernv_led->cdev.name = devm_kasprintf(dev, GFP_KERNEL, "%s:%s", + powernv_led->loc_code, + led_type_desc); + if (!powernv_led->cdev.name) { + dev_err(dev, + "%s: Memory allocation failed for classdev name\n", + __func__); + return -ENOMEM; + } + + powernv_led->cdev.brightness_set = powernv_brightness_set; + powernv_led->cdev.brightness_get = powernv_brightness_get; + powernv_led->cdev.brightness = LED_OFF; + powernv_led->cdev.max_brightness = LED_FULL; + + /* Register the classdev */ + rc = devm_led_classdev_register(dev, &powernv_led->cdev); + if (rc) { + dev_err(dev, "%s: Classdev registration failed for %s\n", + __func__, powernv_led->cdev.name); + } + + return rc; +} + +/* Go through LED device tree node and register LED classdev structure */ +static int powernv_led_classdev(struct platform_device *pdev, + struct device_node *led_node, + struct powernv_led_common *powernv_led_common) +{ + const char *cur = NULL; + int rc = -1; + struct property *p; + struct device_node *np; + struct powernv_led_data *powernv_led; + struct device *dev = &pdev->dev; + + for_each_child_of_node(led_node, np) { + p = of_find_property(np, "led-types", NULL); + if (!p) + continue; + + while ((cur = of_prop_next_string(p, cur)) != NULL) { + powernv_led = devm_kzalloc(dev, sizeof(*powernv_led), + GFP_KERNEL); + if (!powernv_led) + return -ENOMEM; + + powernv_led->common = powernv_led_common; + powernv_led->loc_code = (char *)np->name; + + rc = powernv_led_create(dev, powernv_led, cur); + if (rc) + return rc; + } /* while end */ + } + + return rc; +} + +/* Platform driver probe */ +static int powernv_led_probe(struct platform_device *pdev) +{ + struct device_node *led_node; + struct powernv_led_common *powernv_led_common; + struct device *dev = &pdev->dev; + + led_node = of_find_node_by_path("/ibm,opal/leds"); + if (!led_node) { + dev_err(dev, "%s: LED parent device node not found\n", + __func__); + return -EINVAL; + } + + powernv_led_common = devm_kzalloc(dev, sizeof(*powernv_led_common), + GFP_KERNEL); + if (!powernv_led_common) + return -ENOMEM; + + mutex_init(&powernv_led_common->lock); + powernv_led_common->max_led_type = cpu_to_be64(OPAL_SLOT_LED_TYPE_MAX); + + platform_set_drvdata(pdev, powernv_led_common); + + return powernv_led_classdev(pdev, led_node, powernv_led_common); +} + +/* Platform driver remove */ +static int powernv_led_remove(struct platform_device *pdev) +{ + struct powernv_led_common *powernv_led_common; + + /* Disable LED operation */ + powernv_led_common = platform_get_drvdata(pdev); + powernv_led_common->led_disabled = true; + + /* Destroy lock */ + mutex_destroy(&powernv_led_common->lock); + + dev_info(&pdev->dev, "PowerNV led module unregistered\n"); + return 0; +} + +/* Platform driver property match */ +static const struct of_device_id powernv_led_match[] = { + { + .compatible = "ibm,opal-v3-led", + }, + {}, +}; +MODULE_DEVICE_TABLE(of, powernv_led_match); + +static struct platform_driver powernv_led_driver = { + .probe = powernv_led_probe, + .remove = powernv_led_remove, + .driver = { + .name = "powernv-led-driver", + .of_match_table = powernv_led_match, + }, +}; + +module_platform_driver(powernv_led_driver); + +MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("PowerNV LED driver"); +MODULE_AUTHOR("Vasant Hegde "); diff --git a/drivers/macintosh/therm_windtunnel.c b/drivers/macintosh/therm_windtunnel.c index 109dcaa15934e4..68dcbcb4fc5bc2 100644 --- a/drivers/macintosh/therm_windtunnel.c +++ b/drivers/macintosh/therm_windtunnel.c @@ -408,6 +408,7 @@ static const struct i2c_device_id therm_windtunnel_id[] = { { "therm_adm1030", adm1030 }, { } }; +MODULE_DEVICE_TABLE(i2c, therm_windtunnel_id); static int do_probe(struct i2c_client *cl, const struct i2c_device_id *id) @@ -459,6 +460,7 @@ static const struct of_device_id therm_of_match[] = {{ .compatible = "adm1030" }, {} }; +MODULE_DEVICE_TABLE(of, therm_of_match); static struct platform_driver therm_of_driver = { .driver = { diff --git a/drivers/macintosh/windfarm.h b/drivers/macintosh/windfarm.h index 028cdac2d33d3f..901c42f71b5a14 100644 --- a/drivers/macintosh/windfarm.h +++ b/drivers/macintosh/windfarm.h @@ -53,11 +53,9 @@ struct wf_control { * the kref and wf_unregister_control will decrement it, thus the * object creating/disposing a given control shouldn't assume it * still exists after wf_unregister_control has been called. - * wf_find_control will inc the refcount for you */ extern int wf_register_control(struct wf_control *ct); extern void wf_unregister_control(struct wf_control *ct); -extern struct wf_control * wf_find_control(const char *name); extern int wf_get_control(struct wf_control *ct); extern void wf_put_control(struct wf_control *ct); @@ -117,7 +115,6 @@ struct wf_sensor { /* Same lifetime rules as controls */ extern int wf_register_sensor(struct wf_sensor *sr); extern void wf_unregister_sensor(struct wf_sensor *sr); -extern struct wf_sensor * wf_find_sensor(const char *name); extern int wf_get_sensor(struct wf_sensor *sr); extern void wf_put_sensor(struct wf_sensor *sr); @@ -144,7 +141,6 @@ extern int wf_unregister_client(struct notifier_block *nb); /* Overtemp conditions. Those are refcounted */ extern void wf_set_overtemp(void); extern void wf_clear_overtemp(void); -extern int wf_is_overtemp(void); #define WF_EVENT_NEW_CONTROL 0 /* param is wf_control * */ #define WF_EVENT_NEW_SENSOR 1 /* param is wf_sensor * */ diff --git a/drivers/macintosh/windfarm_core.c b/drivers/macintosh/windfarm_core.c index 3ee198b6584382..465d770ab0bb09 100644 --- a/drivers/macintosh/windfarm_core.c +++ b/drivers/macintosh/windfarm_core.c @@ -72,7 +72,7 @@ static inline void wf_notify(int event, void *param) blocking_notifier_call_chain(&wf_client_list, event, param); } -int wf_critical_overtemp(void) +static int wf_critical_overtemp(void) { static char * critical_overtemp_path = "/sbin/critical_overtemp"; char *argv[] = { critical_overtemp_path, NULL }; @@ -84,7 +84,6 @@ int wf_critical_overtemp(void) return call_usermodehelper(critical_overtemp_path, argv, envp, UMH_WAIT_EXEC); } -EXPORT_SYMBOL_GPL(wf_critical_overtemp); static int wf_thread_func(void *data) { @@ -255,24 +254,6 @@ void wf_unregister_control(struct wf_control *ct) } EXPORT_SYMBOL_GPL(wf_unregister_control); -struct wf_control * wf_find_control(const char *name) -{ - struct wf_control *ct; - - mutex_lock(&wf_lock); - list_for_each_entry(ct, &wf_controls, link) { - if (!strcmp(ct->name, name)) { - if (wf_get_control(ct)) - ct = NULL; - mutex_unlock(&wf_lock); - return ct; - } - } - mutex_unlock(&wf_lock); - return NULL; -} -EXPORT_SYMBOL_GPL(wf_find_control); - int wf_get_control(struct wf_control *ct) { if (!try_module_get(ct->ops->owner)) @@ -368,24 +349,6 @@ void wf_unregister_sensor(struct wf_sensor *sr) } EXPORT_SYMBOL_GPL(wf_unregister_sensor); -struct wf_sensor * wf_find_sensor(const char *name) -{ - struct wf_sensor *sr; - - mutex_lock(&wf_lock); - list_for_each_entry(sr, &wf_sensors, link) { - if (!strcmp(sr->name, name)) { - if (wf_get_sensor(sr)) - sr = NULL; - mutex_unlock(&wf_lock); - return sr; - } - } - mutex_unlock(&wf_lock); - return NULL; -} -EXPORT_SYMBOL_GPL(wf_find_sensor); - int wf_get_sensor(struct wf_sensor *sr) { if (!try_module_get(sr->ops->owner)) @@ -435,7 +398,7 @@ int wf_unregister_client(struct notifier_block *nb) { mutex_lock(&wf_lock); blocking_notifier_chain_unregister(&wf_client_list, nb); - wf_client_count++; + wf_client_count--; if (wf_client_count == 0) wf_stop_thread(); mutex_unlock(&wf_lock); @@ -474,12 +437,6 @@ void wf_clear_overtemp(void) } EXPORT_SYMBOL_GPL(wf_clear_overtemp); -int wf_is_overtemp(void) -{ - return (wf_overtemp != 0); -} -EXPORT_SYMBOL_GPL(wf_is_overtemp); - static int __init windfarm_core_init(void) { DBG("wf: core loaded\n"); diff --git a/drivers/media/i2c/adp1653.c b/drivers/media/i2c/adp1653.c index c70ababce954d1..5dd39775d6ca5c 100644 --- a/drivers/media/i2c/adp1653.c +++ b/drivers/media/i2c/adp1653.c @@ -465,7 +465,7 @@ static int adp1653_of_init(struct i2c_client *client, of_node_put(child); - pd->enable_gpio = devm_gpiod_get(&client->dev, "enable"); + pd->enable_gpio = devm_gpiod_get(&client->dev, "enable", GPIOD_OUT_LOW); if (!pd->enable_gpio) { dev_err(&client->dev, "Error getting GPIO\n"); return -EINVAL; diff --git a/drivers/memory/fsl_ifc.c b/drivers/memory/fsl_ifc.c index 410c39749872b0..e87459f6d686d0 100644 --- a/drivers/memory/fsl_ifc.c +++ b/drivers/memory/fsl_ifc.c @@ -62,7 +62,7 @@ int fsl_ifc_find(phys_addr_t addr_base) return -ENODEV; for (i = 0; i < fsl_ifc_ctrl_dev->banks; i++) { - u32 cspr = in_be32(&fsl_ifc_ctrl_dev->regs->cspr_cs[i].cspr); + u32 cspr = ifc_in32(&fsl_ifc_ctrl_dev->regs->cspr_cs[i].cspr); if (cspr & CSPR_V && (cspr & CSPR_BA) == convert_ifc_address(addr_base)) return i; @@ -79,16 +79,16 @@ static int fsl_ifc_ctrl_init(struct fsl_ifc_ctrl *ctrl) /* * Clear all the common status and event registers */ - if (in_be32(&ifc->cm_evter_stat) & IFC_CM_EVTER_STAT_CSER) - out_be32(&ifc->cm_evter_stat, IFC_CM_EVTER_STAT_CSER); + if (ifc_in32(&ifc->cm_evter_stat) & IFC_CM_EVTER_STAT_CSER) + ifc_out32(IFC_CM_EVTER_STAT_CSER, &ifc->cm_evter_stat); /* enable all error and events */ - out_be32(&ifc->cm_evter_en, IFC_CM_EVTER_EN_CSEREN); + ifc_out32(IFC_CM_EVTER_EN_CSEREN, &ifc->cm_evter_en); /* enable all error and event interrupts */ - out_be32(&ifc->cm_evter_intr_en, IFC_CM_EVTER_INTR_EN_CSERIREN); - out_be32(&ifc->cm_erattr0, 0x0); - out_be32(&ifc->cm_erattr1, 0x0); + ifc_out32(IFC_CM_EVTER_INTR_EN_CSERIREN, &ifc->cm_evter_intr_en); + ifc_out32(0x0, &ifc->cm_erattr0); + ifc_out32(0x0, &ifc->cm_erattr1); return 0; } @@ -127,9 +127,9 @@ static u32 check_nand_stat(struct fsl_ifc_ctrl *ctrl) spin_lock_irqsave(&nand_irq_lock, flags); - stat = in_be32(&ifc->ifc_nand.nand_evter_stat); + stat = ifc_in32(&ifc->ifc_nand.nand_evter_stat); if (stat) { - out_be32(&ifc->ifc_nand.nand_evter_stat, stat); + ifc_out32(stat, &ifc->ifc_nand.nand_evter_stat); ctrl->nand_stat = stat; wake_up(&ctrl->nand_wait); } @@ -161,16 +161,16 @@ static irqreturn_t fsl_ifc_ctrl_irq(int irqno, void *data) irqreturn_t ret = IRQ_NONE; /* read for chip select error */ - cs_err = in_be32(&ifc->cm_evter_stat); + cs_err = ifc_in32(&ifc->cm_evter_stat); if (cs_err) { dev_err(ctrl->dev, "transaction sent to IFC is not mapped to" "any memory bank 0x%08X\n", cs_err); /* clear the chip select error */ - out_be32(&ifc->cm_evter_stat, IFC_CM_EVTER_STAT_CSER); + ifc_out32(IFC_CM_EVTER_STAT_CSER, &ifc->cm_evter_stat); /* read error attribute registers print the error information */ - status = in_be32(&ifc->cm_erattr0); - err_addr = in_be32(&ifc->cm_erattr1); + status = ifc_in32(&ifc->cm_erattr0); + err_addr = ifc_in32(&ifc->cm_erattr1); if (status & IFC_CM_ERATTR0_ERTYP_READ) dev_err(ctrl->dev, "Read transaction error" @@ -231,6 +231,23 @@ static int fsl_ifc_ctrl_probe(struct platform_device *dev) goto err; } + version = ifc_in32(&fsl_ifc_ctrl_dev->regs->ifc_rev) & + FSL_IFC_VERSION_MASK; + banks = (version == FSL_IFC_VERSION_1_0_0) ? 4 : 8; + dev_info(&dev->dev, "IFC version %d.%d, %d banks\n", + version >> 24, (version >> 16) & 0xf, banks); + + fsl_ifc_ctrl_dev->version = version; + fsl_ifc_ctrl_dev->banks = banks; + + if (of_property_read_bool(dev->dev.of_node, "little-endian")) { + fsl_ifc_ctrl_dev->little_endian = true; + dev_dbg(&dev->dev, "IFC REGISTERS are LITTLE endian\n"); + } else { + fsl_ifc_ctrl_dev->little_endian = false; + dev_dbg(&dev->dev, "IFC REGISTERS are BIG endian\n"); + } + version = ioread32be(&fsl_ifc_ctrl_dev->regs->ifc_rev) & FSL_IFC_VERSION_MASK; banks = (version == FSL_IFC_VERSION_1_0_0) ? 4 : 8; diff --git a/drivers/misc/cxl/Kconfig b/drivers/misc/cxl/Kconfig index b6db9ebd52c298..8756d06e2bb822 100644 --- a/drivers/misc/cxl/Kconfig +++ b/drivers/misc/cxl/Kconfig @@ -11,11 +11,16 @@ config CXL_KERNEL_API bool default n +config CXL_EEH + bool + default n + config CXL tristate "Support for IBM Coherent Accelerators (CXL)" - depends on PPC_POWERNV && PCI_MSI + depends on PPC_POWERNV && PCI_MSI && EEH select CXL_BASE select CXL_KERNEL_API + select CXL_EEH default m help Select this option to enable driver support for IBM Coherent diff --git a/drivers/misc/cxl/Makefile b/drivers/misc/cxl/Makefile index 14e3f8219a11cf..6f484dfe78f937 100644 --- a/drivers/misc/cxl/Makefile +++ b/drivers/misc/cxl/Makefile @@ -1,3 +1,5 @@ +ccflags-y := -Werror + cxl-y += main.o file.o irq.o fault.o native.o cxl-y += context.o sysfs.o debugfs.o pci.o trace.o cxl-y += vphb.o api.o diff --git a/drivers/misc/cxl/api.c b/drivers/misc/cxl/api.c index 729e0851167df9..8af12c884b04ee 100644 --- a/drivers/misc/cxl/api.c +++ b/drivers/misc/cxl/api.c @@ -12,11 +12,13 @@ #include #include #include +#include #include "cxl.h" struct cxl_context *cxl_dev_context_init(struct pci_dev *dev) { + struct address_space *mapping; struct cxl_afu *afu; struct cxl_context *ctx; int rc; @@ -25,19 +27,42 @@ struct cxl_context *cxl_dev_context_init(struct pci_dev *dev) get_device(&afu->dev); ctx = cxl_context_alloc(); - if (IS_ERR(ctx)) - return ctx; + if (IS_ERR(ctx)) { + rc = PTR_ERR(ctx); + goto err_dev; + } - /* Make it a slave context. We can promote it later? */ - rc = cxl_context_init(ctx, afu, false, NULL); - if (rc) { - kfree(ctx); - put_device(&afu->dev); - return ERR_PTR(-ENOMEM); + ctx->kernelapi = true; + + /* + * Make our own address space since we won't have one from the + * filesystem like the user api has, and even if we do associate a file + * with this context we don't want to use the global anonymous inode's + * address space as that can invalidate unrelated users: + */ + mapping = kmalloc(sizeof(struct address_space), GFP_KERNEL); + if (!mapping) { + rc = -ENOMEM; + goto err_ctx; } + address_space_init_once(mapping); + + /* Make it a slave context. We can promote it later? */ + rc = cxl_context_init(ctx, afu, false, mapping); + if (rc) + goto err_mapping; + cxl_assign_psn_space(ctx); return ctx; + +err_mapping: + kfree(mapping); +err_ctx: + kfree(ctx); +err_dev: + put_device(&afu->dev); + return ERR_PTR(rc); } EXPORT_SYMBOL_GPL(cxl_dev_context_init); @@ -59,7 +84,7 @@ EXPORT_SYMBOL_GPL(cxl_get_phys_dev); int cxl_release_context(struct cxl_context *ctx) { - if (ctx->status != CLOSED) + if (ctx->status >= STARTED) return -EBUSY; put_device(&ctx->afu->dev); @@ -255,9 +280,16 @@ struct file *cxl_get_fd(struct cxl_context *ctx, struct file_operations *fops, file = anon_inode_getfile("cxl", fops, ctx, flags); if (IS_ERR(file)) - put_unused_fd(fdtmp); + goto err_fd; + + file->f_mapping = ctx->mapping; + *fd = fdtmp; return file; + +err_fd: + put_unused_fd(fdtmp); + return NULL; } EXPORT_SYMBOL_GPL(cxl_get_fd); @@ -327,3 +359,10 @@ int cxl_afu_reset(struct cxl_context *ctx) return cxl_afu_check_and_enable(afu); } EXPORT_SYMBOL_GPL(cxl_afu_reset); + +void cxl_perst_reloads_same_image(struct cxl_afu *afu, + bool perst_reloads_same_image) +{ + afu->adapter->perst_same_image = perst_reloads_same_image; +} +EXPORT_SYMBOL_GPL(cxl_perst_reloads_same_image); diff --git a/drivers/misc/cxl/context.c b/drivers/misc/cxl/context.c index 1287148629c0d6..e762f85ee233a4 100644 --- a/drivers/misc/cxl/context.c +++ b/drivers/misc/cxl/context.c @@ -126,6 +126,18 @@ static int cxl_mmap_fault(struct vm_area_struct *vma, struct vm_fault *vmf) if (ctx->status != STARTED) { mutex_unlock(&ctx->status_mutex); pr_devel("%s: Context not started, failing problem state access\n", __func__); + if (ctx->mmio_err_ff) { + if (!ctx->ff_page) { + ctx->ff_page = alloc_page(GFP_USER); + if (!ctx->ff_page) + return VM_FAULT_OOM; + memset(page_address(ctx->ff_page), 0xff, PAGE_SIZE); + } + get_page(ctx->ff_page); + vmf->page = ctx->ff_page; + vma->vm_page_prot = pgprot_cached(vma->vm_page_prot); + return 0; + } return VM_FAULT_SIGBUS; } @@ -193,7 +205,11 @@ int __detach_context(struct cxl_context *ctx) if (status != STARTED) return -EBUSY; - WARN_ON(cxl_detach_process(ctx)); + /* Only warn if we detached while the link was OK. + * If detach fails when hw is down, we don't care. + */ + WARN_ON(cxl_detach_process(ctx) && + cxl_adapter_link_ok(ctx->afu->adapter)); flush_work(&ctx->fault_work); /* Only needed for dedicated process */ put_pid(ctx->pid); cxl_ctx_put(); @@ -253,7 +269,11 @@ static void reclaim_ctx(struct rcu_head *rcu) struct cxl_context *ctx = container_of(rcu, struct cxl_context, rcu); free_page((u64)ctx->sstp); + if (ctx->ff_page) + __free_page(ctx->ff_page); ctx->sstp = NULL; + if (ctx->kernelapi) + kfree(ctx->mapping); kfree(ctx); } diff --git a/drivers/misc/cxl/cxl.h b/drivers/misc/cxl/cxl.h index 4fd66cabde1ef1..1c30ef77073d60 100644 --- a/drivers/misc/cxl/cxl.h +++ b/drivers/misc/cxl/cxl.h @@ -34,7 +34,7 @@ extern uint cxl_verbose; * Bump version each time a user API change is made, whether it is * backwards compatible ot not. */ -#define CXL_API_VERSION 1 +#define CXL_API_VERSION 2 #define CXL_API_VERSION_COMPATIBLE 1 /* @@ -83,8 +83,10 @@ static const cxl_p1_reg_t CXL_PSL_AFUSEL = {0x00B0}; /* 0x00C0:7EFF Implementation dependent area */ static const cxl_p1_reg_t CXL_PSL_FIR1 = {0x0100}; static const cxl_p1_reg_t CXL_PSL_FIR2 = {0x0108}; +static const cxl_p1_reg_t CXL_PSL_Timebase = {0x0110}; static const cxl_p1_reg_t CXL_PSL_VERSION = {0x0118}; static const cxl_p1_reg_t CXL_PSL_RESLCKTO = {0x0128}; +static const cxl_p1_reg_t CXL_PSL_TB_CTLSTAT = {0x0140}; static const cxl_p1_reg_t CXL_PSL_FIR_CNTL = {0x0148}; static const cxl_p1_reg_t CXL_PSL_DSNDCTL = {0x0150}; static const cxl_p1_reg_t CXL_PSL_SNWRALLOC = {0x0158}; @@ -152,6 +154,9 @@ static const cxl_p2n_reg_t CXL_PSL_WED_An = {0x0A0}; #define CXL_PSL_SPAP_Size_Shift 4 #define CXL_PSL_SPAP_V 0x0000000000000001ULL +/****** CXL_PSL_Control ****************************************************/ +#define CXL_PSL_Control_tb 0x0000000000000001ULL + /****** CXL_PSL_DLCNTL *****************************************************/ #define CXL_PSL_DLCNTL_D (0x1ull << (63-28)) #define CXL_PSL_DLCNTL_C (0x1ull << (63-29)) @@ -418,6 +423,9 @@ struct cxl_context { /* Used to unmap any mmaps when force detaching */ struct address_space *mapping; struct mutex mapping_lock; + struct page *ff_page; + bool mmio_err_ff; + bool kernelapi; spinlock_t sste_lock; /* Protects segment table entries */ struct cxl_sste *sstp; @@ -493,6 +501,7 @@ struct cxl { bool user_image_loaded; bool perst_loads_image; bool perst_select_user; + bool perst_same_image; }; int cxl_alloc_one_irq(struct cxl *adapter); @@ -531,16 +540,33 @@ struct cxl_process_element { __be32 software_state; } __packed; +static inline bool cxl_adapter_link_ok(struct cxl *cxl) +{ + struct pci_dev *pdev; + + pdev = to_pci_dev(cxl->dev.parent); + return !pci_channel_offline(pdev); +} + static inline void __iomem *_cxl_p1_addr(struct cxl *cxl, cxl_p1_reg_t reg) { WARN_ON(!cpu_has_feature(CPU_FTR_HVMODE)); return cxl->p1_mmio + cxl_reg_off(reg); } -#define cxl_p1_write(cxl, reg, val) \ - out_be64(_cxl_p1_addr(cxl, reg), val) -#define cxl_p1_read(cxl, reg) \ - in_be64(_cxl_p1_addr(cxl, reg)) +static inline void cxl_p1_write(struct cxl *cxl, cxl_p1_reg_t reg, u64 val) +{ + if (likely(cxl_adapter_link_ok(cxl))) + out_be64(_cxl_p1_addr(cxl, reg), val); +} + +static inline u64 cxl_p1_read(struct cxl *cxl, cxl_p1_reg_t reg) +{ + if (likely(cxl_adapter_link_ok(cxl))) + return in_be64(_cxl_p1_addr(cxl, reg)); + else + return ~0ULL; +} static inline void __iomem *_cxl_p1n_addr(struct cxl_afu *afu, cxl_p1n_reg_t reg) { @@ -548,26 +574,56 @@ static inline void __iomem *_cxl_p1n_addr(struct cxl_afu *afu, cxl_p1n_reg_t reg return afu->p1n_mmio + cxl_reg_off(reg); } -#define cxl_p1n_write(afu, reg, val) \ - out_be64(_cxl_p1n_addr(afu, reg), val) -#define cxl_p1n_read(afu, reg) \ - in_be64(_cxl_p1n_addr(afu, reg)) +static inline void cxl_p1n_write(struct cxl_afu *afu, cxl_p1n_reg_t reg, u64 val) +{ + if (likely(cxl_adapter_link_ok(afu->adapter))) + out_be64(_cxl_p1n_addr(afu, reg), val); +} + +static inline u64 cxl_p1n_read(struct cxl_afu *afu, cxl_p1n_reg_t reg) +{ + if (likely(cxl_adapter_link_ok(afu->adapter))) + return in_be64(_cxl_p1n_addr(afu, reg)); + else + return ~0ULL; +} static inline void __iomem *_cxl_p2n_addr(struct cxl_afu *afu, cxl_p2n_reg_t reg) { return afu->p2n_mmio + cxl_reg_off(reg); } -#define cxl_p2n_write(afu, reg, val) \ - out_be64(_cxl_p2n_addr(afu, reg), val) -#define cxl_p2n_read(afu, reg) \ - in_be64(_cxl_p2n_addr(afu, reg)) +static inline void cxl_p2n_write(struct cxl_afu *afu, cxl_p2n_reg_t reg, u64 val) +{ + if (likely(cxl_adapter_link_ok(afu->adapter))) + out_be64(_cxl_p2n_addr(afu, reg), val); +} + +static inline u64 cxl_p2n_read(struct cxl_afu *afu, cxl_p2n_reg_t reg) +{ + if (likely(cxl_adapter_link_ok(afu->adapter))) + return in_be64(_cxl_p2n_addr(afu, reg)); + else + return ~0ULL; +} +static inline u64 cxl_afu_cr_read64(struct cxl_afu *afu, int cr, u64 off) +{ + if (likely(cxl_adapter_link_ok(afu->adapter))) + return in_le64((afu)->afu_desc_mmio + (afu)->crs_offset + + ((cr) * (afu)->crs_len) + (off)); + else + return ~0ULL; +} -#define cxl_afu_cr_read64(afu, cr, off) \ - in_le64((afu)->afu_desc_mmio + (afu)->crs_offset + ((cr) * (afu)->crs_len) + (off)) -#define cxl_afu_cr_read32(afu, cr, off) \ - in_le32((afu)->afu_desc_mmio + (afu)->crs_offset + ((cr) * (afu)->crs_len) + (off)) +static inline u32 cxl_afu_cr_read32(struct cxl_afu *afu, int cr, u64 off) +{ + if (likely(cxl_adapter_link_ok(afu->adapter))) + return in_le32((afu)->afu_desc_mmio + (afu)->crs_offset + + ((cr) * (afu)->crs_len) + (off)); + else + return 0xffffffff; +} u16 cxl_afu_cr_read16(struct cxl_afu *afu, int cr, u64 off); u8 cxl_afu_cr_read8(struct cxl_afu *afu, int cr, u64 off); @@ -585,6 +641,9 @@ void unregister_cxl_calls(struct cxl_calls *calls); int cxl_alloc_adapter_nr(struct cxl *adapter); void cxl_remove_adapter_nr(struct cxl *adapter); +int cxl_alloc_spa(struct cxl_afu *afu); +void cxl_release_spa(struct cxl_afu *afu); + int cxl_file_init(void); void cxl_file_exit(void); int cxl_register_adapter(struct cxl *adapter); @@ -675,6 +734,7 @@ int cxl_psl_purge(struct cxl_afu *afu); void cxl_stop_trace(struct cxl *cxl); int cxl_pci_vphb_add(struct cxl_afu *afu); +void cxl_pci_vphb_reconfigure(struct cxl_afu *afu); void cxl_pci_vphb_remove(struct cxl_afu *afu); extern struct pci_driver cxl_pci_driver; diff --git a/drivers/misc/cxl/debugfs.c b/drivers/misc/cxl/debugfs.c index 825c412580bca1..18df6f44af2a80 100644 --- a/drivers/misc/cxl/debugfs.c +++ b/drivers/misc/cxl/debugfs.c @@ -48,7 +48,7 @@ DEFINE_SIMPLE_ATTRIBUTE(fops_io_x64, debugfs_io_u64_get, debugfs_io_u64_set, "0x static struct dentry *debugfs_create_io_x64(const char *name, umode_t mode, struct dentry *parent, u64 __iomem *value) { - return debugfs_create_file(name, mode, parent, (void *)value, &fops_io_x64); + return debugfs_create_file(name, mode, parent, (void __force *)value, &fops_io_x64); } int cxl_debugfs_adapter_add(struct cxl *adapter) diff --git a/drivers/misc/cxl/file.c b/drivers/misc/cxl/file.c index e3f4b69527a9bd..a30bf285b5bdd7 100644 --- a/drivers/misc/cxl/file.c +++ b/drivers/misc/cxl/file.c @@ -73,6 +73,11 @@ static int __afu_open(struct inode *inode, struct file *file, bool master) if (!afu->current_mode) goto err_put_afu; + if (!cxl_adapter_link_ok(adapter)) { + rc = -EIO; + goto err_put_afu; + } + if (!(ctx = cxl_context_alloc())) { rc = -ENOMEM; goto err_put_afu; @@ -179,6 +184,8 @@ static long afu_ioctl_start_work(struct cxl_context *ctx, if (work.flags & CXL_START_WORK_AMR) amr = work.amr & mfspr(SPRN_UAMOR); + ctx->mmio_err_ff = !!(work.flags & CXL_START_WORK_ERR_FF); + /* * We grab the PID here and not in the file open to allow for the case * where a process (master, some daemon, etc) has opened the chardev on @@ -238,6 +245,9 @@ long afu_ioctl(struct file *file, unsigned int cmd, unsigned long arg) if (ctx->status == CLOSED) return -EIO; + if (!cxl_adapter_link_ok(ctx->afu->adapter)) + return -EIO; + pr_devel("afu_ioctl\n"); switch (cmd) { case CXL_IOCTL_START_WORK: @@ -251,7 +261,7 @@ long afu_ioctl(struct file *file, unsigned int cmd, unsigned long arg) return -EINVAL; } -long afu_compat_ioctl(struct file *file, unsigned int cmd, +static long afu_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { return afu_ioctl(file, cmd, arg); @@ -265,6 +275,9 @@ int afu_mmap(struct file *file, struct vm_area_struct *vm) if (ctx->status != STARTED) return -EIO; + if (!cxl_adapter_link_ok(ctx->afu->adapter)) + return -EIO; + return cxl_context_iomap(ctx, vm); } @@ -309,6 +322,9 @@ ssize_t afu_read(struct file *file, char __user *buf, size_t count, int rc; DEFINE_WAIT(wait); + if (!cxl_adapter_link_ok(ctx->afu->adapter)) + return -EIO; + if (count < CXL_READ_MIN_SIZE) return -EINVAL; @@ -319,6 +335,11 @@ ssize_t afu_read(struct file *file, char __user *buf, size_t count, if (ctx_event_pending(ctx)) break; + if (!cxl_adapter_link_ok(ctx->afu->adapter)) { + rc = -EIO; + goto out; + } + if (file->f_flags & O_NONBLOCK) { rc = -EAGAIN; goto out; @@ -396,7 +417,7 @@ const struct file_operations afu_fops = { .mmap = afu_mmap, }; -const struct file_operations afu_master_fops = { +static const struct file_operations afu_master_fops = { .owner = THIS_MODULE, .open = afu_master_open, .poll = afu_poll, @@ -519,7 +540,7 @@ int __init cxl_file_init(void) * If these change we really need to update API. Either change some * flags or update API version number CXL_API_VERSION. */ - BUILD_BUG_ON(CXL_API_VERSION != 1); + BUILD_BUG_ON(CXL_API_VERSION != 2); BUILD_BUG_ON(sizeof(struct cxl_ioctl_start_work) != 64); BUILD_BUG_ON(sizeof(struct cxl_event_header) != 8); BUILD_BUG_ON(sizeof(struct cxl_event_afu_interrupt) != 8); diff --git a/drivers/misc/cxl/irq.c b/drivers/misc/cxl/irq.c index 680cd263436db5..583b42afeda235 100644 --- a/drivers/misc/cxl/irq.c +++ b/drivers/misc/cxl/irq.c @@ -30,12 +30,12 @@ static irqreturn_t handle_psl_slice_error(struct cxl_context *ctx, u64 dsisr, u6 serr = cxl_p1n_read(ctx->afu, CXL_PSL_SERR_An); afu_debug = cxl_p1n_read(ctx->afu, CXL_AFU_DEBUG_An); - dev_crit(&ctx->afu->dev, "PSL ERROR STATUS: 0x%.16llx\n", errstat); - dev_crit(&ctx->afu->dev, "PSL_FIR1: 0x%.16llx\n", fir1); - dev_crit(&ctx->afu->dev, "PSL_FIR2: 0x%.16llx\n", fir2); - dev_crit(&ctx->afu->dev, "PSL_SERR_An: 0x%.16llx\n", serr); - dev_crit(&ctx->afu->dev, "PSL_FIR_SLICE_An: 0x%.16llx\n", fir_slice); - dev_crit(&ctx->afu->dev, "CXL_PSL_AFU_DEBUG_An: 0x%.16llx\n", afu_debug); + dev_crit(&ctx->afu->dev, "PSL ERROR STATUS: 0x%016llx\n", errstat); + dev_crit(&ctx->afu->dev, "PSL_FIR1: 0x%016llx\n", fir1); + dev_crit(&ctx->afu->dev, "PSL_FIR2: 0x%016llx\n", fir2); + dev_crit(&ctx->afu->dev, "PSL_SERR_An: 0x%016llx\n", serr); + dev_crit(&ctx->afu->dev, "PSL_FIR_SLICE_An: 0x%016llx\n", fir_slice); + dev_crit(&ctx->afu->dev, "CXL_PSL_AFU_DEBUG_An: 0x%016llx\n", afu_debug); dev_crit(&ctx->afu->dev, "STOPPING CXL TRACE\n"); cxl_stop_trace(ctx->afu->adapter); @@ -54,10 +54,10 @@ irqreturn_t cxl_slice_irq_err(int irq, void *data) fir_slice = cxl_p1n_read(afu, CXL_PSL_FIR_SLICE_An); errstat = cxl_p2n_read(afu, CXL_PSL_ErrStat_An); afu_debug = cxl_p1n_read(afu, CXL_AFU_DEBUG_An); - dev_crit(&afu->dev, "PSL_SERR_An: 0x%.16llx\n", serr); - dev_crit(&afu->dev, "PSL_FIR_SLICE_An: 0x%.16llx\n", fir_slice); - dev_crit(&afu->dev, "CXL_PSL_ErrStat_An: 0x%.16llx\n", errstat); - dev_crit(&afu->dev, "CXL_PSL_AFU_DEBUG_An: 0x%.16llx\n", afu_debug); + dev_crit(&afu->dev, "PSL_SERR_An: 0x%016llx\n", serr); + dev_crit(&afu->dev, "PSL_FIR_SLICE_An: 0x%016llx\n", fir_slice); + dev_crit(&afu->dev, "CXL_PSL_ErrStat_An: 0x%016llx\n", errstat); + dev_crit(&afu->dev, "CXL_PSL_AFU_DEBUG_An: 0x%016llx\n", afu_debug); cxl_p1n_write(afu, CXL_PSL_SERR_An, serr); @@ -72,7 +72,7 @@ static irqreturn_t cxl_irq_err(int irq, void *data) WARN(1, "CXL ERROR interrupt %i\n", irq); err_ivte = cxl_p1_read(adapter, CXL_PSL_ErrIVTE); - dev_crit(&adapter->dev, "PSL_ErrIVTE: 0x%.16llx\n", err_ivte); + dev_crit(&adapter->dev, "PSL_ErrIVTE: 0x%016llx\n", err_ivte); dev_crit(&adapter->dev, "STOPPING CXL TRACE\n"); cxl_stop_trace(adapter); @@ -80,7 +80,7 @@ static irqreturn_t cxl_irq_err(int irq, void *data) fir1 = cxl_p1_read(adapter, CXL_PSL_FIR1); fir2 = cxl_p1_read(adapter, CXL_PSL_FIR2); - dev_crit(&adapter->dev, "PSL_FIR1: 0x%.16llx\nPSL_FIR2: 0x%.16llx\n", fir1, fir2); + dev_crit(&adapter->dev, "PSL_FIR1: 0x%016llx\nPSL_FIR2: 0x%016llx\n", fir1, fir2); return IRQ_HANDLED; } @@ -147,7 +147,7 @@ static irqreturn_t cxl_irq(int irq, void *data, struct cxl_irq_info *irq_info) if (dsisr & CXL_PSL_DSISR_An_PE) return handle_psl_slice_error(ctx, dsisr, irq_info->errstat); if (dsisr & CXL_PSL_DSISR_An_AE) { - pr_devel("CXL interrupt: AFU Error %.llx\n", irq_info->afu_err); + pr_devel("CXL interrupt: AFU Error 0x%016llx\n", irq_info->afu_err); if (ctx->pending_afu_err) { /* @@ -158,7 +158,7 @@ static irqreturn_t cxl_irq(int irq, void *data, struct cxl_irq_info *irq_info) * probably best that we log them somewhere: */ dev_err_ratelimited(&ctx->afu->dev, "CXL AFU Error " - "undelivered to pe %i: %.llx\n", + "undelivered to pe %i: 0x%016llx\n", ctx->pe, irq_info->afu_err); } else { spin_lock(&ctx->lock); @@ -211,8 +211,8 @@ static irqreturn_t cxl_irq_multiplexed(int irq, void *data) } rcu_read_unlock(); - WARN(1, "Unable to demultiplex CXL PSL IRQ for PE %i DSISR %.16llx DAR" - " %.16llx\n(Possible AFU HW issue - was a term/remove acked" + WARN(1, "Unable to demultiplex CXL PSL IRQ for PE %i DSISR %016llx DAR" + " %016llx\n(Possible AFU HW issue - was a term/remove acked" " with outstanding transactions?)\n", ph, irq_info.dsisr, irq_info.dar); return fail_psl_irq(afu, &irq_info); @@ -341,6 +341,9 @@ int cxl_register_psl_err_irq(struct cxl *adapter) void cxl_release_psl_err_irq(struct cxl *adapter) { + if (adapter->err_virq != irq_find_mapping(NULL, adapter->err_hwirq)) + return; + cxl_p1_write(adapter, CXL_PSL_ErrIVTE, 0x0000000000000000); cxl_unmap_irq(adapter->err_virq, adapter); cxl_release_one_irq(adapter, adapter->err_hwirq); @@ -374,6 +377,9 @@ int cxl_register_serr_irq(struct cxl_afu *afu) void cxl_release_serr_irq(struct cxl_afu *afu) { + if (afu->serr_virq != irq_find_mapping(NULL, afu->serr_hwirq)) + return; + cxl_p1n_write(afu, CXL_PSL_SERR_An, 0x0000000000000000); cxl_unmap_irq(afu->serr_virq, afu); cxl_release_one_irq(afu->adapter, afu->serr_hwirq); @@ -400,12 +406,15 @@ int cxl_register_psl_irq(struct cxl_afu *afu) void cxl_release_psl_irq(struct cxl_afu *afu) { + if (afu->psl_virq != irq_find_mapping(NULL, afu->psl_hwirq)) + return; + cxl_unmap_irq(afu->psl_virq, afu); cxl_release_one_irq(afu->adapter, afu->psl_hwirq); kfree(afu->psl_irq_name); } -void afu_irq_name_free(struct cxl_context *ctx) +static void afu_irq_name_free(struct cxl_context *ctx) { struct cxl_irq_name *irq_name, *tmp; @@ -421,6 +430,9 @@ int afu_allocate_irqs(struct cxl_context *ctx, u32 count) int rc, r, i, j = 1; struct cxl_irq_name *irq_name; + /* Initialize the list head to hold irq names */ + INIT_LIST_HEAD(&ctx->irq_names); + if ((rc = cxl_alloc_irq_ranges(&ctx->irqs, ctx->afu->adapter, count))) return rc; @@ -432,13 +444,12 @@ int afu_allocate_irqs(struct cxl_context *ctx, u32 count) ctx->irq_bitmap = kcalloc(BITS_TO_LONGS(count), sizeof(*ctx->irq_bitmap), GFP_KERNEL); if (!ctx->irq_bitmap) - return -ENOMEM; + goto out; /* * Allocate names first. If any fail, bail out before allocating * actual hardware IRQs. */ - INIT_LIST_HEAD(&ctx->irq_names); for (r = 1; r < CXL_IRQ_RANGES; r++) { for (i = 0; i < ctx->irqs.range[r]; i++) { irq_name = kmalloc(sizeof(struct cxl_irq_name), @@ -460,11 +471,12 @@ int afu_allocate_irqs(struct cxl_context *ctx, u32 count) return 0; out: + cxl_release_irq_ranges(&ctx->irqs, ctx->afu->adapter); afu_irq_name_free(ctx); return -ENOMEM; } -void afu_register_hwirqs(struct cxl_context *ctx) +static void afu_register_hwirqs(struct cxl_context *ctx) { irq_hw_number_t hwirq; struct cxl_irq_name *irq_name; @@ -511,4 +523,8 @@ void afu_release_irqs(struct cxl_context *ctx, void *cookie) afu_irq_name_free(ctx); cxl_release_irq_ranges(&ctx->irqs, ctx->afu->adapter); + + kfree(ctx->irq_bitmap); + ctx->irq_bitmap = NULL; + ctx->irq_count = 0; } diff --git a/drivers/misc/cxl/main.c b/drivers/misc/cxl/main.c index 4a164ab8b35a65..9fde75ed4facd5 100644 --- a/drivers/misc/cxl/main.c +++ b/drivers/misc/cxl/main.c @@ -222,6 +222,7 @@ static void exit_cxl(void) cxl_debugfs_exit(); cxl_file_exit(); unregister_cxl_calls(&cxl_calls); + idr_destroy(&cxl_adapter_idr); } module_init(init_cxl); diff --git a/drivers/misc/cxl/native.c b/drivers/misc/cxl/native.c index 10567f245818b7..b37f2e8004f5bc 100644 --- a/drivers/misc/cxl/native.c +++ b/drivers/misc/cxl/native.c @@ -41,7 +41,14 @@ static int afu_control(struct cxl_afu *afu, u64 command, rc = -EBUSY; goto out; } - pr_devel_ratelimited("AFU control... (0x%.16llx)\n", + + if (!cxl_adapter_link_ok(afu->adapter)) { + afu->enabled = enabled; + rc = -EIO; + goto out; + } + + pr_devel_ratelimited("AFU control... (0x%016llx)\n", AFU_Cntl | command); cpu_relax(); AFU_Cntl = cxl_p2n_read(afu, CXL_AFU_Cntl_An); @@ -85,6 +92,10 @@ int __cxl_afu_reset(struct cxl_afu *afu) int cxl_afu_check_and_enable(struct cxl_afu *afu) { + if (!cxl_adapter_link_ok(afu->adapter)) { + WARN(1, "Refusing to enable afu while link down!\n"); + return -EIO; + } if (afu->enabled) return 0; return afu_enable(afu); @@ -103,6 +114,12 @@ int cxl_psl_purge(struct cxl_afu *afu) pr_devel("PSL purge request\n"); + if (!cxl_adapter_link_ok(afu->adapter)) { + dev_warn(&afu->dev, "PSL Purge called with link down, ignoring\n"); + rc = -EIO; + goto out; + } + if ((AFU_Cntl & CXL_AFU_Cntl_An_ES_MASK) != CXL_AFU_Cntl_An_ES_Disabled) { WARN(1, "psl_purge request while AFU not disabled!\n"); cxl_afu_disable(afu); @@ -119,14 +136,19 @@ int cxl_psl_purge(struct cxl_afu *afu) rc = -EBUSY; goto out; } + if (!cxl_adapter_link_ok(afu->adapter)) { + rc = -EIO; + goto out; + } + dsisr = cxl_p2n_read(afu, CXL_PSL_DSISR_An); - pr_devel_ratelimited("PSL purging... PSL_CNTL: 0x%.16llx PSL_DSISR: 0x%.16llx\n", PSL_CNTL, dsisr); + pr_devel_ratelimited("PSL purging... PSL_CNTL: 0x%016llx PSL_DSISR: 0x%016llx\n", PSL_CNTL, dsisr); if (dsisr & CXL_PSL_DSISR_TRANS) { dar = cxl_p2n_read(afu, CXL_PSL_DAR_An); - dev_notice(&afu->dev, "PSL purge terminating pending translation, DSISR: 0x%.16llx, DAR: 0x%.16llx\n", dsisr, dar); + dev_notice(&afu->dev, "PSL purge terminating pending translation, DSISR: 0x%016llx, DAR: 0x%016llx\n", dsisr, dar); cxl_p2n_write(afu, CXL_PSL_TFC_An, CXL_PSL_TFC_An_AE); } else if (dsisr) { - dev_notice(&afu->dev, "PSL purge acknowledging pending non-translation fault, DSISR: 0x%.16llx\n", dsisr); + dev_notice(&afu->dev, "PSL purge acknowledging pending non-translation fault, DSISR: 0x%016llx\n", dsisr); cxl_p2n_write(afu, CXL_PSL_TFC_An, CXL_PSL_TFC_An_A); } else { cpu_relax(); @@ -161,10 +183,8 @@ static int spa_max_procs(int spa_size) return ((spa_size / 8) - 96) / 17; } -static int alloc_spa(struct cxl_afu *afu) +int cxl_alloc_spa(struct cxl_afu *afu) { - u64 spap; - /* Work out how many pages to allocate */ afu->spa_order = 0; do { @@ -183,6 +203,13 @@ static int alloc_spa(struct cxl_afu *afu) pr_devel("spa pages: %i afu->spa_max_procs: %i afu->num_procs: %i\n", 1<spa_order, afu->spa_max_procs, afu->num_procs); + return 0; +} + +static void attach_spa(struct cxl_afu *afu) +{ + u64 spap; + afu->sw_command_status = (__be64 *)((char *)afu->spa + ((afu->spa_max_procs + 3) * 128)); @@ -191,14 +218,19 @@ static int alloc_spa(struct cxl_afu *afu) spap |= CXL_PSL_SPAP_V; pr_devel("cxl: SPA allocated at 0x%p. Max processes: %i, sw_command_status: 0x%p CXL_PSL_SPAP_An=0x%016llx\n", afu->spa, afu->spa_max_procs, afu->sw_command_status, spap); cxl_p1n_write(afu, CXL_PSL_SPAP_An, spap); - - return 0; } -static void release_spa(struct cxl_afu *afu) +static inline void detach_spa(struct cxl_afu *afu) { cxl_p1n_write(afu, CXL_PSL_SPAP_An, 0); - free_pages((unsigned long) afu->spa, afu->spa_order); +} + +void cxl_release_spa(struct cxl_afu *afu) +{ + if (afu->spa) { + free_pages((unsigned long) afu->spa, afu->spa_order); + afu->spa = NULL; + } } int cxl_tlb_slb_invalidate(struct cxl *adapter) @@ -215,6 +247,8 @@ int cxl_tlb_slb_invalidate(struct cxl *adapter) dev_warn(&adapter->dev, "WARNING: CXL adapter wide TLBIA timed out!\n"); return -EBUSY; } + if (!cxl_adapter_link_ok(adapter)) + return -EIO; cpu_relax(); } @@ -224,6 +258,8 @@ int cxl_tlb_slb_invalidate(struct cxl *adapter) dev_warn(&adapter->dev, "WARNING: CXL adapter wide SLBIA timed out!\n"); return -EBUSY; } + if (!cxl_adapter_link_ok(adapter)) + return -EIO; cpu_relax(); } return 0; @@ -240,6 +276,11 @@ int cxl_afu_slbia(struct cxl_afu *afu) dev_warn(&afu->dev, "WARNING: CXL AFU SLBIA timed out!\n"); return -EBUSY; } + /* If the adapter has gone down, we can assume that we + * will PERST it and that will invalidate everything. + */ + if (!cxl_adapter_link_ok(afu->adapter)) + return -EIO; cpu_relax(); } return 0; @@ -279,6 +320,8 @@ static void slb_invalid(struct cxl_context *ctx) cxl_p1_write(adapter, CXL_PSL_SLBIA, CXL_TLB_SLB_IQ_LPIDPID); while (1) { + if (!cxl_adapter_link_ok(adapter)) + break; slbia = cxl_p1_read(adapter, CXL_PSL_SLBIA); if (!(slbia & CXL_TLB_SLB_P)) break; @@ -308,6 +351,11 @@ static int do_process_element_cmd(struct cxl_context *ctx, rc = -EBUSY; goto out; } + if (!cxl_adapter_link_ok(ctx->afu->adapter)) { + dev_warn(&ctx->afu->dev, "WARNING: Device link down, aborting Process Element Command!\n"); + rc = -EIO; + goto out; + } state = be64_to_cpup(ctx->afu->sw_command_status); if (state == ~0ULL) { pr_err("cxl: Error adding process element to AFU\n"); @@ -355,8 +403,13 @@ static int terminate_process_element(struct cxl_context *ctx) mutex_lock(&ctx->afu->spa_mutex); pr_devel("%s Terminate pe: %i started\n", __func__, ctx->pe); - rc = do_process_element_cmd(ctx, CXL_SPA_SW_CMD_TERMINATE, - CXL_PE_SOFTWARE_STATE_V | CXL_PE_SOFTWARE_STATE_T); + /* We could be asked to terminate when the hw is down. That + * should always succeed: it's not running if the hw has gone + * away and is being reset. + */ + if (cxl_adapter_link_ok(ctx->afu->adapter)) + rc = do_process_element_cmd(ctx, CXL_SPA_SW_CMD_TERMINATE, + CXL_PE_SOFTWARE_STATE_V | CXL_PE_SOFTWARE_STATE_T); ctx->elem->software_state = 0; /* Remove Valid bit */ pr_devel("%s Terminate pe: %i finished\n", __func__, ctx->pe); mutex_unlock(&ctx->afu->spa_mutex); @@ -369,7 +422,14 @@ static int remove_process_element(struct cxl_context *ctx) mutex_lock(&ctx->afu->spa_mutex); pr_devel("%s Remove pe: %i started\n", __func__, ctx->pe); - if (!(rc = do_process_element_cmd(ctx, CXL_SPA_SW_CMD_REMOVE, 0))) + + /* We could be asked to remove when the hw is down. Again, if + * the hw is down, the PE is gone, so we succeed. + */ + if (cxl_adapter_link_ok(ctx->afu->adapter)) + rc = do_process_element_cmd(ctx, CXL_SPA_SW_CMD_REMOVE, 0); + + if (!rc) ctx->pe_inserted = false; slb_invalid(ctx); pr_devel("%s Remove pe: %i finished\n", __func__, ctx->pe); @@ -397,8 +457,11 @@ static int activate_afu_directed(struct cxl_afu *afu) dev_info(&afu->dev, "Activating AFU directed mode\n"); - if (alloc_spa(afu)) - return -ENOMEM; + if (afu->spa == NULL) { + if (cxl_alloc_spa(afu)) + return -ENOMEM; + } + attach_spa(afu); cxl_p1n_write(afu, CXL_PSL_SCNTL_An, CXL_PSL_SCNTL_An_PM_AFU); cxl_p1n_write(afu, CXL_PSL_AMOR_An, 0xFFFFFFFFFFFFFFFFULL); @@ -492,9 +555,7 @@ static int attach_afu_directed(struct cxl_context *ctx, u64 wed, u64 amr) if ((result = cxl_afu_check_and_enable(ctx->afu))) return result; - add_process_element(ctx); - - return 0; + return add_process_element(ctx); } static int deactivate_afu_directed(struct cxl_afu *afu) @@ -511,8 +572,6 @@ static int deactivate_afu_directed(struct cxl_afu *afu) cxl_afu_disable(afu); cxl_psl_purge(afu); - release_spa(afu); - return 0; } @@ -614,6 +673,11 @@ int cxl_afu_activate_mode(struct cxl_afu *afu, int mode) if (!(mode & afu->modes_supported)) return -EINVAL; + if (!cxl_adapter_link_ok(afu->adapter)) { + WARN(1, "Device link is down, refusing to activate!\n"); + return -EIO; + } + if (mode == CXL_MODE_DIRECTED) return activate_afu_directed(afu); if (mode == CXL_MODE_DEDICATED) @@ -624,6 +688,11 @@ int cxl_afu_activate_mode(struct cxl_afu *afu, int mode) int cxl_attach_process(struct cxl_context *ctx, bool kernel, u64 wed, u64 amr) { + if (!cxl_adapter_link_ok(ctx->afu->adapter)) { + WARN(1, "Device link is down, refusing to attach process!\n"); + return -EIO; + } + ctx->kernel = kernel; if (ctx->afu->current_mode == CXL_MODE_DIRECTED) return attach_afu_directed(ctx, wed, amr); @@ -668,6 +737,12 @@ int cxl_get_irq(struct cxl_afu *afu, struct cxl_irq_info *info) { u64 pidtid; + /* If the adapter has gone away, we can't get any meaningful + * information. + */ + if (!cxl_adapter_link_ok(afu->adapter)) + return -EIO; + info->dsisr = cxl_p2n_read(afu, CXL_PSL_DSISR_An); info->dar = cxl_p2n_read(afu, CXL_PSL_DAR_An); info->dsr = cxl_p2n_read(afu, CXL_PSL_DSR_An); @@ -684,7 +759,7 @@ static void recover_psl_err(struct cxl_afu *afu, u64 errstat) { u64 dsisr; - pr_devel("RECOVERING FROM PSL ERROR... (0x%.16llx)\n", errstat); + pr_devel("RECOVERING FROM PSL ERROR... (0x%016llx)\n", errstat); /* Clear PSL_DSISR[PE] */ dsisr = cxl_p2n_read(afu, CXL_PSL_DSISR_An); diff --git a/drivers/misc/cxl/pci.c b/drivers/misc/cxl/pci.c index 32ad097059498c..02c85160bfe9f6 100644 --- a/drivers/misc/cxl/pci.c +++ b/drivers/misc/cxl/pci.c @@ -24,6 +24,7 @@ #include #include "cxl.h" +#include #define CXL_PCI_VSEC_ID 0x1280 @@ -133,7 +134,7 @@ u8 cxl_afu_cr_read8(struct cxl_afu *afu, int cr, u64 off) return (val >> ((off & 0x3) * 8)) & 0xff; } -static DEFINE_PCI_DEVICE_TABLE(cxl_pci_tbl) = { +static const struct pci_device_id cxl_pci_tbl[] = { { PCI_DEVICE(PCI_VENDOR_ID_IBM, 0x0477), }, { PCI_DEVICE(PCI_VENDOR_ID_IBM, 0x044b), }, { PCI_DEVICE(PCI_VENDOR_ID_IBM, 0x04cf), }, @@ -369,6 +370,55 @@ static int init_implementation_adapter_regs(struct cxl *adapter, struct pci_dev return 0; } +#define TBSYNC_CNT(n) (((u64)n & 0x7) << (63-6)) +#define _2048_250MHZ_CYCLES 1 + +static int cxl_setup_psl_timebase(struct cxl *adapter, struct pci_dev *dev) +{ + u64 psl_tb; + int delta; + unsigned int retry = 0; + struct device_node *np; + + if (!(np = pnv_pci_get_phb_node(dev))) + return -ENODEV; + + /* Do not fail when CAPP timebase sync is not supported by OPAL */ + of_node_get(np); + if (! of_get_property(np, "ibm,capp-timebase-sync", NULL)) { + of_node_put(np); + pr_err("PSL: Timebase sync: OPAL support missing\n"); + return 0; + } + of_node_put(np); + + /* + * Setup PSL Timebase Control and Status register + * with the recommended Timebase Sync Count value + */ + cxl_p1_write(adapter, CXL_PSL_TB_CTLSTAT, + TBSYNC_CNT(2 * _2048_250MHZ_CYCLES)); + + /* Enable PSL Timebase */ + cxl_p1_write(adapter, CXL_PSL_Control, 0x0000000000000000); + cxl_p1_write(adapter, CXL_PSL_Control, CXL_PSL_Control_tb); + + /* Wait until CORE TB and PSL TB difference <= 16usecs */ + do { + msleep(1); + if (retry++ > 5) { + pr_err("PSL: Timebase sync: giving up!\n"); + return -EIO; + } + psl_tb = cxl_p1_read(adapter, CXL_PSL_Timebase); + delta = mftb() - psl_tb; + if (delta < 0) + delta = -delta; + } while (cputime_to_usecs(delta) > 16); + + return 0; +} + static int init_implementation_afu_regs(struct cxl_afu *afu) { /* read/write masks for this slice */ @@ -539,10 +589,18 @@ static int cxl_map_slice_regs(struct cxl_afu *afu, struct cxl *adapter, struct p static void cxl_unmap_slice_regs(struct cxl_afu *afu) { - if (afu->p2n_mmio) + if (afu->p2n_mmio) { iounmap(afu->p2n_mmio); - if (afu->p1n_mmio) + afu->p2n_mmio = NULL; + } + if (afu->p1n_mmio) { iounmap(afu->p1n_mmio); + afu->p1n_mmio = NULL; + } + if (afu->afu_desc_mmio) { + iounmap(afu->afu_desc_mmio); + afu->afu_desc_mmio = NULL; + } } static void cxl_release_afu(struct device *dev) @@ -551,6 +609,9 @@ static void cxl_release_afu(struct device *dev) pr_devel("cxl_release_afu\n"); + idr_destroy(&afu->contexts_idr); + cxl_release_spa(afu); + kfree(afu); } @@ -656,7 +717,7 @@ static int sanitise_afu_regs(struct cxl_afu *afu) */ reg = cxl_p2n_read(afu, CXL_AFU_Cntl_An); if ((reg & CXL_AFU_Cntl_An_ES_MASK) != CXL_AFU_Cntl_An_ES_Disabled) { - dev_warn(&afu->dev, "WARNING: AFU was not disabled: %#.16llx\n", reg); + dev_warn(&afu->dev, "WARNING: AFU was not disabled: %#016llx\n", reg); if (__cxl_afu_reset(afu)) return -EIO; if (cxl_afu_disable(afu)) @@ -677,7 +738,7 @@ static int sanitise_afu_regs(struct cxl_afu *afu) cxl_p2n_write(afu, CXL_SSTP0_An, 0x0000000000000000); reg = cxl_p2n_read(afu, CXL_PSL_DSISR_An); if (reg) { - dev_warn(&afu->dev, "AFU had pending DSISR: %#.16llx\n", reg); + dev_warn(&afu->dev, "AFU had pending DSISR: %#016llx\n", reg); if (reg & CXL_PSL_DSISR_TRANS) cxl_p2n_write(afu, CXL_PSL_TFC_An, CXL_PSL_TFC_An_AE); else @@ -686,12 +747,12 @@ static int sanitise_afu_regs(struct cxl_afu *afu) reg = cxl_p1n_read(afu, CXL_PSL_SERR_An); if (reg) { if (reg & ~0xffff) - dev_warn(&afu->dev, "AFU had pending SERR: %#.16llx\n", reg); + dev_warn(&afu->dev, "AFU had pending SERR: %#016llx\n", reg); cxl_p1n_write(afu, CXL_PSL_SERR_An, reg & ~0xffff); } reg = cxl_p2n_read(afu, CXL_PSL_ErrStat_An); if (reg) { - dev_warn(&afu->dev, "AFU had pending error status: %#.16llx\n", reg); + dev_warn(&afu->dev, "AFU had pending error status: %#016llx\n", reg); cxl_p2n_write(afu, CXL_PSL_ErrStat_An, reg); } @@ -742,45 +803,70 @@ ssize_t cxl_afu_read_err_buffer(struct cxl_afu *afu, char *buf, return count; } -static int cxl_init_afu(struct cxl *adapter, int slice, struct pci_dev *dev) +static int cxl_configure_afu(struct cxl_afu *afu, struct cxl *adapter, struct pci_dev *dev) { - struct cxl_afu *afu; - bool free = true; int rc; - if (!(afu = cxl_alloc_afu(adapter, slice))) - return -ENOMEM; - - if ((rc = dev_set_name(&afu->dev, "afu%i.%i", adapter->adapter_num, slice))) - goto err1; - if ((rc = cxl_map_slice_regs(afu, adapter, dev))) - goto err1; + return rc; if ((rc = sanitise_afu_regs(afu))) - goto err2; + goto err1; /* We need to reset the AFU before we can read the AFU descriptor */ if ((rc = __cxl_afu_reset(afu))) - goto err2; + goto err1; if (cxl_verbose) dump_afu_descriptor(afu); if ((rc = cxl_read_afu_descriptor(afu))) - goto err2; + goto err1; if ((rc = cxl_afu_descriptor_looks_ok(afu))) - goto err2; + goto err1; if ((rc = init_implementation_afu_regs(afu))) - goto err2; + goto err1; if ((rc = cxl_register_serr_irq(afu))) - goto err2; + goto err1; if ((rc = cxl_register_psl_irq(afu))) - goto err3; + goto err2; + + return 0; + +err2: + cxl_release_serr_irq(afu); +err1: + cxl_unmap_slice_regs(afu); + return rc; +} + +static void cxl_deconfigure_afu(struct cxl_afu *afu) +{ + cxl_release_psl_irq(afu); + cxl_release_serr_irq(afu); + cxl_unmap_slice_regs(afu); +} + +static int cxl_init_afu(struct cxl *adapter, int slice, struct pci_dev *dev) +{ + struct cxl_afu *afu; + int rc; + + afu = cxl_alloc_afu(adapter, slice); + if (!afu) + return -ENOMEM; + + rc = dev_set_name(&afu->dev, "afu%i.%i", adapter->adapter_num, slice); + if (rc) + goto err_free; + + rc = cxl_configure_afu(afu, adapter, dev); + if (rc) + goto err_free; /* Don't care if this fails */ cxl_debugfs_afu_add(afu); @@ -795,10 +881,6 @@ static int cxl_init_afu(struct cxl *adapter, int slice, struct pci_dev *dev) if ((rc = cxl_sysfs_afu_add(afu))) goto err_put1; - - if ((rc = cxl_afu_select_best_mode(afu))) - goto err_put2; - adapter->afu[afu->slice] = afu; if ((rc = cxl_pci_vphb_add(afu))) @@ -806,21 +888,16 @@ static int cxl_init_afu(struct cxl *adapter, int slice, struct pci_dev *dev) return 0; -err_put2: - cxl_sysfs_afu_remove(afu); err_put1: - device_unregister(&afu->dev); - free = false; + cxl_deconfigure_afu(afu); cxl_debugfs_afu_remove(afu); - cxl_release_psl_irq(afu); -err3: - cxl_release_serr_irq(afu); -err2: - cxl_unmap_slice_regs(afu); -err1: - if (free) - kfree(afu); + device_unregister(&afu->dev); + return rc; + +err_free: + kfree(afu); return rc; + } static void cxl_remove_afu(struct cxl_afu *afu) @@ -840,10 +917,7 @@ static void cxl_remove_afu(struct cxl_afu *afu) cxl_context_detach_all(afu); cxl_afu_deactivate_mode(afu); - cxl_release_psl_irq(afu); - cxl_release_serr_irq(afu); - cxl_unmap_slice_regs(afu); - + cxl_deconfigure_afu(afu); device_unregister(&afu->dev); } @@ -851,16 +925,15 @@ int cxl_reset(struct cxl *adapter) { struct pci_dev *dev = to_pci_dev(adapter->dev.parent); int rc; - int i; - u32 val; - - dev_info(&dev->dev, "CXL reset\n"); - for (i = 0; i < adapter->slices; i++) { - cxl_pci_vphb_remove(adapter->afu[i]); - cxl_remove_afu(adapter->afu[i]); + if (adapter->perst_same_image) { + dev_warn(&dev->dev, + "cxl: refusing to reset/reflash when perst_reloads_same_image is set.\n"); + return -EINVAL; } + dev_info(&dev->dev, "CXL reset\n"); + /* pcie_warm_reset requests a fundamental pci reset which includes a * PERST assert/deassert. PERST triggers a loading of the image * if "user" or "factory" is selected in sysfs */ @@ -869,20 +942,6 @@ int cxl_reset(struct cxl *adapter) return rc; } - /* the PERST done above fences the PHB. So, reset depends on EEH - * to unbind the driver, tell Sapphire to reinit the PHB, and rebind - * the driver. Do an mmio read explictly to ensure EEH notices the - * fenced PHB. Retry for a few seconds before giving up. */ - i = 0; - while (((val = mmio_read32be(adapter->p1_mmio)) != 0xffffffff) && - (i < 5)) { - msleep(500); - i++; - } - - if (val != 0xffffffff) - dev_err(&dev->dev, "cxl: PERST failed to trigger EEH\n"); - return rc; } @@ -893,7 +952,7 @@ static int cxl_map_adapter_regs(struct cxl *adapter, struct pci_dev *dev) if (pci_request_region(dev, 0, "priv 1 regs")) goto err2; - pr_devel("cxl_map_adapter_regs: p1: %#.16llx %#llx, p2: %#.16llx %#llx", + pr_devel("cxl_map_adapter_regs: p1: %#016llx %#llx, p2: %#016llx %#llx", p1_base(dev), p1_size(dev), p2_base(dev), p2_size(dev)); if (!(adapter->p1_mmio = ioremap(p1_base(dev), p1_size(dev)))) @@ -917,10 +976,16 @@ static int cxl_map_adapter_regs(struct cxl *adapter, struct pci_dev *dev) static void cxl_unmap_adapter_regs(struct cxl *adapter) { - if (adapter->p1_mmio) + if (adapter->p1_mmio) { iounmap(adapter->p1_mmio); - if (adapter->p2_mmio) + adapter->p1_mmio = NULL; + pci_release_region(to_pci_dev(adapter->dev.parent), 2); + } + if (adapter->p2_mmio) { iounmap(adapter->p2_mmio); + adapter->p2_mmio = NULL; + pci_release_region(to_pci_dev(adapter->dev.parent), 0); + } } static int cxl_read_vsec(struct cxl *adapter, struct pci_dev *dev) @@ -949,7 +1014,6 @@ static int cxl_read_vsec(struct cxl *adapter, struct pci_dev *dev) CXL_READ_VSEC_BASE_IMAGE(dev, vsec, &adapter->base_image); CXL_READ_VSEC_IMAGE_STATE(dev, vsec, &image_state); adapter->user_image_loaded = !!(image_state & CXL_VSEC_USER_IMAGE_LOADED); - adapter->perst_loads_image = true; adapter->perst_select_user = !!(image_state & CXL_VSEC_USER_IMAGE_LOADED); CXL_READ_VSEC_NAFUS(dev, vsec, &adapter->slices); @@ -1009,81 +1073,138 @@ static void cxl_release_adapter(struct device *dev) pr_devel("cxl_release_adapter\n"); + cxl_remove_adapter_nr(adapter); + kfree(adapter); } -static struct cxl *cxl_alloc_adapter(struct pci_dev *dev) +static struct cxl *cxl_alloc_adapter(void) { struct cxl *adapter; if (!(adapter = kzalloc(sizeof(struct cxl), GFP_KERNEL))) return NULL; - adapter->dev.parent = &dev->dev; - adapter->dev.release = cxl_release_adapter; - pci_set_drvdata(dev, adapter); spin_lock_init(&adapter->afu_list_lock); + if (cxl_alloc_adapter_nr(adapter)) + goto err1; + + if (dev_set_name(&adapter->dev, "card%i", adapter->adapter_num)) + goto err2; + return adapter; + +err2: + cxl_remove_adapter_nr(adapter); +err1: + kfree(adapter); + return NULL; } +#define CXL_PSL_ErrIVTE_tberror (0x1ull << (63-31)) + static int sanitise_adapter_regs(struct cxl *adapter) { - cxl_p1_write(adapter, CXL_PSL_ErrIVTE, 0x0000000000000000); + /* Clear PSL tberror bit by writing 1 to it */ + cxl_p1_write(adapter, CXL_PSL_ErrIVTE, CXL_PSL_ErrIVTE_tberror); return cxl_tlb_slb_invalidate(adapter); } -static struct cxl *cxl_init_adapter(struct pci_dev *dev) +/* This should contain *only* operations that can safely be done in + * both creation and recovery. + */ +static int cxl_configure_adapter(struct cxl *adapter, struct pci_dev *dev) { - struct cxl *adapter; - bool free = true; int rc; + adapter->dev.parent = &dev->dev; + adapter->dev.release = cxl_release_adapter; + pci_set_drvdata(dev, adapter); - if (!(adapter = cxl_alloc_adapter(dev))) - return ERR_PTR(-ENOMEM); + rc = pci_enable_device(dev); + if (rc) { + dev_err(&dev->dev, "pci_enable_device failed: %i\n", rc); + return rc; + } if ((rc = cxl_read_vsec(adapter, dev))) - goto err1; + return rc; if ((rc = cxl_vsec_looks_ok(adapter, dev))) - goto err1; + return rc; if ((rc = setup_cxl_bars(dev))) - goto err1; + return rc; if ((rc = switch_card_to_cxl(dev))) - goto err1; - - if ((rc = cxl_alloc_adapter_nr(adapter))) - goto err1; - - if ((rc = dev_set_name(&adapter->dev, "card%i", adapter->adapter_num))) - goto err2; + return rc; if ((rc = cxl_update_image_control(adapter))) - goto err2; + return rc; if ((rc = cxl_map_adapter_regs(adapter, dev))) - goto err2; + return rc; if ((rc = sanitise_adapter_regs(adapter))) - goto err2; + goto err; if ((rc = init_implementation_adapter_regs(adapter, dev))) - goto err3; + goto err; if ((rc = pnv_phb_to_cxl_mode(dev, OPAL_PHB_CAPI_MODE_CAPI))) - goto err3; + goto err; /* If recovery happened, the last step is to turn on snooping. * In the non-recovery case this has no effect */ - if ((rc = pnv_phb_to_cxl_mode(dev, OPAL_PHB_CAPI_MODE_SNOOP_ON))) { - goto err3; - } + if ((rc = pnv_phb_to_cxl_mode(dev, OPAL_PHB_CAPI_MODE_SNOOP_ON))) + goto err; + + if ((rc = cxl_setup_psl_timebase(adapter, dev))) + goto err; if ((rc = cxl_register_psl_err_irq(adapter))) - goto err3; + goto err; + + return 0; + +err: + cxl_unmap_adapter_regs(adapter); + return rc; + +} + +static void cxl_deconfigure_adapter(struct cxl *adapter) +{ + struct pci_dev *pdev = to_pci_dev(adapter->dev.parent); + + cxl_release_psl_err_irq(adapter); + cxl_unmap_adapter_regs(adapter); + + pci_disable_device(pdev); +} + +static struct cxl *cxl_init_adapter(struct pci_dev *dev) +{ + struct cxl *adapter; + int rc; + + adapter = cxl_alloc_adapter(); + if (!adapter) + return ERR_PTR(-ENOMEM); + + /* Set defaults for parameters which need to persist over + * configure/reconfigure + */ + adapter->perst_loads_image = true; + adapter->perst_same_image = false; + + rc = cxl_configure_adapter(adapter, dev); + if (rc) { + pci_disable_device(dev); + cxl_release_adapter(&adapter->dev); + return ERR_PTR(rc); + } /* Don't care if this one fails: */ cxl_debugfs_adapter_add(adapter); @@ -1101,37 +1222,25 @@ static struct cxl *cxl_init_adapter(struct pci_dev *dev) return adapter; err_put1: - device_unregister(&adapter->dev); - free = false; + /* This should mirror cxl_remove_adapter, except without the + * sysfs parts + */ cxl_debugfs_adapter_remove(adapter); - cxl_release_psl_err_irq(adapter); -err3: - cxl_unmap_adapter_regs(adapter); -err2: - cxl_remove_adapter_nr(adapter); -err1: - if (free) - kfree(adapter); + cxl_deconfigure_adapter(adapter); + device_unregister(&adapter->dev); return ERR_PTR(rc); } static void cxl_remove_adapter(struct cxl *adapter) { - struct pci_dev *pdev = to_pci_dev(adapter->dev.parent); - - pr_devel("cxl_release_adapter\n"); + pr_devel("cxl_remove_adapter\n"); cxl_sysfs_adapter_remove(adapter); cxl_debugfs_adapter_remove(adapter); - cxl_release_psl_err_irq(adapter); - cxl_unmap_adapter_regs(adapter); - cxl_remove_adapter_nr(adapter); - device_unregister(&adapter->dev); + cxl_deconfigure_adapter(adapter); - pci_release_region(pdev, 0); - pci_release_region(pdev, 2); - pci_disable_device(pdev); + device_unregister(&adapter->dev); } static int cxl_probe(struct pci_dev *dev, const struct pci_device_id *id) @@ -1145,21 +1254,21 @@ static int cxl_probe(struct pci_dev *dev, const struct pci_device_id *id) if (cxl_verbose) dump_cxl_config_space(dev); - if ((rc = pci_enable_device(dev))) { - dev_err(&dev->dev, "pci_enable_device failed: %i\n", rc); - return rc; - } - adapter = cxl_init_adapter(dev); if (IS_ERR(adapter)) { dev_err(&dev->dev, "cxl_init_adapter failed: %li\n", PTR_ERR(adapter)); - pci_disable_device(dev); return PTR_ERR(adapter); } for (slice = 0; slice < adapter->slices; slice++) { - if ((rc = cxl_init_afu(adapter, slice, dev))) + if ((rc = cxl_init_afu(adapter, slice, dev))) { dev_err(&dev->dev, "AFU %i failed to initialise: %i\n", slice, rc); + continue; + } + + rc = cxl_afu_select_best_mode(adapter->afu[slice]); + if (rc) + dev_err(&dev->dev, "AFU %i failed to start: %i\n", slice, rc); } return 0; @@ -1183,10 +1292,262 @@ static void cxl_remove(struct pci_dev *dev) cxl_remove_adapter(adapter); } +static pci_ers_result_t cxl_vphb_error_detected(struct cxl_afu *afu, + pci_channel_state_t state) +{ + struct pci_dev *afu_dev; + pci_ers_result_t result = PCI_ERS_RESULT_NEED_RESET; + pci_ers_result_t afu_result = PCI_ERS_RESULT_NEED_RESET; + + /* There should only be one entry, but go through the list + * anyway + */ + list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) { + if (!afu_dev->driver) + continue; + + afu_dev->error_state = state; + + if (afu_dev->driver->err_handler) + afu_result = afu_dev->driver->err_handler->error_detected(afu_dev, + state); + /* Disconnect trumps all, NONE trumps NEED_RESET */ + if (afu_result == PCI_ERS_RESULT_DISCONNECT) + result = PCI_ERS_RESULT_DISCONNECT; + else if ((afu_result == PCI_ERS_RESULT_NONE) && + (result == PCI_ERS_RESULT_NEED_RESET)) + result = PCI_ERS_RESULT_NONE; + } + return result; +} + +static pci_ers_result_t cxl_pci_error_detected(struct pci_dev *pdev, + pci_channel_state_t state) +{ + struct cxl *adapter = pci_get_drvdata(pdev); + struct cxl_afu *afu; + pci_ers_result_t result = PCI_ERS_RESULT_NEED_RESET; + int i; + + /* At this point, we could still have an interrupt pending. + * Let's try to get them out of the way before they do + * anything we don't like. + */ + schedule(); + + /* If we're permanently dead, give up. */ + if (state == pci_channel_io_perm_failure) { + /* Tell the AFU drivers; but we don't care what they + * say, we're going away. + */ + for (i = 0; i < adapter->slices; i++) { + afu = adapter->afu[i]; + cxl_vphb_error_detected(afu, state); + } + return PCI_ERS_RESULT_DISCONNECT; + } + + /* Are we reflashing? + * + * If we reflash, we could come back as something entirely + * different, including a non-CAPI card. As such, by default + * we don't participate in the process. We'll be unbound and + * the slot re-probed. (TODO: check EEH doesn't blindly rebind + * us!) + * + * However, this isn't the entire story: for reliablity + * reasons, we usually want to reflash the FPGA on PERST in + * order to get back to a more reliable known-good state. + * + * This causes us a bit of a problem: if we reflash we can't + * trust that we'll come back the same - we could have a new + * image and been PERSTed in order to load that + * image. However, most of the time we actually *will* come + * back the same - for example a regular EEH event. + * + * Therefore, we allow the user to assert that the image is + * indeed the same and that we should continue on into EEH + * anyway. + */ + if (adapter->perst_loads_image && !adapter->perst_same_image) { + /* TODO take the PHB out of CXL mode */ + dev_info(&pdev->dev, "reflashing, so opting out of EEH!\n"); + return PCI_ERS_RESULT_NONE; + } + + /* + * At this point, we want to try to recover. We'll always + * need a complete slot reset: we don't trust any other reset. + * + * Now, we go through each AFU: + * - We send the driver, if bound, an error_detected callback. + * We expect it to clean up, but it can also tell us to give + * up and permanently detach the card. To simplify things, if + * any bound AFU driver doesn't support EEH, we give up on EEH. + * + * - We detach all contexts associated with the AFU. This + * does not free them, but puts them into a CLOSED state + * which causes any the associated files to return useful + * errors to userland. It also unmaps, but does not free, + * any IRQs. + * + * - We clean up our side: releasing and unmapping resources we hold + * so we can wire them up again when the hardware comes back up. + * + * Driver authors should note: + * + * - Any contexts you create in your kernel driver (except + * those associated with anonymous file descriptors) are + * your responsibility to free and recreate. Likewise with + * any attached resources. + * + * - We will take responsibility for re-initialising the + * device context (the one set up for you in + * cxl_pci_enable_device_hook and accessed through + * cxl_get_context). If you've attached IRQs or other + * resources to it, they remains yours to free. + * + * You can call the same functions to release resources as you + * normally would: we make sure that these functions continue + * to work when the hardware is down. + * + * Two examples: + * + * 1) If you normally free all your resources at the end of + * each request, or if you use anonymous FDs, your + * error_detected callback can simply set a flag to tell + * your driver not to start any new calls. You can then + * clear the flag in the resume callback. + * + * 2) If you normally allocate your resources on startup: + * * Set a flag in error_detected as above. + * * Let CXL detach your contexts. + * * In slot_reset, free the old resources and allocate new ones. + * * In resume, clear the flag to allow things to start. + */ + for (i = 0; i < adapter->slices; i++) { + afu = adapter->afu[i]; + + result = cxl_vphb_error_detected(afu, state); + + /* Only continue if everyone agrees on NEED_RESET */ + if (result != PCI_ERS_RESULT_NEED_RESET) + return result; + + cxl_context_detach_all(afu); + cxl_afu_deactivate_mode(afu); + cxl_deconfigure_afu(afu); + } + cxl_deconfigure_adapter(adapter); + + return result; +} + +static pci_ers_result_t cxl_pci_slot_reset(struct pci_dev *pdev) +{ + struct cxl *adapter = pci_get_drvdata(pdev); + struct cxl_afu *afu; + struct cxl_context *ctx; + struct pci_dev *afu_dev; + pci_ers_result_t afu_result = PCI_ERS_RESULT_RECOVERED; + pci_ers_result_t result = PCI_ERS_RESULT_RECOVERED; + int i; + + if (cxl_configure_adapter(adapter, pdev)) + goto err; + + for (i = 0; i < adapter->slices; i++) { + afu = adapter->afu[i]; + + if (cxl_configure_afu(afu, adapter, pdev)) + goto err; + + if (cxl_afu_select_best_mode(afu)) + goto err; + + cxl_pci_vphb_reconfigure(afu); + + list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) { + /* Reset the device context. + * TODO: make this less disruptive + */ + ctx = cxl_get_context(afu_dev); + + if (ctx && cxl_release_context(ctx)) + goto err; + + ctx = cxl_dev_context_init(afu_dev); + if (!ctx) + goto err; + + afu_dev->dev.archdata.cxl_ctx = ctx; + + if (cxl_afu_check_and_enable(afu)) + goto err; + + afu_dev->error_state = pci_channel_io_normal; + + /* If there's a driver attached, allow it to + * chime in on recovery. Drivers should check + * if everything has come back OK, but + * shouldn't start new work until we call + * their resume function. + */ + if (!afu_dev->driver) + continue; + + if (afu_dev->driver->err_handler && + afu_dev->driver->err_handler->slot_reset) + afu_result = afu_dev->driver->err_handler->slot_reset(afu_dev); + + if (afu_result == PCI_ERS_RESULT_DISCONNECT) + result = PCI_ERS_RESULT_DISCONNECT; + } + } + return result; + +err: + /* All the bits that happen in both error_detected and cxl_remove + * should be idempotent, so we don't need to worry about leaving a mix + * of unconfigured and reconfigured resources. + */ + dev_err(&pdev->dev, "EEH recovery failed. Asking to be disconnected.\n"); + return PCI_ERS_RESULT_DISCONNECT; +} + +static void cxl_pci_resume(struct pci_dev *pdev) +{ + struct cxl *adapter = pci_get_drvdata(pdev); + struct cxl_afu *afu; + struct pci_dev *afu_dev; + int i; + + /* Everything is back now. Drivers should restart work now. + * This is not the place to be checking if everything came back up + * properly, because there's no return value: do that in slot_reset. + */ + for (i = 0; i < adapter->slices; i++) { + afu = adapter->afu[i]; + + list_for_each_entry(afu_dev, &afu->phb->bus->devices, bus_list) { + if (afu_dev->driver && afu_dev->driver->err_handler && + afu_dev->driver->err_handler->resume) + afu_dev->driver->err_handler->resume(afu_dev); + } + } +} + +static const struct pci_error_handlers cxl_err_handler = { + .error_detected = cxl_pci_error_detected, + .slot_reset = cxl_pci_slot_reset, + .resume = cxl_pci_resume, +}; + struct pci_driver cxl_pci_driver = { .name = "cxl-pci", .id_table = cxl_pci_tbl, .probe = cxl_probe, .remove = cxl_remove, .shutdown = cxl_remove, + .err_handler = &cxl_err_handler, }; diff --git a/drivers/misc/cxl/sysfs.c b/drivers/misc/cxl/sysfs.c index 87cd747bb511c8..25868c2ec03ea7 100644 --- a/drivers/misc/cxl/sysfs.c +++ b/drivers/misc/cxl/sysfs.c @@ -112,12 +112,38 @@ static ssize_t load_image_on_perst_store(struct device *device, return count; } +static ssize_t perst_reloads_same_image_show(struct device *device, + struct device_attribute *attr, + char *buf) +{ + struct cxl *adapter = to_cxl_adapter(device); + + return scnprintf(buf, PAGE_SIZE, "%i\n", adapter->perst_same_image); +} + +static ssize_t perst_reloads_same_image_store(struct device *device, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct cxl *adapter = to_cxl_adapter(device); + int rc; + int val; + + rc = sscanf(buf, "%i", &val); + if ((rc != 1) || !(val == 1 || val == 0)) + return -EINVAL; + + adapter->perst_same_image = (val == 1 ? true : false); + return count; +} + static struct device_attribute adapter_attrs[] = { __ATTR_RO(caia_version), __ATTR_RO(psl_revision), __ATTR_RO(base_image), __ATTR_RO(image_loaded), __ATTR_RW(load_image_on_perst), + __ATTR_RW(perst_reloads_same_image), __ATTR(reset, S_IWUSR, NULL, reset_adapter_store), }; diff --git a/drivers/misc/cxl/trace.h b/drivers/misc/cxl/trace.h index ae434d87887e86..6e1e2adfba8ec1 100644 --- a/drivers/misc/cxl/trace.h +++ b/drivers/misc/cxl/trace.h @@ -105,7 +105,7 @@ TRACE_EVENT(cxl_attach, __entry->num_interrupts = num_interrupts; ), - TP_printk("afu%i.%i pid=%i pe=%i wed=0x%.16llx irqs=%i amr=0x%llx", + TP_printk("afu%i.%i pid=%i pe=%i wed=0x%016llx irqs=%i amr=0x%llx", __entry->card, __entry->afu, __entry->pid, @@ -177,7 +177,7 @@ TRACE_EVENT(cxl_psl_irq, __entry->dar = dar; ), - TP_printk("afu%i.%i pe=%i irq=%i dsisr=%s dar=0x%.16llx", + TP_printk("afu%i.%i pe=%i irq=%i dsisr=%s dar=0x%016llx", __entry->card, __entry->afu, __entry->pe, @@ -233,7 +233,7 @@ TRACE_EVENT(cxl_ste_miss, __entry->dar = dar; ), - TP_printk("afu%i.%i pe=%i dar=0x%.16llx", + TP_printk("afu%i.%i pe=%i dar=0x%016llx", __entry->card, __entry->afu, __entry->pe, @@ -264,7 +264,7 @@ TRACE_EVENT(cxl_ste_write, __entry->v = v; ), - TP_printk("afu%i.%i pe=%i SSTE[%i] E=0x%.16llx V=0x%.16llx", + TP_printk("afu%i.%i pe=%i SSTE[%i] E=0x%016llx V=0x%016llx", __entry->card, __entry->afu, __entry->pe, @@ -295,7 +295,7 @@ TRACE_EVENT(cxl_pte_miss, __entry->dar = dar; ), - TP_printk("afu%i.%i pe=%i dsisr=%s dar=0x%.16llx", + TP_printk("afu%i.%i pe=%i dsisr=%s dar=0x%016llx", __entry->card, __entry->afu, __entry->pe, diff --git a/drivers/misc/cxl/vphb.c b/drivers/misc/cxl/vphb.c index 2eba002b580b4c..6dd16a6d153f0e 100644 --- a/drivers/misc/cxl/vphb.c +++ b/drivers/misc/cxl/vphb.c @@ -138,6 +138,26 @@ static int cxl_pcie_config_info(struct pci_bus *bus, unsigned int devfn, return 0; } + +static inline bool cxl_config_link_ok(struct pci_bus *bus) +{ + struct pci_controller *phb; + struct cxl_afu *afu; + + /* Config space IO is based on phb->cfg_addr, which is based on + * afu_desc_mmio. This isn't safe to read/write when the link + * goes down, as EEH tears down MMIO space. + * + * Check if the link is OK before proceeding. + */ + + phb = pci_bus_to_host(bus); + if (phb == NULL) + return false; + afu = (struct cxl_afu *)phb->private_data; + return cxl_adapter_link_ok(afu->adapter); +} + static int cxl_pcie_read_config(struct pci_bus *bus, unsigned int devfn, int offset, int len, u32 *val) { @@ -150,6 +170,9 @@ static int cxl_pcie_read_config(struct pci_bus *bus, unsigned int devfn, if (rc) return rc; + if (!cxl_config_link_ok(bus)) + return PCIBIOS_DEVICE_NOT_FOUND; + /* Can only read 32 bits */ *val = (in_le32(ioaddr) >> shift) & mask; return PCIBIOS_SUCCESSFUL; @@ -167,6 +190,9 @@ static int cxl_pcie_write_config(struct pci_bus *bus, unsigned int devfn, if (rc) return rc; + if (!cxl_config_link_ok(bus)) + return PCIBIOS_DEVICE_NOT_FOUND; + /* Can only write 32 bits so do read-modify-write */ mask <<= shift; val <<= shift; @@ -240,6 +266,14 @@ int cxl_pci_vphb_add(struct cxl_afu *afu) return 0; } +void cxl_pci_vphb_reconfigure(struct cxl_afu *afu) +{ + /* When we are reconfigured, the AFU's MMIO space is unmapped + * and remapped. We need to reflect this in the PHB's view of + * the world. + */ + afu->phb->cfg_addr = afu->afu_desc_mmio + afu->crs_offset; +} void cxl_pci_vphb_remove(struct cxl_afu *afu) { diff --git a/drivers/mtd/nand/fsl_ifc_nand.c b/drivers/mtd/nand/fsl_ifc_nand.c index 51394e59901bbc..a4e27e8911531e 100644 --- a/drivers/mtd/nand/fsl_ifc_nand.c +++ b/drivers/mtd/nand/fsl_ifc_nand.c @@ -238,8 +238,8 @@ static void set_addr(struct mtd_info *mtd, int column, int page_addr, int oob) ifc_nand_ctrl->page = page_addr; /* Program ROW0/COL0 */ - iowrite32be(page_addr, &ifc->ifc_nand.row0); - iowrite32be((oob ? IFC_NAND_COL_MS : 0) | column, &ifc->ifc_nand.col0); + ifc_out32(page_addr, &ifc->ifc_nand.row0); + ifc_out32((oob ? IFC_NAND_COL_MS : 0) | column, &ifc->ifc_nand.col0); buf_num = page_addr & priv->bufnum_mask; @@ -301,19 +301,19 @@ static void fsl_ifc_run_command(struct mtd_info *mtd) int i; /* set the chip select for NAND Transaction */ - iowrite32be(priv->bank << IFC_NAND_CSEL_SHIFT, - &ifc->ifc_nand.nand_csel); + ifc_out32(priv->bank << IFC_NAND_CSEL_SHIFT, + &ifc->ifc_nand.nand_csel); dev_vdbg(priv->dev, "%s: fir0=%08x fcr0=%08x\n", __func__, - ioread32be(&ifc->ifc_nand.nand_fir0), - ioread32be(&ifc->ifc_nand.nand_fcr0)); + ifc_in32(&ifc->ifc_nand.nand_fir0), + ifc_in32(&ifc->ifc_nand.nand_fcr0)); ctrl->nand_stat = 0; /* start read/write seq */ - iowrite32be(IFC_NAND_SEQ_STRT_FIR_STRT, &ifc->ifc_nand.nandseq_strt); + ifc_out32(IFC_NAND_SEQ_STRT_FIR_STRT, &ifc->ifc_nand.nandseq_strt); /* wait for command complete flag or timeout */ wait_event_timeout(ctrl->nand_wait, ctrl->nand_stat, @@ -336,7 +336,7 @@ static void fsl_ifc_run_command(struct mtd_info *mtd) int sector_end = sector + chip->ecc.steps - 1; for (i = sector / 4; i <= sector_end / 4; i++) - eccstat[i] = ioread32be(&ifc->ifc_nand.nand_eccstat[i]); + eccstat[i] = ifc_in32(&ifc->ifc_nand.nand_eccstat[i]); for (i = sector; i <= sector_end; i++) { errors = check_read_ecc(mtd, ctrl, eccstat, i); @@ -376,33 +376,33 @@ static void fsl_ifc_do_read(struct nand_chip *chip, /* Program FIR/IFC_NAND_FCR0 for Small/Large page */ if (mtd->writesize > 512) { - iowrite32be((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | - (IFC_FIR_OP_CA0 << IFC_NAND_FIR0_OP1_SHIFT) | - (IFC_FIR_OP_RA0 << IFC_NAND_FIR0_OP2_SHIFT) | - (IFC_FIR_OP_CMD1 << IFC_NAND_FIR0_OP3_SHIFT) | - (IFC_FIR_OP_RBCD << IFC_NAND_FIR0_OP4_SHIFT), - &ifc->ifc_nand.nand_fir0); - iowrite32be(0x0, &ifc->ifc_nand.nand_fir1); - - iowrite32be((NAND_CMD_READ0 << IFC_NAND_FCR0_CMD0_SHIFT) | - (NAND_CMD_READSTART << IFC_NAND_FCR0_CMD1_SHIFT), - &ifc->ifc_nand.nand_fcr0); + ifc_out32((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | + (IFC_FIR_OP_CA0 << IFC_NAND_FIR0_OP1_SHIFT) | + (IFC_FIR_OP_RA0 << IFC_NAND_FIR0_OP2_SHIFT) | + (IFC_FIR_OP_CMD1 << IFC_NAND_FIR0_OP3_SHIFT) | + (IFC_FIR_OP_RBCD << IFC_NAND_FIR0_OP4_SHIFT), + &ifc->ifc_nand.nand_fir0); + ifc_out32(0x0, &ifc->ifc_nand.nand_fir1); + + ifc_out32((NAND_CMD_READ0 << IFC_NAND_FCR0_CMD0_SHIFT) | + (NAND_CMD_READSTART << IFC_NAND_FCR0_CMD1_SHIFT), + &ifc->ifc_nand.nand_fcr0); } else { - iowrite32be((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | - (IFC_FIR_OP_CA0 << IFC_NAND_FIR0_OP1_SHIFT) | - (IFC_FIR_OP_RA0 << IFC_NAND_FIR0_OP2_SHIFT) | - (IFC_FIR_OP_RBCD << IFC_NAND_FIR0_OP3_SHIFT), - &ifc->ifc_nand.nand_fir0); - iowrite32be(0x0, &ifc->ifc_nand.nand_fir1); + ifc_out32((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | + (IFC_FIR_OP_CA0 << IFC_NAND_FIR0_OP1_SHIFT) | + (IFC_FIR_OP_RA0 << IFC_NAND_FIR0_OP2_SHIFT) | + (IFC_FIR_OP_RBCD << IFC_NAND_FIR0_OP3_SHIFT), + &ifc->ifc_nand.nand_fir0); + ifc_out32(0x0, &ifc->ifc_nand.nand_fir1); if (oob) - iowrite32be(NAND_CMD_READOOB << - IFC_NAND_FCR0_CMD0_SHIFT, - &ifc->ifc_nand.nand_fcr0); + ifc_out32(NAND_CMD_READOOB << + IFC_NAND_FCR0_CMD0_SHIFT, + &ifc->ifc_nand.nand_fcr0); else - iowrite32be(NAND_CMD_READ0 << - IFC_NAND_FCR0_CMD0_SHIFT, - &ifc->ifc_nand.nand_fcr0); + ifc_out32(NAND_CMD_READ0 << + IFC_NAND_FCR0_CMD0_SHIFT, + &ifc->ifc_nand.nand_fcr0); } } @@ -422,7 +422,7 @@ static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command, switch (command) { /* READ0 read the entire buffer to use hardware ECC. */ case NAND_CMD_READ0: - iowrite32be(0, &ifc->ifc_nand.nand_fbcr); + ifc_out32(0, &ifc->ifc_nand.nand_fbcr); set_addr(mtd, 0, page_addr, 0); ifc_nand_ctrl->read_bytes = mtd->writesize + mtd->oobsize; @@ -437,7 +437,7 @@ static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command, /* READOOB reads only the OOB because no ECC is performed. */ case NAND_CMD_READOOB: - iowrite32be(mtd->oobsize - column, &ifc->ifc_nand.nand_fbcr); + ifc_out32(mtd->oobsize - column, &ifc->ifc_nand.nand_fbcr); set_addr(mtd, column, page_addr, 1); ifc_nand_ctrl->read_bytes = mtd->writesize + mtd->oobsize; @@ -453,19 +453,19 @@ static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command, if (command == NAND_CMD_PARAM) timing = IFC_FIR_OP_RBCD; - iowrite32be((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | - (IFC_FIR_OP_UA << IFC_NAND_FIR0_OP1_SHIFT) | - (timing << IFC_NAND_FIR0_OP2_SHIFT), - &ifc->ifc_nand.nand_fir0); - iowrite32be(command << IFC_NAND_FCR0_CMD0_SHIFT, - &ifc->ifc_nand.nand_fcr0); - iowrite32be(column, &ifc->ifc_nand.row3); + ifc_out32((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | + (IFC_FIR_OP_UA << IFC_NAND_FIR0_OP1_SHIFT) | + (timing << IFC_NAND_FIR0_OP2_SHIFT), + &ifc->ifc_nand.nand_fir0); + ifc_out32(command << IFC_NAND_FCR0_CMD0_SHIFT, + &ifc->ifc_nand.nand_fcr0); + ifc_out32(column, &ifc->ifc_nand.row3); /* * although currently it's 8 bytes for READID, we always read * the maximum 256 bytes(for PARAM) */ - iowrite32be(256, &ifc->ifc_nand.nand_fbcr); + ifc_out32(256, &ifc->ifc_nand.nand_fbcr); ifc_nand_ctrl->read_bytes = 256; set_addr(mtd, 0, 0, 0); @@ -480,16 +480,16 @@ static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command, /* ERASE2 uses the block and page address from ERASE1 */ case NAND_CMD_ERASE2: - iowrite32be((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | - (IFC_FIR_OP_RA0 << IFC_NAND_FIR0_OP1_SHIFT) | - (IFC_FIR_OP_CMD1 << IFC_NAND_FIR0_OP2_SHIFT), - &ifc->ifc_nand.nand_fir0); + ifc_out32((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | + (IFC_FIR_OP_RA0 << IFC_NAND_FIR0_OP1_SHIFT) | + (IFC_FIR_OP_CMD1 << IFC_NAND_FIR0_OP2_SHIFT), + &ifc->ifc_nand.nand_fir0); - iowrite32be((NAND_CMD_ERASE1 << IFC_NAND_FCR0_CMD0_SHIFT) | - (NAND_CMD_ERASE2 << IFC_NAND_FCR0_CMD1_SHIFT), - &ifc->ifc_nand.nand_fcr0); + ifc_out32((NAND_CMD_ERASE1 << IFC_NAND_FCR0_CMD0_SHIFT) | + (NAND_CMD_ERASE2 << IFC_NAND_FCR0_CMD1_SHIFT), + &ifc->ifc_nand.nand_fcr0); - iowrite32be(0, &ifc->ifc_nand.nand_fbcr); + ifc_out32(0, &ifc->ifc_nand.nand_fbcr); ifc_nand_ctrl->read_bytes = 0; fsl_ifc_run_command(mtd); return; @@ -506,19 +506,18 @@ static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command, (NAND_CMD_STATUS << IFC_NAND_FCR0_CMD1_SHIFT) | (NAND_CMD_PAGEPROG << IFC_NAND_FCR0_CMD2_SHIFT); - iowrite32be( - (IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | - (IFC_FIR_OP_CA0 << IFC_NAND_FIR0_OP1_SHIFT) | - (IFC_FIR_OP_RA0 << IFC_NAND_FIR0_OP2_SHIFT) | - (IFC_FIR_OP_WBCD << IFC_NAND_FIR0_OP3_SHIFT) | - (IFC_FIR_OP_CMD2 << IFC_NAND_FIR0_OP4_SHIFT), - &ifc->ifc_nand.nand_fir0); - iowrite32be( - (IFC_FIR_OP_CW1 << IFC_NAND_FIR1_OP5_SHIFT) | - (IFC_FIR_OP_RDSTAT << - IFC_NAND_FIR1_OP6_SHIFT) | - (IFC_FIR_OP_NOP << IFC_NAND_FIR1_OP7_SHIFT), - &ifc->ifc_nand.nand_fir1); + ifc_out32( + (IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | + (IFC_FIR_OP_CA0 << IFC_NAND_FIR0_OP1_SHIFT) | + (IFC_FIR_OP_RA0 << IFC_NAND_FIR0_OP2_SHIFT) | + (IFC_FIR_OP_WBCD << IFC_NAND_FIR0_OP3_SHIFT) | + (IFC_FIR_OP_CMD2 << IFC_NAND_FIR0_OP4_SHIFT), + &ifc->ifc_nand.nand_fir0); + ifc_out32( + (IFC_FIR_OP_CW1 << IFC_NAND_FIR1_OP5_SHIFT) | + (IFC_FIR_OP_RDSTAT << IFC_NAND_FIR1_OP6_SHIFT) | + (IFC_FIR_OP_NOP << IFC_NAND_FIR1_OP7_SHIFT), + &ifc->ifc_nand.nand_fir1); } else { nand_fcr0 = ((NAND_CMD_PAGEPROG << IFC_NAND_FCR0_CMD1_SHIFT) | @@ -527,20 +526,19 @@ static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command, (NAND_CMD_STATUS << IFC_NAND_FCR0_CMD3_SHIFT)); - iowrite32be( + ifc_out32( (IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | (IFC_FIR_OP_CMD2 << IFC_NAND_FIR0_OP1_SHIFT) | (IFC_FIR_OP_CA0 << IFC_NAND_FIR0_OP2_SHIFT) | (IFC_FIR_OP_RA0 << IFC_NAND_FIR0_OP3_SHIFT) | (IFC_FIR_OP_WBCD << IFC_NAND_FIR0_OP4_SHIFT), &ifc->ifc_nand.nand_fir0); - iowrite32be( - (IFC_FIR_OP_CMD1 << IFC_NAND_FIR1_OP5_SHIFT) | - (IFC_FIR_OP_CW3 << IFC_NAND_FIR1_OP6_SHIFT) | - (IFC_FIR_OP_RDSTAT << - IFC_NAND_FIR1_OP7_SHIFT) | - (IFC_FIR_OP_NOP << IFC_NAND_FIR1_OP8_SHIFT), - &ifc->ifc_nand.nand_fir1); + ifc_out32( + (IFC_FIR_OP_CMD1 << IFC_NAND_FIR1_OP5_SHIFT) | + (IFC_FIR_OP_CW3 << IFC_NAND_FIR1_OP6_SHIFT) | + (IFC_FIR_OP_RDSTAT << IFC_NAND_FIR1_OP7_SHIFT) | + (IFC_FIR_OP_NOP << IFC_NAND_FIR1_OP8_SHIFT), + &ifc->ifc_nand.nand_fir1); if (column >= mtd->writesize) nand_fcr0 |= @@ -555,7 +553,7 @@ static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command, column -= mtd->writesize; ifc_nand_ctrl->oob = 1; } - iowrite32be(nand_fcr0, &ifc->ifc_nand.nand_fcr0); + ifc_out32(nand_fcr0, &ifc->ifc_nand.nand_fcr0); set_addr(mtd, column, page_addr, ifc_nand_ctrl->oob); return; } @@ -563,24 +561,26 @@ static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command, /* PAGEPROG reuses all of the setup from SEQIN and adds the length */ case NAND_CMD_PAGEPROG: { if (ifc_nand_ctrl->oob) { - iowrite32be(ifc_nand_ctrl->index - - ifc_nand_ctrl->column, - &ifc->ifc_nand.nand_fbcr); + ifc_out32(ifc_nand_ctrl->index - + ifc_nand_ctrl->column, + &ifc->ifc_nand.nand_fbcr); } else { - iowrite32be(0, &ifc->ifc_nand.nand_fbcr); + ifc_out32(0, &ifc->ifc_nand.nand_fbcr); } fsl_ifc_run_command(mtd); return; } - case NAND_CMD_STATUS: - iowrite32be((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | - (IFC_FIR_OP_RB << IFC_NAND_FIR0_OP1_SHIFT), - &ifc->ifc_nand.nand_fir0); - iowrite32be(NAND_CMD_STATUS << IFC_NAND_FCR0_CMD0_SHIFT, - &ifc->ifc_nand.nand_fcr0); - iowrite32be(1, &ifc->ifc_nand.nand_fbcr); + case NAND_CMD_STATUS: { + void __iomem *addr; + + ifc_out32((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | + (IFC_FIR_OP_RB << IFC_NAND_FIR0_OP1_SHIFT), + &ifc->ifc_nand.nand_fir0); + ifc_out32(NAND_CMD_STATUS << IFC_NAND_FCR0_CMD0_SHIFT, + &ifc->ifc_nand.nand_fcr0); + ifc_out32(1, &ifc->ifc_nand.nand_fbcr); set_addr(mtd, 0, 0, 0); ifc_nand_ctrl->read_bytes = 1; @@ -590,17 +590,19 @@ static void fsl_ifc_cmdfunc(struct mtd_info *mtd, unsigned int command, * The chip always seems to report that it is * write-protected, even when it is not. */ + addr = ifc_nand_ctrl->addr; if (chip->options & NAND_BUSWIDTH_16) - setbits16(ifc_nand_ctrl->addr, NAND_STATUS_WP); + ifc_out16(ifc_in16(addr) | (NAND_STATUS_WP), addr); else - setbits8(ifc_nand_ctrl->addr, NAND_STATUS_WP); + ifc_out8(ifc_in8(addr) | (NAND_STATUS_WP), addr); return; + } case NAND_CMD_RESET: - iowrite32be(IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT, - &ifc->ifc_nand.nand_fir0); - iowrite32be(NAND_CMD_RESET << IFC_NAND_FCR0_CMD0_SHIFT, - &ifc->ifc_nand.nand_fcr0); + ifc_out32(IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT, + &ifc->ifc_nand.nand_fir0); + ifc_out32(NAND_CMD_RESET << IFC_NAND_FCR0_CMD0_SHIFT, + &ifc->ifc_nand.nand_fcr0); fsl_ifc_run_command(mtd); return; @@ -658,7 +660,7 @@ static uint8_t fsl_ifc_read_byte(struct mtd_info *mtd) */ if (ifc_nand_ctrl->index < ifc_nand_ctrl->read_bytes) { offset = ifc_nand_ctrl->index++; - return in_8(ifc_nand_ctrl->addr + offset); + return ifc_in8(ifc_nand_ctrl->addr + offset); } dev_err(priv->dev, "%s: beyond end of buffer\n", __func__); @@ -680,7 +682,7 @@ static uint8_t fsl_ifc_read_byte16(struct mtd_info *mtd) * next byte. */ if (ifc_nand_ctrl->index < ifc_nand_ctrl->read_bytes) { - data = in_be16(ifc_nand_ctrl->addr + ifc_nand_ctrl->index); + data = ifc_in16(ifc_nand_ctrl->addr + ifc_nand_ctrl->index); ifc_nand_ctrl->index += 2; return (uint8_t) data; } @@ -726,18 +728,18 @@ static int fsl_ifc_wait(struct mtd_info *mtd, struct nand_chip *chip) u32 nand_fsr; /* Use READ_STATUS command, but wait for the device to be ready */ - iowrite32be((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | - (IFC_FIR_OP_RDSTAT << IFC_NAND_FIR0_OP1_SHIFT), - &ifc->ifc_nand.nand_fir0); - iowrite32be(NAND_CMD_STATUS << IFC_NAND_FCR0_CMD0_SHIFT, - &ifc->ifc_nand.nand_fcr0); - iowrite32be(1, &ifc->ifc_nand.nand_fbcr); + ifc_out32((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | + (IFC_FIR_OP_RDSTAT << IFC_NAND_FIR0_OP1_SHIFT), + &ifc->ifc_nand.nand_fir0); + ifc_out32(NAND_CMD_STATUS << IFC_NAND_FCR0_CMD0_SHIFT, + &ifc->ifc_nand.nand_fcr0); + ifc_out32(1, &ifc->ifc_nand.nand_fbcr); set_addr(mtd, 0, 0, 0); ifc_nand_ctrl->read_bytes = 1; fsl_ifc_run_command(mtd); - nand_fsr = ioread32be(&ifc->ifc_nand.nand_fsr); + nand_fsr = ifc_in32(&ifc->ifc_nand.nand_fsr); /* * The chip always seems to report that it is @@ -829,34 +831,34 @@ static void fsl_ifc_sram_init(struct fsl_ifc_mtd *priv) uint32_t cs = priv->bank; /* Save CSOR and CSOR_ext */ - csor = ioread32be(&ifc->csor_cs[cs].csor); - csor_ext = ioread32be(&ifc->csor_cs[cs].csor_ext); + csor = ifc_in32(&ifc->csor_cs[cs].csor); + csor_ext = ifc_in32(&ifc->csor_cs[cs].csor_ext); /* chage PageSize 8K and SpareSize 1K*/ csor_8k = (csor & ~(CSOR_NAND_PGS_MASK)) | 0x0018C000; - iowrite32be(csor_8k, &ifc->csor_cs[cs].csor); - iowrite32be(0x0000400, &ifc->csor_cs[cs].csor_ext); + ifc_out32(csor_8k, &ifc->csor_cs[cs].csor); + ifc_out32(0x0000400, &ifc->csor_cs[cs].csor_ext); /* READID */ - iowrite32be((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | - (IFC_FIR_OP_UA << IFC_NAND_FIR0_OP1_SHIFT) | - (IFC_FIR_OP_RB << IFC_NAND_FIR0_OP2_SHIFT), - &ifc->ifc_nand.nand_fir0); - iowrite32be(NAND_CMD_READID << IFC_NAND_FCR0_CMD0_SHIFT, - &ifc->ifc_nand.nand_fcr0); - iowrite32be(0x0, &ifc->ifc_nand.row3); + ifc_out32((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) | + (IFC_FIR_OP_UA << IFC_NAND_FIR0_OP1_SHIFT) | + (IFC_FIR_OP_RB << IFC_NAND_FIR0_OP2_SHIFT), + &ifc->ifc_nand.nand_fir0); + ifc_out32(NAND_CMD_READID << IFC_NAND_FCR0_CMD0_SHIFT, + &ifc->ifc_nand.nand_fcr0); + ifc_out32(0x0, &ifc->ifc_nand.row3); - iowrite32be(0x0, &ifc->ifc_nand.nand_fbcr); + ifc_out32(0x0, &ifc->ifc_nand.nand_fbcr); /* Program ROW0/COL0 */ - iowrite32be(0x0, &ifc->ifc_nand.row0); - iowrite32be(0x0, &ifc->ifc_nand.col0); + ifc_out32(0x0, &ifc->ifc_nand.row0); + ifc_out32(0x0, &ifc->ifc_nand.col0); /* set the chip select for NAND Transaction */ - iowrite32be(cs << IFC_NAND_CSEL_SHIFT, &ifc->ifc_nand.nand_csel); + ifc_out32(cs << IFC_NAND_CSEL_SHIFT, &ifc->ifc_nand.nand_csel); /* start read seq */ - iowrite32be(IFC_NAND_SEQ_STRT_FIR_STRT, &ifc->ifc_nand.nandseq_strt); + ifc_out32(IFC_NAND_SEQ_STRT_FIR_STRT, &ifc->ifc_nand.nandseq_strt); /* wait for command complete flag or timeout */ wait_event_timeout(ctrl->nand_wait, ctrl->nand_stat, @@ -866,8 +868,8 @@ static void fsl_ifc_sram_init(struct fsl_ifc_mtd *priv) printk(KERN_ERR "fsl-ifc: Failed to Initialise SRAM\n"); /* Restore CSOR and CSOR_ext */ - iowrite32be(csor, &ifc->csor_cs[cs].csor); - iowrite32be(csor_ext, &ifc->csor_cs[cs].csor_ext); + ifc_out32(csor, &ifc->csor_cs[cs].csor); + ifc_out32(csor_ext, &ifc->csor_cs[cs].csor_ext); } static int fsl_ifc_chip_init(struct fsl_ifc_mtd *priv) @@ -884,7 +886,7 @@ static int fsl_ifc_chip_init(struct fsl_ifc_mtd *priv) /* fill in nand_chip structure */ /* set up function call table */ - if ((ioread32be(&ifc->cspr_cs[priv->bank].cspr)) & CSPR_PORT_SIZE_16) + if ((ifc_in32(&ifc->cspr_cs[priv->bank].cspr)) & CSPR_PORT_SIZE_16) chip->read_byte = fsl_ifc_read_byte16; else chip->read_byte = fsl_ifc_read_byte; @@ -898,13 +900,13 @@ static int fsl_ifc_chip_init(struct fsl_ifc_mtd *priv) chip->bbt_td = &bbt_main_descr; chip->bbt_md = &bbt_mirror_descr; - iowrite32be(0x0, &ifc->ifc_nand.ncfgr); + ifc_out32(0x0, &ifc->ifc_nand.ncfgr); /* set up nand options */ chip->bbt_options = NAND_BBT_USE_FLASH; chip->options = NAND_NO_SUBPAGE_WRITE; - if (ioread32be(&ifc->cspr_cs[priv->bank].cspr) & CSPR_PORT_SIZE_16) { + if (ifc_in32(&ifc->cspr_cs[priv->bank].cspr) & CSPR_PORT_SIZE_16) { chip->read_byte = fsl_ifc_read_byte16; chip->options |= NAND_BUSWIDTH_16; } else { @@ -917,7 +919,7 @@ static int fsl_ifc_chip_init(struct fsl_ifc_mtd *priv) chip->ecc.read_page = fsl_ifc_read_page; chip->ecc.write_page = fsl_ifc_write_page; - csor = ioread32be(&ifc->csor_cs[priv->bank].csor); + csor = ifc_in32(&ifc->csor_cs[priv->bank].csor); /* Hardware generates ECC per 512 Bytes */ chip->ecc.size = 512; @@ -1006,7 +1008,7 @@ static int fsl_ifc_chip_remove(struct fsl_ifc_mtd *priv) static int match_bank(struct fsl_ifc_regs __iomem *ifc, int bank, phys_addr_t addr) { - u32 cspr = ioread32be(&ifc->cspr_cs[bank].cspr); + u32 cspr = ifc_in32(&ifc->cspr_cs[bank].cspr); if (!(cspr & CSPR_V)) return 0; @@ -1092,16 +1094,16 @@ static int fsl_ifc_nand_probe(struct platform_device *dev) dev_set_drvdata(priv->dev, priv); - iowrite32be(IFC_NAND_EVTER_EN_OPC_EN | - IFC_NAND_EVTER_EN_FTOER_EN | - IFC_NAND_EVTER_EN_WPER_EN, - &ifc->ifc_nand.nand_evter_en); + ifc_out32(IFC_NAND_EVTER_EN_OPC_EN | + IFC_NAND_EVTER_EN_FTOER_EN | + IFC_NAND_EVTER_EN_WPER_EN, + &ifc->ifc_nand.nand_evter_en); /* enable NAND Machine Interrupts */ - iowrite32be(IFC_NAND_EVTER_INTR_OPCIR_EN | - IFC_NAND_EVTER_INTR_FTOERIR_EN | - IFC_NAND_EVTER_INTR_WPERIR_EN, - &ifc->ifc_nand.nand_evter_intr_en); + ifc_out32(IFC_NAND_EVTER_INTR_OPCIR_EN | + IFC_NAND_EVTER_INTR_FTOERIR_EN | + IFC_NAND_EVTER_INTR_WPERIR_EN, + &ifc->ifc_nand.nand_evter_intr_en); priv->mtd.name = kasprintf(GFP_KERNEL, "%llx.flash", (u64)res.start); if (!priv->mtd.name) { ret = -ENOMEM; diff --git a/drivers/net/ethernet/ti/cpmac.c b/drivers/net/ethernet/ti/cpmac.c index dd943004353654..cba3d9fcb46535 100644 --- a/drivers/net/ethernet/ti/cpmac.c +++ b/drivers/net/ethernet/ti/cpmac.c @@ -41,6 +41,8 @@ #include #include +#include + MODULE_AUTHOR("Eugene Konev "); MODULE_DESCRIPTION("TI AR7 ethernet driver (CPMAC)"); MODULE_LICENSE("GPL"); diff --git a/drivers/nfc/nxp-nci/i2c.c b/drivers/nfc/nxp-nci/i2c.c index 2f77f1d0363850..fac80c691914b6 100644 --- a/drivers/nfc/nxp-nci/i2c.c +++ b/drivers/nfc/nxp-nci/i2c.c @@ -318,19 +318,15 @@ static int nxp_nci_i2c_acpi_config(struct nxp_nci_i2c_phy *phy) struct i2c_client *client = phy->i2c_dev; struct gpio_desc *gpiod_en, *gpiod_fw, *gpiod_irq; - gpiod_en = devm_gpiod_get_index(&client->dev, NULL, 2); - gpiod_fw = devm_gpiod_get_index(&client->dev, NULL, 1); - gpiod_irq = devm_gpiod_get_index(&client->dev, NULL, 0); + gpiod_en = devm_gpiod_get_index(&client->dev, NULL, 2, GPIOD_OUT_LOW); + gpiod_fw = devm_gpiod_get_index(&client->dev, NULL, 1, GPIOD_OUT_LOW); + gpiod_irq = devm_gpiod_get_index(&client->dev, NULL, 0, GPIOD_IN); if (IS_ERR(gpiod_en) || IS_ERR(gpiod_fw) || IS_ERR(gpiod_irq)) { nfc_err(&client->dev, "No GPIOs\n"); return -EINVAL; } - gpiod_direction_output(gpiod_en, 0); - gpiod_direction_output(gpiod_fw, 0); - gpiod_direction_input(gpiod_irq); - client->irq = gpiod_to_irq(gpiod_irq); if (client->irq < 0) { nfc_err(&client->dev, "No IRQ\n"); diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c index 07496560e5b9e2..6e82bc42373ba3 100644 --- a/drivers/of/fdt.c +++ b/drivers/of/fdt.c @@ -967,7 +967,9 @@ int __init early_init_dt_scan_chosen(unsigned long node, const char *uname, } #ifdef CONFIG_HAVE_MEMBLOCK -#define MAX_PHYS_ADDR ((phys_addr_t)~0) +#ifndef MAX_MEMBLOCK_ADDR +#define MAX_MEMBLOCK_ADDR ((phys_addr_t)~0) +#endif void __init __weak early_init_dt_add_memory_arch(u64 base, u64 size) { @@ -984,16 +986,16 @@ void __init __weak early_init_dt_add_memory_arch(u64 base, u64 size) } size &= PAGE_MASK; - if (base > MAX_PHYS_ADDR) { + if (base > MAX_MEMBLOCK_ADDR) { pr_warning("Ignoring memory block 0x%llx - 0x%llx\n", base, base + size); return; } - if (base + size - 1 > MAX_PHYS_ADDR) { + if (base + size - 1 > MAX_MEMBLOCK_ADDR) { pr_warning("Ignoring memory range 0x%llx - 0x%llx\n", - ((u64)MAX_PHYS_ADDR) + 1, base + size); - size = MAX_PHYS_ADDR - base + 1; + ((u64)MAX_MEMBLOCK_ADDR) + 1, base + size); + size = MAX_MEMBLOCK_ADDR - base + 1; } if (base + size < phys_offset) { diff --git a/drivers/pcmcia/pxa2xx_base.c b/drivers/pcmcia/pxa2xx_base.c index 984a8ff559d887..483f919e0d2e24 100644 --- a/drivers/pcmcia/pxa2xx_base.c +++ b/drivers/pcmcia/pxa2xx_base.c @@ -296,20 +296,18 @@ static int pxa2xx_drv_pcmcia_probe(struct platform_device *dev) goto err0; } - clk = clk_get(&dev->dev, NULL); + clk = devm_clk_get(&dev->dev, NULL); if (IS_ERR(clk)) return -ENODEV; pxa2xx_drv_pcmcia_ops(ops); - sinfo = kzalloc(SKT_DEV_INFO_SIZE(ops->nr), GFP_KERNEL); - if (!sinfo) { - clk_put(clk); + sinfo = devm_kzalloc(&dev->dev, SKT_DEV_INFO_SIZE(ops->nr), + GFP_KERNEL); + if (!sinfo) return -ENOMEM; - } sinfo->nskt = ops->nr; - sinfo->clk = clk; /* Initialize processor specific parameters */ for (i = 0; i < ops->nr; i++) { @@ -332,8 +330,7 @@ static int pxa2xx_drv_pcmcia_probe(struct platform_device *dev) err1: while (--i >= 0) soc_pcmcia_remove_one(&sinfo->skt[i]); - clk_put(clk); - kfree(sinfo); + err0: return ret; } @@ -343,13 +340,9 @@ static int pxa2xx_drv_pcmcia_remove(struct platform_device *dev) struct skt_dev_info *sinfo = platform_get_drvdata(dev); int i; - platform_set_drvdata(dev, NULL); - for (i = 0; i < sinfo->nskt; i++) soc_pcmcia_remove_one(&sinfo->skt[i]); - clk_put(sinfo->clk); - kfree(sinfo); return 0; } diff --git a/drivers/pcmcia/sa1100_generic.c b/drivers/pcmcia/sa1100_generic.c index 803945259da80d..66acdc85727cce 100644 --- a/drivers/pcmcia/sa1100_generic.c +++ b/drivers/pcmcia/sa1100_generic.c @@ -93,8 +93,6 @@ static int sa11x0_drv_pcmcia_remove(struct platform_device *dev) for (i = 0; i < sinfo->nskt; i++) soc_pcmcia_remove_one(&sinfo->skt[i]); - clk_put(sinfo->clk); - kfree(sinfo); return 0; } diff --git a/drivers/pcmcia/sa1111_generic.c b/drivers/pcmcia/sa1111_generic.c index 80b8e9d052752d..a1531feb8460c3 100644 --- a/drivers/pcmcia/sa1111_generic.c +++ b/drivers/pcmcia/sa1111_generic.c @@ -135,8 +135,13 @@ int sa1111_pcmcia_add(struct sa1111_dev *dev, struct pcmcia_low_level *ops, int (*add)(struct soc_pcmcia_socket *)) { struct sa1111_pcmcia_socket *s; + struct clk *clk; int i, ret = 0; + clk = devm_clk_get(&dev->dev, NULL); + if (IS_ERR(clk)) + return PTR_ERR(clk); + ops->socket_state = sa1111_pcmcia_socket_state; for (i = 0; i < ops->nr; i++) { @@ -145,12 +150,8 @@ int sa1111_pcmcia_add(struct sa1111_dev *dev, struct pcmcia_low_level *ops, return -ENOMEM; s->soc.nr = ops->first + i; - s->soc.clk = clk_get(&dev->dev, NULL); - if (IS_ERR(s->soc.clk)) { - ret = PTR_ERR(s->soc.clk); - kfree(s); - return ret; - } + s->soc.clk = clk; + soc_pcmcia_init_one(&s->soc, ops, &dev->dev); s->dev = dev; if (s->soc.nr) { @@ -226,7 +227,6 @@ static int pcmcia_remove(struct sa1111_dev *dev) for (; s; s = next) { next = s->next; soc_pcmcia_remove_one(&s->soc); - clk_put(s->soc.clk); kfree(s); } diff --git a/drivers/pcmcia/sa11xx_base.c b/drivers/pcmcia/sa11xx_base.c index cf6de2c2b3293e..9f6ec87b9f9e13 100644 --- a/drivers/pcmcia/sa11xx_base.c +++ b/drivers/pcmcia/sa11xx_base.c @@ -222,18 +222,17 @@ int sa11xx_drv_pcmcia_probe(struct device *dev, struct pcmcia_low_level *ops, int i, ret = 0; struct clk *clk; - clk = clk_get(dev, NULL); + clk = devm_clk_get(dev, NULL); if (IS_ERR(clk)) return PTR_ERR(clk); sa11xx_drv_pcmcia_ops(ops); - sinfo = kzalloc(SKT_DEV_INFO_SIZE(nr), GFP_KERNEL); + sinfo = devm_kzalloc(dev, SKT_DEV_INFO_SIZE(nr), GFP_KERNEL); if (!sinfo) return -ENOMEM; sinfo->nskt = nr; - sinfo->clk = clk; /* Initialize processor specific parameters */ for (i = 0; i < nr; i++) { @@ -251,8 +250,6 @@ int sa11xx_drv_pcmcia_probe(struct device *dev, struct pcmcia_low_level *ops, if (ret) { while (--i >= 0) soc_pcmcia_remove_one(&sinfo->skt[i]); - clk_put(clk); - kfree(sinfo); } else { dev_set_drvdata(dev, sinfo); } @@ -261,16 +258,6 @@ int sa11xx_drv_pcmcia_probe(struct device *dev, struct pcmcia_low_level *ops, } EXPORT_SYMBOL(sa11xx_drv_pcmcia_probe); -static int __init sa11xx_pcmcia_init(void) -{ - return 0; -} -fs_initcall(sa11xx_pcmcia_init); - -static void __exit sa11xx_pcmcia_exit(void) {} - -module_exit(sa11xx_pcmcia_exit); - MODULE_AUTHOR("John Dorsey "); MODULE_DESCRIPTION("Linux PCMCIA Card Services: SA-11xx core socket driver"); MODULE_LICENSE("Dual MPL/GPL"); diff --git a/drivers/pcmcia/soc_common.h b/drivers/pcmcia/soc_common.h index e6fcbea5b682e8..94762a54d73197 100644 --- a/drivers/pcmcia/soc_common.h +++ b/drivers/pcmcia/soc_common.h @@ -68,7 +68,6 @@ struct soc_pcmcia_socket { struct skt_dev_info { int nskt; - struct clk *clk; struct soc_pcmcia_socket skt[0]; }; diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig new file mode 100644 index 00000000000000..d9de36ee165de1 --- /dev/null +++ b/drivers/perf/Kconfig @@ -0,0 +1,15 @@ +# +# Performance Monitor Drivers +# + +menu "Performance monitor support" + +config ARM_PMU + depends on PERF_EVENTS && ARM + bool "ARM PMU framework" + default y + help + Say y if you want to use CPU performance monitors on ARM-based + systems. + +endmenu diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile new file mode 100644 index 00000000000000..acd2397ded94dd --- /dev/null +++ b/drivers/perf/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_ARM_PMU) += arm_pmu.o diff --git a/arch/arm/kernel/perf_event.c b/drivers/perf/arm_pmu.c similarity index 94% rename from arch/arm/kernel/perf_event.c rename to drivers/perf/arm_pmu.c index 54272e0be7137c..2365a32a595e42 100644 --- a/arch/arm/kernel/perf_event.c +++ b/drivers/perf/arm_pmu.c @@ -15,7 +15,8 @@ #include #include #include -#include +#include +#include #include #include #include @@ -24,7 +25,6 @@ #include #include -#include static int armpmu_map_cache_event(const unsigned (*cache_map) @@ -790,52 +790,77 @@ static int probe_current_pmu(struct arm_pmu *pmu, static int of_pmu_irq_cfg(struct arm_pmu *pmu) { - int i, irq, *irqs; + int *irqs, i = 0; + bool using_spi = false; struct platform_device *pdev = pmu->plat_device; - /* Don't bother with PPIs; they're already affine */ - irq = platform_get_irq(pdev, 0); - if (irq >= 0 && irq_is_percpu(irq)) - return 0; - irqs = kcalloc(pdev->num_resources, sizeof(*irqs), GFP_KERNEL); if (!irqs) return -ENOMEM; - for (i = 0; i < pdev->num_resources; ++i) { + do { struct device_node *dn; - int cpu; + int cpu, irq; - dn = of_parse_phandle(pdev->dev.of_node, "interrupt-affinity", - i); - if (!dn) { - pr_warn("Failed to parse %s/interrupt-affinity[%d]\n", - of_node_full_name(pdev->dev.of_node), i); + /* See if we have an affinity entry */ + dn = of_parse_phandle(pdev->dev.of_node, "interrupt-affinity", i); + if (!dn) break; + + /* Check the IRQ type and prohibit a mix of PPIs and SPIs */ + irq = platform_get_irq(pdev, i); + if (irq >= 0) { + bool spi = !irq_is_percpu(irq); + + if (i > 0 && spi != using_spi) { + pr_err("PPI/SPI IRQ type mismatch for %s!\n", + dn->name); + kfree(irqs); + return -EINVAL; + } + + using_spi = spi; } + /* Now look up the logical CPU number */ for_each_possible_cpu(cpu) - if (arch_find_n_match_cpu_physical_id(dn, cpu, NULL)) + if (dn == of_cpu_device_node_get(cpu)) break; if (cpu >= nr_cpu_ids) { pr_warn("Failed to find logical CPU for %s\n", dn->name); of_node_put(dn); + cpumask_setall(&pmu->supported_cpus); break; } of_node_put(dn); - irqs[i] = cpu; + /* For SPIs, we need to track the affinity per IRQ */ + if (using_spi) { + if (i >= pdev->num_resources) { + of_node_put(dn); + break; + } + + irqs[i] = cpu; + } + + /* Keep track of the CPUs containing this PMU type */ cpumask_set_cpu(cpu, &pmu->supported_cpus); - } + of_node_put(dn); + i++; + } while (1); + + /* If we didn't manage to parse anything, claim to support all CPUs */ + if (cpumask_weight(&pmu->supported_cpus) == 0) + cpumask_setall(&pmu->supported_cpus); - if (i == pdev->num_resources) { + /* If we matched up the IRQ affinities, use them to route the SPIs */ + if (using_spi && i == pdev->num_resources) pmu->irq_affinity = irqs; - } else { + else kfree(irqs); - cpumask_setall(&pmu->supported_cpus); - } return 0; } diff --git a/drivers/phy/phy-tusb1210.c b/drivers/phy/phy-tusb1210.c index 2535d792d57af6..4f6d5e71507d1c 100644 --- a/drivers/phy/phy-tusb1210.c +++ b/drivers/phy/phy-tusb1210.c @@ -61,32 +61,26 @@ static const struct phy_ops phy_ops = { static int tusb1210_probe(struct ulpi *ulpi) { - struct gpio_desc *gpio; struct tusb1210 *tusb; u8 val, reg; - int ret; tusb = devm_kzalloc(&ulpi->dev, sizeof(*tusb), GFP_KERNEL); if (!tusb) return -ENOMEM; - gpio = devm_gpiod_get(&ulpi->dev, "reset"); - if (!IS_ERR(gpio)) { - ret = gpiod_direction_output(gpio, 0); - if (ret) - return ret; - gpiod_set_value_cansleep(gpio, 1); - tusb->gpio_reset = gpio; - } + tusb->gpio_reset = devm_gpiod_get_optional(&ulpi->dev, "reset", + GPIOD_OUT_LOW); + if (IS_ERR(tusb->gpio_reset)) + return PTR_ERR(tusb->gpio_reset); - gpio = devm_gpiod_get(&ulpi->dev, "cs"); - if (!IS_ERR(gpio)) { - ret = gpiod_direction_output(gpio, 0); - if (ret) - return ret; - gpiod_set_value_cansleep(gpio, 1); - tusb->gpio_cs = gpio; - } + gpiod_set_value_cansleep(tusb->gpio_reset, 1); + + tusb->gpio_cs = devm_gpiod_get_optional(&ulpi->dev, "cs", + GPIOD_OUT_LOW); + if (IS_ERR(tusb->gpio_cs)) + return PTR_ERR(tusb->gpio_cs); + + gpiod_set_value_cansleep(tusb->gpio_cs, 1); /* * VENDOR_SPECIFIC2 register in TUSB1210 can be used for configuring eye diff --git a/drivers/pinctrl/Kconfig b/drivers/pinctrl/Kconfig index 100d9ac2ae1f63..84dd2ed47a928e 100644 --- a/drivers/pinctrl/Kconfig +++ b/drivers/pinctrl/Kconfig @@ -82,6 +82,12 @@ config PINCTRL_AMD Requires ACPI/FDT device enumeration code to set up a platform device. +config PINCTRL_DIGICOLOR + bool + depends on OF && (ARCH_DIGICOLOR || COMPILE_TEST) + select PINMUX + select GENERIC_PINCONF + config PINCTRL_LANTIQ bool depends on LANTIQ @@ -240,6 +246,7 @@ source "drivers/pinctrl/samsung/Kconfig" source "drivers/pinctrl/sh-pfc/Kconfig" source "drivers/pinctrl/spear/Kconfig" source "drivers/pinctrl/sunxi/Kconfig" +source "drivers/pinctrl/uniphier/Kconfig" source "drivers/pinctrl/vt8500/Kconfig" source "drivers/pinctrl/mediatek/Kconfig" diff --git a/drivers/pinctrl/Makefile b/drivers/pinctrl/Makefile index f4216d9347e2cb..cad077c43fb731 100644 --- a/drivers/pinctrl/Makefile +++ b/drivers/pinctrl/Makefile @@ -2,12 +2,10 @@ subdir-ccflags-$(CONFIG_DEBUG_PINCTRL) += -DDEBUG -obj-$(CONFIG_PINCTRL) += core.o pinctrl-utils.o +obj-y += core.o pinctrl-utils.o obj-$(CONFIG_PINMUX) += pinmux.o obj-$(CONFIG_PINCONF) += pinconf.o -ifeq ($(CONFIG_OF),y) -obj-$(CONFIG_PINCTRL) += devicetree.o -endif +obj-$(CONFIG_OF) += devicetree.o obj-$(CONFIG_GENERIC_PINCONF) += pinconf-generic.o obj-$(CONFIG_PINCTRL_ADI2) += pinctrl-adi2.o obj-$(CONFIG_PINCTRL_AS3722) += pinctrl-as3722.o @@ -15,6 +13,7 @@ obj-$(CONFIG_PINCTRL_BF54x) += pinctrl-adi2-bf54x.o obj-$(CONFIG_PINCTRL_BF60x) += pinctrl-adi2-bf60x.o obj-$(CONFIG_PINCTRL_AT91) += pinctrl-at91.o obj-$(CONFIG_PINCTRL_AMD) += pinctrl-amd.o +obj-$(CONFIG_PINCTRL_DIGICOLOR) += pinctrl-digicolor.o obj-$(CONFIG_PINCTRL_FALCON) += pinctrl-falcon.o obj-$(CONFIG_PINCTRL_MESON) += meson/ obj-$(CONFIG_PINCTRL_PALMAS) += pinctrl-palmas.o @@ -51,5 +50,6 @@ obj-$(CONFIG_PINCTRL_SAMSUNG) += samsung/ obj-$(CONFIG_PINCTRL_SH_PFC) += sh-pfc/ obj-$(CONFIG_PLAT_SPEAR) += spear/ obj-$(CONFIG_ARCH_SUNXI) += sunxi/ +obj-$(CONFIG_ARCH_UNIPHIER) += uniphier/ obj-$(CONFIG_ARCH_VT8500) += vt8500/ obj-$(CONFIG_ARCH_MEDIATEK) += mediatek/ diff --git a/drivers/pinctrl/bcm/pinctrl-bcm2835.c b/drivers/pinctrl/bcm/pinctrl-bcm2835.c index 6177315ab74e5e..8efa235ca1c96f 100644 --- a/drivers/pinctrl/bcm/pinctrl-bcm2835.c +++ b/drivers/pinctrl/bcm/pinctrl-bcm2835.c @@ -586,9 +586,9 @@ static int bcm2835_gpio_irq_set_type(struct irq_data *data, unsigned int type) ret = __bcm2835_gpio_irq_set_type_disabled(pc, gpio, type); if (type & IRQ_TYPE_EDGE_BOTH) - __irq_set_handler_locked(data->irq, handle_edge_irq); + irq_set_handler_locked(data, handle_edge_irq); else - __irq_set_handler_locked(data->irq, handle_level_irq); + irq_set_handler_locked(data, handle_level_irq); spin_unlock_irqrestore(&pc->irq_lock[bank], flags); @@ -989,7 +989,6 @@ static int bcm2835_pinctrl_probe(struct platform_device *pdev) irq_set_chip_and_handler(irq, &bcm2835_gpio_irq_chip, handle_level_irq); irq_set_chip_data(irq, pc); - set_irq_flags(irq, IRQF_VALID); } for (i = 0; i < BCM2835_NUM_BANKS; i++) { diff --git a/drivers/pinctrl/core.c b/drivers/pinctrl/core.c index 8b8f3a04c35371..69723e07036baf 100644 --- a/drivers/pinctrl/core.c +++ b/drivers/pinctrl/core.c @@ -231,8 +231,7 @@ static int pinctrl_register_one_pin(struct pinctrl_dev *pctldev, pindesc = pin_desc_get(pctldev, number); if (pindesc != NULL) { - pr_err("pin %d already registered on %s\n", number, - pctldev->desc->name); + dev_err(pctldev->dev, "pin %d already registered\n", number); return -EINVAL; } diff --git a/drivers/pinctrl/devicetree.c b/drivers/pinctrl/devicetree.c index 0bbf7d71b28112..fe04e748dfe4b5 100644 --- a/drivers/pinctrl/devicetree.c +++ b/drivers/pinctrl/devicetree.c @@ -97,13 +97,7 @@ static int dt_remember_or_free_map(struct pinctrl *p, const char *statename, struct pinctrl_dev *of_pinctrl_get(struct device_node *np) { - struct pinctrl_dev *pctldev; - - pctldev = get_pinctrl_dev_from_of_node(np); - if (!pctldev) - return NULL; - - return pctldev; + return get_pinctrl_dev_from_of_node(np); } static int dt_to_map_one_config(struct pinctrl *p, const char *statename, diff --git a/drivers/pinctrl/freescale/Kconfig b/drivers/pinctrl/freescale/Kconfig index 12ef544b48942a..debe1219d76d61 100644 --- a/drivers/pinctrl/freescale/Kconfig +++ b/drivers/pinctrl/freescale/Kconfig @@ -87,6 +87,13 @@ config PINCTRL_IMX6SX help Say Y here to enable the imx6sx pinctrl driver +config PINCTRL_IMX6UL + bool "IMX6UL pinctrl driver" + depends on SOC_IMX6UL + select PINCTRL_IMX + help + Say Y here to enable the imx6ul pinctrl driver + config PINCTRL_IMX7D bool "IMX7D pinctrl driver" depends on SOC_IMX7D diff --git a/drivers/pinctrl/freescale/Makefile b/drivers/pinctrl/freescale/Makefile index 343cb436ab1757..d44c9e253f216a 100644 --- a/drivers/pinctrl/freescale/Makefile +++ b/drivers/pinctrl/freescale/Makefile @@ -12,6 +12,7 @@ obj-$(CONFIG_PINCTRL_IMX6Q) += pinctrl-imx6q.o obj-$(CONFIG_PINCTRL_IMX6Q) += pinctrl-imx6dl.o obj-$(CONFIG_PINCTRL_IMX6SL) += pinctrl-imx6sl.o obj-$(CONFIG_PINCTRL_IMX6SX) += pinctrl-imx6sx.o +obj-$(CONFIG_PINCTRL_IMX6UL) += pinctrl-imx6ul.o obj-$(CONFIG_PINCTRL_IMX7D) += pinctrl-imx7d.o obj-$(CONFIG_PINCTRL_VF610) += pinctrl-vf610.o obj-$(CONFIG_PINCTRL_MXS) += pinctrl-mxs.o diff --git a/drivers/pinctrl/freescale/pinctrl-imx6ul.c b/drivers/pinctrl/freescale/pinctrl-imx6ul.c new file mode 100644 index 00000000000000..08e75764e7be01 --- /dev/null +++ b/drivers/pinctrl/freescale/pinctrl-imx6ul.c @@ -0,0 +1,322 @@ +/* + * Copyright (C) 2015 Freescale Semiconductor, Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "pinctrl-imx.h" + +enum imx6ul_pads { + MX6UL_PAD_RESERVE0 = 0, + MX6UL_PAD_RESERVE1 = 1, + MX6UL_PAD_RESERVE2 = 2, + MX6UL_PAD_RESERVE3 = 3, + MX6UL_PAD_RESERVE4 = 4, + MX6UL_PAD_RESERVE5 = 5, + MX6UL_PAD_RESERVE6 = 6, + MX6UL_PAD_RESERVE7 = 7, + MX6UL_PAD_RESERVE8 = 8, + MX6UL_PAD_RESERVE9 = 9, + MX6UL_PAD_RESERVE10 = 10, + MX6UL_PAD_SNVS_TAMPER4 = 11, + MX6UL_PAD_RESERVE12 = 12, + MX6UL_PAD_RESERVE13 = 13, + MX6UL_PAD_RESERVE14 = 14, + MX6UL_PAD_RESERVE15 = 15, + MX6UL_PAD_RESERVE16 = 16, + MX6UL_PAD_JTAG_MOD = 17, + MX6UL_PAD_JTAG_TMS = 18, + MX6UL_PAD_JTAG_TDO = 19, + MX6UL_PAD_JTAG_TDI = 20, + MX6UL_PAD_JTAG_TCK = 21, + MX6UL_PAD_JTAG_TRST_B = 22, + MX6UL_PAD_GPIO1_IO00 = 23, + MX6UL_PAD_GPIO1_IO01 = 24, + MX6UL_PAD_GPIO1_IO02 = 25, + MX6UL_PAD_GPIO1_IO03 = 26, + MX6UL_PAD_GPIO1_IO04 = 27, + MX6UL_PAD_GPIO1_IO05 = 28, + MX6UL_PAD_GPIO1_IO06 = 29, + MX6UL_PAD_GPIO1_IO07 = 30, + MX6UL_PAD_GPIO1_IO08 = 31, + MX6UL_PAD_GPIO1_IO09 = 32, + MX6UL_PAD_UART1_TX_DATA = 33, + MX6UL_PAD_UART1_RX_DATA = 34, + MX6UL_PAD_UART1_CTS_B = 35, + MX6UL_PAD_UART1_RTS_B = 36, + MX6UL_PAD_UART2_TX_DATA = 37, + MX6UL_PAD_UART2_RX_DATA = 38, + MX6UL_PAD_UART2_CTS_B = 39, + MX6UL_PAD_UART2_RTS_B = 40, + MX6UL_PAD_UART3_TX_DATA = 41, + MX6UL_PAD_UART3_RX_DATA = 42, + MX6UL_PAD_UART3_CTS_B = 43, + MX6UL_PAD_UART3_RTS_B = 44, + MX6UL_PAD_UART4_TX_DATA = 45, + MX6UL_PAD_UART4_RX_DATA = 46, + MX6UL_PAD_UART5_TX_DATA = 47, + MX6UL_PAD_UART5_RX_DATA = 48, + MX6UL_PAD_ENET1_RX_DATA0 = 49, + MX6UL_PAD_ENET1_RX_DATA1 = 50, + MX6UL_PAD_ENET1_RX_EN = 51, + MX6UL_PAD_ENET1_TX_DATA0 = 52, + MX6UL_PAD_ENET1_TX_DATA1 = 53, + MX6UL_PAD_ENET1_TX_EN = 54, + MX6UL_PAD_ENET1_TX_CLK = 55, + MX6UL_PAD_ENET1_RX_ER = 56, + MX6UL_PAD_ENET2_RX_DATA0 = 57, + MX6UL_PAD_ENET2_RX_DATA1 = 58, + MX6UL_PAD_ENET2_RX_EN = 59, + MX6UL_PAD_ENET2_TX_DATA0 = 60, + MX6UL_PAD_ENET2_TX_DATA1 = 61, + MX6UL_PAD_ENET2_TX_EN = 62, + MX6UL_PAD_ENET2_TX_CLK = 63, + MX6UL_PAD_ENET2_RX_ER = 64, + MX6UL_PAD_LCD_CLK = 65, + MX6UL_PAD_LCD_ENABLE = 66, + MX6UL_PAD_LCD_HSYNC = 67, + MX6UL_PAD_LCD_VSYNC = 68, + MX6UL_PAD_LCD_RESET = 69, + MX6UL_PAD_LCD_DATA00 = 70, + MX6UL_PAD_LCD_DATA01 = 71, + MX6UL_PAD_LCD_DATA02 = 72, + MX6UL_PAD_LCD_DATA03 = 73, + MX6UL_PAD_LCD_DATA04 = 74, + MX6UL_PAD_LCD_DATA05 = 75, + MX6UL_PAD_LCD_DATA06 = 76, + MX6UL_PAD_LCD_DATA07 = 77, + MX6UL_PAD_LCD_DATA08 = 78, + MX6UL_PAD_LCD_DATA09 = 79, + MX6UL_PAD_LCD_DATA10 = 80, + MX6UL_PAD_LCD_DATA11 = 81, + MX6UL_PAD_LCD_DATA12 = 82, + MX6UL_PAD_LCD_DATA13 = 83, + MX6UL_PAD_LCD_DATA14 = 84, + MX6UL_PAD_LCD_DATA15 = 85, + MX6UL_PAD_LCD_DATA16 = 86, + MX6UL_PAD_LCD_DATA17 = 87, + MX6UL_PAD_LCD_DATA18 = 88, + MX6UL_PAD_LCD_DATA19 = 89, + MX6UL_PAD_LCD_DATA20 = 90, + MX6UL_PAD_LCD_DATA21 = 91, + MX6UL_PAD_LCD_DATA22 = 92, + MX6UL_PAD_LCD_DATA23 = 93, + MX6UL_PAD_NAND_RE_B = 94, + MX6UL_PAD_NAND_WE_B = 95, + MX6UL_PAD_NAND_DATA00 = 96, + MX6UL_PAD_NAND_DATA01 = 97, + MX6UL_PAD_NAND_DATA02 = 98, + MX6UL_PAD_NAND_DATA03 = 99, + MX6UL_PAD_NAND_DATA04 = 100, + MX6UL_PAD_NAND_DATA05 = 101, + MX6UL_PAD_NAND_DATA06 = 102, + MX6UL_PAD_NAND_DATA07 = 103, + MX6UL_PAD_NAND_ALE = 104, + MX6UL_PAD_NAND_WP_B = 105, + MX6UL_PAD_NAND_READY_B = 106, + MX6UL_PAD_NAND_CE0_B = 107, + MX6UL_PAD_NAND_CE1_B = 108, + MX6UL_PAD_NAND_CLE = 109, + MX6UL_PAD_NAND_DQS = 110, + MX6UL_PAD_SD1_CMD = 111, + MX6UL_PAD_SD1_CLK = 112, + MX6UL_PAD_SD1_DATA0 = 113, + MX6UL_PAD_SD1_DATA1 = 114, + MX6UL_PAD_SD1_DATA2 = 115, + MX6UL_PAD_SD1_DATA3 = 116, + MX6UL_PAD_CSI_MCLK = 117, + MX6UL_PAD_CSI_PIXCLK = 118, + MX6UL_PAD_CSI_VSYNC = 119, + MX6UL_PAD_CSI_HSYNC = 120, + MX6UL_PAD_CSI_DATA00 = 121, + MX6UL_PAD_CSI_DATA01 = 122, + MX6UL_PAD_CSI_DATA02 = 123, + MX6UL_PAD_CSI_DATA03 = 124, + MX6UL_PAD_CSI_DATA04 = 125, + MX6UL_PAD_CSI_DATA05 = 126, + MX6UL_PAD_CSI_DATA06 = 127, + MX6UL_PAD_CSI_DATA07 = 128, +}; + +/* Pad names for the pinmux subsystem */ +static const struct pinctrl_pin_desc imx6ul_pinctrl_pads[] = { + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE0), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE1), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE2), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE3), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE4), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE5), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE6), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE7), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE8), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE9), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE10), + IMX_PINCTRL_PIN(MX6UL_PAD_SNVS_TAMPER4), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE12), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE13), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE14), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE15), + IMX_PINCTRL_PIN(MX6UL_PAD_RESERVE16), + IMX_PINCTRL_PIN(MX6UL_PAD_JTAG_MOD), + IMX_PINCTRL_PIN(MX6UL_PAD_JTAG_TMS), + IMX_PINCTRL_PIN(MX6UL_PAD_JTAG_TDO), + IMX_PINCTRL_PIN(MX6UL_PAD_JTAG_TDI), + IMX_PINCTRL_PIN(MX6UL_PAD_JTAG_TCK), + IMX_PINCTRL_PIN(MX6UL_PAD_JTAG_TRST_B), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO00), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO01), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO02), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO03), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO04), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO05), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO06), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO07), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO08), + IMX_PINCTRL_PIN(MX6UL_PAD_GPIO1_IO09), + IMX_PINCTRL_PIN(MX6UL_PAD_UART1_TX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_UART1_RX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_UART1_CTS_B), + IMX_PINCTRL_PIN(MX6UL_PAD_UART1_RTS_B), + IMX_PINCTRL_PIN(MX6UL_PAD_UART2_TX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_UART2_RX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_UART2_CTS_B), + IMX_PINCTRL_PIN(MX6UL_PAD_UART2_RTS_B), + IMX_PINCTRL_PIN(MX6UL_PAD_UART3_TX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_UART3_RX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_UART3_CTS_B), + IMX_PINCTRL_PIN(MX6UL_PAD_UART3_RTS_B), + IMX_PINCTRL_PIN(MX6UL_PAD_UART4_TX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_UART4_RX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_UART5_TX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_UART5_RX_DATA), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET1_RX_DATA0), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET1_RX_DATA1), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET1_RX_EN), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET1_TX_DATA0), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET1_TX_DATA1), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET1_TX_EN), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET1_TX_CLK), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET1_RX_ER), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET2_RX_DATA0), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET2_RX_DATA1), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET2_RX_EN), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET2_TX_DATA0), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET2_TX_DATA1), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET2_TX_EN), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET2_TX_CLK), + IMX_PINCTRL_PIN(MX6UL_PAD_ENET2_RX_ER), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_CLK), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_ENABLE), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_HSYNC), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_VSYNC), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_RESET), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA00), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA01), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA02), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA03), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA04), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA05), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA06), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA07), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA08), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA09), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA10), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA11), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA12), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA13), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA14), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA15), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA16), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA17), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA18), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA19), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA20), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA21), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA22), + IMX_PINCTRL_PIN(MX6UL_PAD_LCD_DATA23), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_RE_B), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_WE_B), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_DATA00), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_DATA01), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_DATA02), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_DATA03), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_DATA04), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_DATA05), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_DATA06), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_DATA07), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_ALE), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_WP_B), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_READY_B), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_CE0_B), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_CE1_B), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_CLE), + IMX_PINCTRL_PIN(MX6UL_PAD_NAND_DQS), + IMX_PINCTRL_PIN(MX6UL_PAD_SD1_CMD), + IMX_PINCTRL_PIN(MX6UL_PAD_SD1_CLK), + IMX_PINCTRL_PIN(MX6UL_PAD_SD1_DATA0), + IMX_PINCTRL_PIN(MX6UL_PAD_SD1_DATA1), + IMX_PINCTRL_PIN(MX6UL_PAD_SD1_DATA2), + IMX_PINCTRL_PIN(MX6UL_PAD_SD1_DATA3), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_MCLK), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_PIXCLK), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_VSYNC), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_HSYNC), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_DATA00), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_DATA01), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_DATA02), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_DATA03), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_DATA04), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_DATA05), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_DATA06), + IMX_PINCTRL_PIN(MX6UL_PAD_CSI_DATA07), +}; + +static struct imx_pinctrl_soc_info imx6ul_pinctrl_info = { + .pins = imx6ul_pinctrl_pads, + .npins = ARRAY_SIZE(imx6ul_pinctrl_pads), +}; + +static struct of_device_id imx6ul_pinctrl_of_match[] = { + { .compatible = "fsl,imx6ul-iomuxc", }, + { /* sentinel */ } +}; + +static int imx6ul_pinctrl_probe(struct platform_device *pdev) +{ + return imx_pinctrl_probe(pdev, &imx6ul_pinctrl_info); +} + +static struct platform_driver imx6ul_pinctrl_driver = { + .driver = { + .name = "imx6ul-pinctrl", + .of_match_table = of_match_ptr(imx6ul_pinctrl_of_match), + }, + .probe = imx6ul_pinctrl_probe, + .remove = imx_pinctrl_remove, +}; + +static int __init imx6ul_pinctrl_init(void) +{ + return platform_driver_register(&imx6ul_pinctrl_driver); +} +arch_initcall(imx6ul_pinctrl_init); + +static void __exit imx6ul_pinctrl_exit(void) +{ + platform_driver_unregister(&imx6ul_pinctrl_driver); +} +module_exit(imx6ul_pinctrl_exit); + +MODULE_AUTHOR("Anson Huang "); +MODULE_DESCRIPTION("Freescale imx6ul pinctrl driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pinctrl/intel/pinctrl-baytrail.c b/drivers/pinctrl/intel/pinctrl-baytrail.c index 2062c224e32fbe..dac4865f3203d1 100644 --- a/drivers/pinctrl/intel/pinctrl-baytrail.c +++ b/drivers/pinctrl/intel/pinctrl-baytrail.c @@ -12,11 +12,6 @@ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for * more details. - * - * You should have received a copy of the GNU General Public License along with - * this program; if not, write to the Free Software Foundation, Inc., - * 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA. - * */ #include @@ -146,7 +141,7 @@ struct byt_gpio_pin_context { struct byt_gpio { struct gpio_chip chip; struct platform_device *pdev; - spinlock_t lock; + raw_spinlock_t lock; void __iomem *reg_base; struct pinctrl_gpio_range *range; struct byt_gpio_pin_context *saved_context; @@ -174,11 +169,11 @@ static void byt_gpio_clear_triggering(struct byt_gpio *vg, unsigned offset) unsigned long flags; u32 value; - spin_lock_irqsave(&vg->lock, flags); + raw_spin_lock_irqsave(&vg->lock, flags); value = readl(reg); value &= ~(BYT_TRIG_POS | BYT_TRIG_NEG | BYT_TRIG_LVL); writel(value, reg); - spin_unlock_irqrestore(&vg->lock, flags); + raw_spin_unlock_irqrestore(&vg->lock, flags); } static u32 byt_get_gpio_mux(struct byt_gpio *vg, unsigned offset) @@ -201,6 +196,9 @@ static int byt_gpio_request(struct gpio_chip *chip, unsigned offset) struct byt_gpio *vg = to_byt_gpio(chip); void __iomem *reg = byt_gpio_reg(chip, offset, BYT_CONF0_REG); u32 value, gpio_mux; + unsigned long flags; + + raw_spin_lock_irqsave(&vg->lock, flags); /* * In most cases, func pin mux 000 means GPIO function. @@ -214,18 +212,16 @@ static int byt_gpio_request(struct gpio_chip *chip, unsigned offset) value = readl(reg) & BYT_PIN_MUX; gpio_mux = byt_get_gpio_mux(vg, offset); if (WARN_ON(gpio_mux != value)) { - unsigned long flags; - - spin_lock_irqsave(&vg->lock, flags); value = readl(reg) & ~BYT_PIN_MUX; value |= gpio_mux; writel(value, reg); - spin_unlock_irqrestore(&vg->lock, flags); dev_warn(&vg->pdev->dev, "pin %u forcibly re-configured as GPIO\n", offset); } + raw_spin_unlock_irqrestore(&vg->lock, flags); + pm_runtime_get(&vg->pdev->dev); return 0; @@ -250,7 +246,7 @@ static int byt_irq_type(struct irq_data *d, unsigned type) if (offset >= vg->chip.ngpio) return -EINVAL; - spin_lock_irqsave(&vg->lock, flags); + raw_spin_lock_irqsave(&vg->lock, flags); value = readl(reg); WARN(value & BYT_DIRECT_IRQ_EN, @@ -265,11 +261,11 @@ static int byt_irq_type(struct irq_data *d, unsigned type) writel(value, reg); if (type & IRQ_TYPE_EDGE_BOTH) - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); else if (type & IRQ_TYPE_LEVEL_MASK) - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); - spin_unlock_irqrestore(&vg->lock, flags); + raw_spin_unlock_irqrestore(&vg->lock, flags); return 0; } @@ -277,7 +273,15 @@ static int byt_irq_type(struct irq_data *d, unsigned type) static int byt_gpio_get(struct gpio_chip *chip, unsigned offset) { void __iomem *reg = byt_gpio_reg(chip, offset, BYT_VAL_REG); - return readl(reg) & BYT_LEVEL; + struct byt_gpio *vg = to_byt_gpio(chip); + unsigned long flags; + u32 val; + + raw_spin_lock_irqsave(&vg->lock, flags); + val = readl(reg); + raw_spin_unlock_irqrestore(&vg->lock, flags); + + return val & BYT_LEVEL; } static void byt_gpio_set(struct gpio_chip *chip, unsigned offset, int value) @@ -287,7 +291,7 @@ static void byt_gpio_set(struct gpio_chip *chip, unsigned offset, int value) unsigned long flags; u32 old_val; - spin_lock_irqsave(&vg->lock, flags); + raw_spin_lock_irqsave(&vg->lock, flags); old_val = readl(reg); @@ -296,7 +300,7 @@ static void byt_gpio_set(struct gpio_chip *chip, unsigned offset, int value) else writel(old_val & ~BYT_LEVEL, reg); - spin_unlock_irqrestore(&vg->lock, flags); + raw_spin_unlock_irqrestore(&vg->lock, flags); } static int byt_gpio_direction_input(struct gpio_chip *chip, unsigned offset) @@ -306,13 +310,13 @@ static int byt_gpio_direction_input(struct gpio_chip *chip, unsigned offset) unsigned long flags; u32 value; - spin_lock_irqsave(&vg->lock, flags); + raw_spin_lock_irqsave(&vg->lock, flags); value = readl(reg) | BYT_DIR_MASK; value &= ~BYT_INPUT_EN; /* active low */ writel(value, reg); - spin_unlock_irqrestore(&vg->lock, flags); + raw_spin_unlock_irqrestore(&vg->lock, flags); return 0; } @@ -326,7 +330,7 @@ static int byt_gpio_direction_output(struct gpio_chip *chip, unsigned long flags; u32 reg_val; - spin_lock_irqsave(&vg->lock, flags); + raw_spin_lock_irqsave(&vg->lock, flags); /* * Before making any direction modifications, do a check if gpio @@ -345,7 +349,7 @@ static int byt_gpio_direction_output(struct gpio_chip *chip, else writel(reg_val & ~BYT_LEVEL, reg); - spin_unlock_irqrestore(&vg->lock, flags); + raw_spin_unlock_irqrestore(&vg->lock, flags); return 0; } @@ -354,18 +358,19 @@ static void byt_gpio_dbg_show(struct seq_file *s, struct gpio_chip *chip) { struct byt_gpio *vg = to_byt_gpio(chip); int i; - unsigned long flags; u32 conf0, val, offs; - spin_lock_irqsave(&vg->lock, flags); - for (i = 0; i < vg->chip.ngpio; i++) { const char *pull_str = NULL; const char *pull = NULL; + unsigned long flags; const char *label; offs = vg->range->pins[i] * 16; + + raw_spin_lock_irqsave(&vg->lock, flags); conf0 = readl(vg->reg_base + offs + BYT_CONF0_REG); val = readl(vg->reg_base + offs + BYT_VAL_REG); + raw_spin_unlock_irqrestore(&vg->lock, flags); label = gpiochip_is_requested(chip, i); if (!label) @@ -418,7 +423,6 @@ static void byt_gpio_dbg_show(struct seq_file *s, struct gpio_chip *chip) seq_puts(s, "\n"); } - spin_unlock_irqrestore(&vg->lock, flags); } static void byt_gpio_irq_handler(unsigned irq, struct irq_desc *desc) @@ -450,8 +454,10 @@ static void byt_irq_ack(struct irq_data *d) unsigned offset = irqd_to_hwirq(d); void __iomem *reg; + raw_spin_lock(&vg->lock); reg = byt_gpio_reg(&vg->chip, offset, BYT_INT_STAT_REG); writel(BIT(offset % 32), reg); + raw_spin_unlock(&vg->lock); } static void byt_irq_unmask(struct irq_data *d) @@ -463,9 +469,9 @@ static void byt_irq_unmask(struct irq_data *d) void __iomem *reg; u32 value; - spin_lock_irqsave(&vg->lock, flags); - reg = byt_gpio_reg(&vg->chip, offset, BYT_CONF0_REG); + + raw_spin_lock_irqsave(&vg->lock, flags); value = readl(reg); switch (irqd_get_trigger_type(d)) { @@ -486,7 +492,7 @@ static void byt_irq_unmask(struct irq_data *d) writel(value, reg); - spin_unlock_irqrestore(&vg->lock, flags); + raw_spin_unlock_irqrestore(&vg->lock, flags); } static void byt_irq_mask(struct irq_data *d) @@ -578,7 +584,7 @@ static int byt_gpio_probe(struct platform_device *pdev) if (IS_ERR(vg->reg_base)) return PTR_ERR(vg->reg_base); - spin_lock_init(&vg->lock); + raw_spin_lock_init(&vg->lock); gc = &vg->chip; gc->label = dev_name(&pdev->dev); diff --git a/drivers/pinctrl/intel/pinctrl-cherryview.c b/drivers/pinctrl/intel/pinctrl-cherryview.c index 3f737daa3fd269..2d5d3ddc36e5b3 100644 --- a/drivers/pinctrl/intel/pinctrl-cherryview.c +++ b/drivers/pinctrl/intel/pinctrl-cherryview.c @@ -174,7 +174,7 @@ struct chv_pinctrl { struct pinctrl_dev *pctldev; struct gpio_chip chip; void __iomem *regs; - spinlock_t lock; + raw_spinlock_t lock; unsigned intr_lines[16]; const struct chv_community *community; u32 saved_intmask; @@ -720,13 +720,13 @@ static void chv_pin_dbg_show(struct pinctrl_dev *pctldev, struct seq_file *s, u32 ctrl0, ctrl1; bool locked; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); ctrl0 = readl(chv_padreg(pctrl, offset, CHV_PADCTRL0)); ctrl1 = readl(chv_padreg(pctrl, offset, CHV_PADCTRL1)); locked = chv_pad_locked(pctrl, offset); - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); if (ctrl0 & CHV_PADCTRL0_GPIOEN) { seq_puts(s, "GPIO "); @@ -789,14 +789,14 @@ static int chv_pinmux_set_mux(struct pinctrl_dev *pctldev, unsigned function, grp = &pctrl->community->groups[group]; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); /* Check first that the pad is not locked */ for (i = 0; i < grp->npins; i++) { if (chv_pad_locked(pctrl, grp->pins[i])) { dev_warn(pctrl->dev, "unable to set mode for locked pin %u\n", grp->pins[i]); - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return -EBUSY; } } @@ -839,7 +839,7 @@ static int chv_pinmux_set_mux(struct pinctrl_dev *pctldev, unsigned function, pin, altfunc->mode, altfunc->invert_oe ? "" : "not "); } - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return 0; } @@ -853,13 +853,13 @@ static int chv_gpio_request_enable(struct pinctrl_dev *pctldev, void __iomem *reg; u32 value; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); if (chv_pad_locked(pctrl, offset)) { value = readl(chv_padreg(pctrl, offset, CHV_PADCTRL0)); if (!(value & CHV_PADCTRL0_GPIOEN)) { /* Locked so cannot enable */ - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return -EBUSY; } } else { @@ -899,7 +899,7 @@ static int chv_gpio_request_enable(struct pinctrl_dev *pctldev, chv_writel(value, reg); } - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return 0; } @@ -913,13 +913,13 @@ static void chv_gpio_disable_free(struct pinctrl_dev *pctldev, void __iomem *reg; u32 value; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); reg = chv_padreg(pctrl, offset, CHV_PADCTRL0); value = readl(reg) & ~CHV_PADCTRL0_GPIOEN; chv_writel(value, reg); - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); } static int chv_gpio_set_direction(struct pinctrl_dev *pctldev, @@ -931,7 +931,7 @@ static int chv_gpio_set_direction(struct pinctrl_dev *pctldev, unsigned long flags; u32 ctrl0; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); ctrl0 = readl(reg) & ~CHV_PADCTRL0_GPIOCFG_MASK; if (input) @@ -940,7 +940,7 @@ static int chv_gpio_set_direction(struct pinctrl_dev *pctldev, ctrl0 |= CHV_PADCTRL0_GPIOCFG_GPO << CHV_PADCTRL0_GPIOCFG_SHIFT; chv_writel(ctrl0, reg); - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return 0; } @@ -965,10 +965,10 @@ static int chv_config_get(struct pinctrl_dev *pctldev, unsigned pin, u16 arg = 0; u32 term; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); ctrl0 = readl(chv_padreg(pctrl, pin, CHV_PADCTRL0)); ctrl1 = readl(chv_padreg(pctrl, pin, CHV_PADCTRL1)); - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); term = (ctrl0 & CHV_PADCTRL0_TERM_MASK) >> CHV_PADCTRL0_TERM_SHIFT; @@ -1042,7 +1042,7 @@ static int chv_config_set_pull(struct chv_pinctrl *pctrl, unsigned pin, unsigned long flags; u32 ctrl0, pull; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); ctrl0 = readl(reg); switch (param) { @@ -1065,7 +1065,7 @@ static int chv_config_set_pull(struct chv_pinctrl *pctrl, unsigned pin, pull = CHV_PADCTRL0_TERM_20K << CHV_PADCTRL0_TERM_SHIFT; break; default: - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return -EINVAL; } @@ -1083,7 +1083,7 @@ static int chv_config_set_pull(struct chv_pinctrl *pctrl, unsigned pin, pull = CHV_PADCTRL0_TERM_20K << CHV_PADCTRL0_TERM_SHIFT; break; default: - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return -EINVAL; } @@ -1091,12 +1091,12 @@ static int chv_config_set_pull(struct chv_pinctrl *pctrl, unsigned pin, break; default: - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return -EINVAL; } chv_writel(ctrl0, reg); - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return 0; } @@ -1169,9 +1169,12 @@ static int chv_gpio_get(struct gpio_chip *chip, unsigned offset) { struct chv_pinctrl *pctrl = gpiochip_to_pinctrl(chip); int pin = chv_gpio_offset_to_pin(pctrl, offset); + unsigned long flags; u32 ctrl0, cfg; + raw_spin_lock_irqsave(&pctrl->lock, flags); ctrl0 = readl(chv_padreg(pctrl, pin, CHV_PADCTRL0)); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); cfg = ctrl0 & CHV_PADCTRL0_GPIOCFG_MASK; cfg >>= CHV_PADCTRL0_GPIOCFG_SHIFT; @@ -1189,7 +1192,7 @@ static void chv_gpio_set(struct gpio_chip *chip, unsigned offset, int value) void __iomem *reg; u32 ctrl0; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); reg = chv_padreg(pctrl, pin, CHV_PADCTRL0); ctrl0 = readl(reg); @@ -1201,7 +1204,7 @@ static void chv_gpio_set(struct gpio_chip *chip, unsigned offset, int value) chv_writel(ctrl0, reg); - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); } static int chv_gpio_get_direction(struct gpio_chip *chip, unsigned offset) @@ -1209,8 +1212,11 @@ static int chv_gpio_get_direction(struct gpio_chip *chip, unsigned offset) struct chv_pinctrl *pctrl = gpiochip_to_pinctrl(chip); unsigned pin = chv_gpio_offset_to_pin(pctrl, offset); u32 ctrl0, direction; + unsigned long flags; + raw_spin_lock_irqsave(&pctrl->lock, flags); ctrl0 = readl(chv_padreg(pctrl, pin, CHV_PADCTRL0)); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); direction = ctrl0 & CHV_PADCTRL0_GPIOCFG_MASK; direction >>= CHV_PADCTRL0_GPIOCFG_SHIFT; @@ -1248,14 +1254,14 @@ static void chv_gpio_irq_ack(struct irq_data *d) int pin = chv_gpio_offset_to_pin(pctrl, irqd_to_hwirq(d)); u32 intr_line; - spin_lock(&pctrl->lock); + raw_spin_lock(&pctrl->lock); intr_line = readl(chv_padreg(pctrl, pin, CHV_PADCTRL0)); intr_line &= CHV_PADCTRL0_INTSEL_MASK; intr_line >>= CHV_PADCTRL0_INTSEL_SHIFT; chv_writel(BIT(intr_line), pctrl->regs + CHV_INTSTAT); - spin_unlock(&pctrl->lock); + raw_spin_unlock(&pctrl->lock); } static void chv_gpio_irq_mask_unmask(struct irq_data *d, bool mask) @@ -1266,7 +1272,7 @@ static void chv_gpio_irq_mask_unmask(struct irq_data *d, bool mask) u32 value, intr_line; unsigned long flags; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); intr_line = readl(chv_padreg(pctrl, pin, CHV_PADCTRL0)); intr_line &= CHV_PADCTRL0_INTSEL_MASK; @@ -1279,7 +1285,7 @@ static void chv_gpio_irq_mask_unmask(struct irq_data *d, bool mask) value |= BIT(intr_line); chv_writel(value, pctrl->regs + CHV_INTMASK); - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); } static void chv_gpio_irq_mask(struct irq_data *d) @@ -1313,6 +1319,7 @@ static unsigned chv_gpio_irq_startup(struct irq_data *d) unsigned long flags; u32 intsel, value; + raw_spin_lock_irqsave(&pctrl->lock, flags); intsel = readl(chv_padreg(pctrl, pin, CHV_PADCTRL0)); intsel &= CHV_PADCTRL0_INTSEL_MASK; intsel >>= CHV_PADCTRL0_INTSEL_SHIFT; @@ -1323,12 +1330,11 @@ static unsigned chv_gpio_irq_startup(struct irq_data *d) else handler = handle_edge_irq; - spin_lock_irqsave(&pctrl->lock, flags); if (!pctrl->intr_lines[intsel]) { - __irq_set_handler_locked(d->irq, handler); + irq_set_handler_locked(d, handler); pctrl->intr_lines[intsel] = offset; } - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); } chv_gpio_irq_unmask(d); @@ -1344,7 +1350,7 @@ static int chv_gpio_irq_type(struct irq_data *d, unsigned type) unsigned long flags; u32 value; - spin_lock_irqsave(&pctrl->lock, flags); + raw_spin_lock_irqsave(&pctrl->lock, flags); /* * Pins which can be used as shared interrupt are configured in @@ -1389,11 +1395,11 @@ static int chv_gpio_irq_type(struct irq_data *d, unsigned type) pctrl->intr_lines[value] = offset; if (type & IRQ_TYPE_EDGE_BOTH) - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); else if (type & IRQ_TYPE_LEVEL_MASK) - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); - spin_unlock_irqrestore(&pctrl->lock, flags); + raw_spin_unlock_irqrestore(&pctrl->lock, flags); return 0; } @@ -1412,7 +1418,7 @@ static void chv_gpio_irq_handler(unsigned irq, struct irq_desc *desc) { struct gpio_chip *gc = irq_desc_get_handler_data(desc); struct chv_pinctrl *pctrl = gpiochip_to_pinctrl(gc); - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); unsigned long pending; u32 intr_line; @@ -1505,7 +1511,7 @@ static int chv_pinctrl_probe(struct platform_device *pdev) if (i == ARRAY_SIZE(chv_communities)) return -ENODEV; - spin_lock_init(&pctrl->lock); + raw_spin_lock_init(&pctrl->lock); pctrl->dev = &pdev->dev; #ifdef CONFIG_PM_SLEEP diff --git a/drivers/pinctrl/intel/pinctrl-intel.c b/drivers/pinctrl/intel/pinctrl-intel.c index f9ee0d68b2888c..bb377c110541b3 100644 --- a/drivers/pinctrl/intel/pinctrl-intel.c +++ b/drivers/pinctrl/intel/pinctrl-intel.c @@ -758,9 +758,9 @@ static int intel_gpio_irq_type(struct irq_data *d, unsigned type) writel(value, reg); if (type & IRQ_TYPE_EDGE_BOTH) - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); else if (type & IRQ_TYPE_LEVEL_MASK) - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); spin_unlock_irqrestore(&pctrl->lock, flags); @@ -840,7 +840,7 @@ static void intel_gpio_irq_handler(unsigned irq, struct irq_desc *desc) { struct gpio_chip *gc = irq_desc_get_handler_data(desc); struct intel_pinctrl *pctrl = gpiochip_to_pinctrl(gc); - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); int i; chained_irq_enter(chip, desc); diff --git a/drivers/pinctrl/mediatek/pinctrl-mt8173.c b/drivers/pinctrl/mediatek/pinctrl-mt8173.c index d0c811d5f07b73..ad271840d8652f 100644 --- a/drivers/pinctrl/mediatek/pinctrl-mt8173.c +++ b/drivers/pinctrl/mediatek/pinctrl-mt8173.c @@ -385,6 +385,7 @@ static struct platform_driver mtk_pinctrl_driver = { .driver = { .name = "mediatek-mt8173-pinctrl", .of_match_table = mt8173_pctrl_match, + .pm = &mtk_eint_pm_ops, }, }; diff --git a/drivers/pinctrl/mediatek/pinctrl-mtk-common.c b/drivers/pinctrl/mediatek/pinctrl-mtk-common.c index ad1ea1695b4ae4..7726c6caaf8350 100644 --- a/drivers/pinctrl/mediatek/pinctrl-mtk-common.c +++ b/drivers/pinctrl/mediatek/pinctrl-mtk-common.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include "../core.h" @@ -702,7 +703,7 @@ static int mtk_pmx_set_mux(struct pinctrl_dev *pctldev, ret = mtk_pctrl_is_function_valid(pctl, g->pin, function); if (!ret) { - dev_err(pctl->dev, "invaild function %d on group %d .\n", + dev_err(pctl->dev, "invalid function %d on group %d .\n", function, group); return -EINVAL; } @@ -1062,6 +1063,77 @@ static int mtk_eint_set_type(struct irq_data *d, return 0; } +static int mtk_eint_irq_set_wake(struct irq_data *d, unsigned int on) +{ + struct mtk_pinctrl *pctl = irq_data_get_irq_chip_data(d); + int shift = d->hwirq & 0x1f; + int reg = d->hwirq >> 5; + + if (on) + pctl->wake_mask[reg] |= BIT(shift); + else + pctl->wake_mask[reg] &= ~BIT(shift); + + return 0; +} + +static void mtk_eint_chip_write_mask(const struct mtk_eint_offsets *chip, + void __iomem *eint_reg_base, u32 *buf) +{ + int port; + void __iomem *reg; + + for (port = 0; port < chip->ports; port++) { + reg = eint_reg_base + (port << 2); + writel_relaxed(~buf[port], reg + chip->mask_set); + writel_relaxed(buf[port], reg + chip->mask_clr); + } +} + +static void mtk_eint_chip_read_mask(const struct mtk_eint_offsets *chip, + void __iomem *eint_reg_base, u32 *buf) +{ + int port; + void __iomem *reg; + + for (port = 0; port < chip->ports; port++) { + reg = eint_reg_base + chip->mask + (port << 2); + buf[port] = ~readl_relaxed(reg); + /* Mask is 0 when irq is enabled, and 1 when disabled. */ + } +} + +static int mtk_eint_suspend(struct device *device) +{ + void __iomem *reg; + struct mtk_pinctrl *pctl = dev_get_drvdata(device); + const struct mtk_eint_offsets *eint_offsets = + &pctl->devdata->eint_offsets; + + reg = pctl->eint_reg_base; + mtk_eint_chip_read_mask(eint_offsets, reg, pctl->cur_mask); + mtk_eint_chip_write_mask(eint_offsets, reg, pctl->wake_mask); + + return 0; +} + +static int mtk_eint_resume(struct device *device) +{ + struct mtk_pinctrl *pctl = dev_get_drvdata(device); + const struct mtk_eint_offsets *eint_offsets = + &pctl->devdata->eint_offsets; + + mtk_eint_chip_write_mask(eint_offsets, + pctl->eint_reg_base, pctl->cur_mask); + + return 0; +} + +const struct dev_pm_ops mtk_eint_pm_ops = { + .suspend = mtk_eint_suspend, + .resume = mtk_eint_resume, +}; + static void mtk_eint_ack(struct irq_data *d) { struct mtk_pinctrl *pctl = irq_data_get_irq_chip_data(d); @@ -1076,10 +1148,12 @@ static void mtk_eint_ack(struct irq_data *d) static struct irq_chip mtk_pinctrl_irq_chip = { .name = "mt-eint", + .irq_disable = mtk_eint_mask, .irq_mask = mtk_eint_mask, .irq_unmask = mtk_eint_unmask, .irq_ack = mtk_eint_ack, .irq_set_type = mtk_eint_set_type, + .irq_set_wake = mtk_eint_irq_set_wake, .irq_request_resources = mtk_pinctrl_irq_request_resources, .irq_release_resources = mtk_pinctrl_irq_release_resources, }; @@ -1118,8 +1192,8 @@ mtk_eint_debounce_process(struct mtk_pinctrl *pctl, int index) static void mtk_eint_irq_handler(unsigned irq, struct irq_desc *desc) { - struct irq_chip *chip = irq_get_chip(irq); - struct mtk_pinctrl *pctl = irq_get_handler_data(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); + struct mtk_pinctrl *pctl = irq_desc_get_handler_data(desc); unsigned int status, eint_num; int offset, index, virq; const struct mtk_eint_offsets *eint_offsets = @@ -1202,12 +1276,6 @@ static int mtk_pctrl_build_state(struct platform_device *pdev) return 0; } -static struct pinctrl_desc mtk_pctrl_desc = { - .confops = &mtk_pconf_ops, - .pctlops = &mtk_pctrl_ops, - .pmxops = &mtk_pmx_ops, -}; - int mtk_pctrl_init(struct platform_device *pdev, const struct mtk_pinctrl_devdata *data, struct regmap *regmap) @@ -1217,7 +1285,7 @@ int mtk_pctrl_init(struct platform_device *pdev, struct device_node *np = pdev->dev.of_node, *node; struct property *prop; struct resource *res; - int i, ret, irq; + int i, ret, irq, ports_buf; pctl = devm_kzalloc(&pdev->dev, sizeof(*pctl), GFP_KERNEL); if (!pctl) @@ -1265,12 +1333,17 @@ int mtk_pctrl_init(struct platform_device *pdev, for (i = 0; i < pctl->devdata->npins; i++) pins[i] = pctl->devdata->pins[i].pin; - mtk_pctrl_desc.name = dev_name(&pdev->dev); - mtk_pctrl_desc.owner = THIS_MODULE; - mtk_pctrl_desc.pins = pins; - mtk_pctrl_desc.npins = pctl->devdata->npins; + + pctl->pctl_desc.name = dev_name(&pdev->dev); + pctl->pctl_desc.owner = THIS_MODULE; + pctl->pctl_desc.pins = pins; + pctl->pctl_desc.npins = pctl->devdata->npins; + pctl->pctl_desc.confops = &mtk_pconf_ops; + pctl->pctl_desc.pctlops = &mtk_pctrl_ops; + pctl->pctl_desc.pmxops = &mtk_pmx_ops; pctl->dev = &pdev->dev; - pctl->pctl_dev = pinctrl_register(&mtk_pctrl_desc, &pdev->dev, pctl); + + pctl->pctl_dev = pinctrl_register(&pctl->pctl_desc, &pdev->dev, pctl); if (IS_ERR(pctl->pctl_dev)) { dev_err(&pdev->dev, "couldn't register pinctrl driver\n"); return PTR_ERR(pctl->pctl_dev); @@ -1319,6 +1392,21 @@ int mtk_pctrl_init(struct platform_device *pdev, goto chip_error; } + ports_buf = pctl->devdata->eint_offsets.ports; + pctl->wake_mask = devm_kcalloc(&pdev->dev, ports_buf, + sizeof(*pctl->wake_mask), GFP_KERNEL); + if (!pctl->wake_mask) { + ret = -ENOMEM; + goto chip_error; + } + + pctl->cur_mask = devm_kcalloc(&pdev->dev, ports_buf, + sizeof(*pctl->cur_mask), GFP_KERNEL); + if (!pctl->cur_mask) { + ret = -ENOMEM; + goto chip_error; + } + pctl->eint_dual_edges = devm_kcalloc(&pdev->dev, pctl->devdata->ap_num, sizeof(int), GFP_KERNEL); if (!pctl->eint_dual_edges) { @@ -1348,11 +1436,9 @@ int mtk_pctrl_init(struct platform_device *pdev, irq_set_chip_and_handler(virq, &mtk_pinctrl_irq_chip, handle_level_irq); irq_set_chip_data(virq, pctl); - set_irq_flags(virq, IRQF_VALID); }; irq_set_chained_handler_and_data(irq, mtk_eint_irq_handler, pctl); - set_irq_flags(irq, IRQF_VALID); return 0; chip_error: diff --git a/drivers/pinctrl/mediatek/pinctrl-mtk-common.h b/drivers/pinctrl/mediatek/pinctrl-mtk-common.h index 30213e514c2f1e..55a5343389315e 100644 --- a/drivers/pinctrl/mediatek/pinctrl-mtk-common.h +++ b/drivers/pinctrl/mediatek/pinctrl-mtk-common.h @@ -256,6 +256,7 @@ struct mtk_pinctrl_devdata { struct mtk_pinctrl { struct regmap *regmap1; struct regmap *regmap2; + struct pinctrl_desc pctl_desc; struct device *dev; struct gpio_chip *chip; struct mtk_pinctrl_group *groups; @@ -266,6 +267,8 @@ struct mtk_pinctrl { void __iomem *eint_reg_base; struct irq_domain *domain; int *eint_dual_edges; + u32 *wake_mask; + u32 *cur_mask; }; int mtk_pctrl_init(struct platform_device *pdev, @@ -281,4 +284,6 @@ int mtk_pconf_spec_set_ies_smt_range(struct regmap *regmap, const struct mtk_pin_ies_smt_set *ies_smt_infos, unsigned int info_num, unsigned int pin, unsigned char align, int value); +extern const struct dev_pm_ops mtk_eint_pm_ops; + #endif /* __PINCTRL_MTK_COMMON_H */ diff --git a/drivers/pinctrl/nomadik/pinctrl-nomadik-db8500.c b/drivers/pinctrl/nomadik/pinctrl-nomadik-db8500.c index c74840729648e0..8392083514fb63 100644 --- a/drivers/pinctrl/nomadik/pinctrl-nomadik-db8500.c +++ b/drivers/pinctrl/nomadik/pinctrl-nomadik-db8500.c @@ -355,25 +355,6 @@ static const struct pinctrl_pin_desc nmk_db8500_pins[] = { PINCTRL_PIN(DB8500_PIN_AC27, "GPIO267_AC27"), }; -#define DB8500_GPIO_RANGE(a, b, c) { .name = "DB8500", .id = a, .base = b, \ - .pin_base = b, .npins = c } - -/* - * This matches the 32-pin gpio chips registered by the GPIO portion. This - * cannot be const since we assign the struct gpio_chip * pointer at runtime. - */ -static struct pinctrl_gpio_range nmk_db8500_ranges[] = { - DB8500_GPIO_RANGE(0, 0, 32), - DB8500_GPIO_RANGE(1, 32, 5), - DB8500_GPIO_RANGE(2, 64, 32), - DB8500_GPIO_RANGE(3, 96, 2), - DB8500_GPIO_RANGE(4, 128, 32), - DB8500_GPIO_RANGE(5, 160, 12), - DB8500_GPIO_RANGE(6, 192, 32), - DB8500_GPIO_RANGE(7, 224, 7), - DB8500_GPIO_RANGE(8, 256, 12), -}; - /* * Read the pin group names like this: * u0_a_1 = first groups of pins for uart0 on alt function a @@ -1238,8 +1219,6 @@ static const u16 db8500_prcm_gpiocr_regs[] = { }; static const struct nmk_pinctrl_soc_data nmk_db8500_soc = { - .gpio_ranges = nmk_db8500_ranges, - .gpio_num_ranges = ARRAY_SIZE(nmk_db8500_ranges), .pins = nmk_db8500_pins, .npins = ARRAY_SIZE(nmk_db8500_pins), .functions = nmk_db8500_functions, diff --git a/drivers/pinctrl/nomadik/pinctrl-nomadik-db8540.c b/drivers/pinctrl/nomadik/pinctrl-nomadik-db8540.c index d7ba5443bae055..2860eafd1b4200 100644 --- a/drivers/pinctrl/nomadik/pinctrl-nomadik-db8540.c +++ b/drivers/pinctrl/nomadik/pinctrl-nomadik-db8540.c @@ -341,28 +341,6 @@ static const struct pinctrl_pin_desc nmk_db8540_pins[] = { PINCTRL_PIN(DB8540_PIN_D17, "GPIO267_D17"), }; -#define DB8540_GPIO_RANGE(a, b, c) { .name = "db8540", .id = a, .base = b, \ - .pin_base = b, .npins = c } - -/* - * This matches the 32-pin gpio chips registered by the GPIO portion. This - * cannot be const since we assign the struct gpio_chip * pointer at runtime. - */ -static struct pinctrl_gpio_range nmk_db8540_ranges[] = { - DB8540_GPIO_RANGE(0, 0, 18), - DB8540_GPIO_RANGE(0, 22, 7), - DB8540_GPIO_RANGE(1, 33, 6), - DB8540_GPIO_RANGE(2, 64, 4), - DB8540_GPIO_RANGE(2, 70, 18), - DB8540_GPIO_RANGE(3, 116, 12), - DB8540_GPIO_RANGE(4, 128, 32), - DB8540_GPIO_RANGE(5, 160, 9), - DB8540_GPIO_RANGE(6, 192, 23), - DB8540_GPIO_RANGE(6, 219, 5), - DB8540_GPIO_RANGE(7, 224, 9), - DB8540_GPIO_RANGE(8, 256, 12), -}; - /* * Read the pin group names like this: * u0_a_1 = first groups of pins for uart0 on alt function a @@ -1247,8 +1225,6 @@ static const u16 db8540_prcm_gpiocr_regs[] = { }; static const struct nmk_pinctrl_soc_data nmk_db8540_soc = { - .gpio_ranges = nmk_db8540_ranges, - .gpio_num_ranges = ARRAY_SIZE(nmk_db8540_ranges), .pins = nmk_db8540_pins, .npins = ARRAY_SIZE(nmk_db8540_pins), .functions = nmk_db8540_functions, diff --git a/drivers/pinctrl/nomadik/pinctrl-nomadik-stn8815.c b/drivers/pinctrl/nomadik/pinctrl-nomadik-stn8815.c index 2cd71470f27007..587b222f12f3cf 100644 --- a/drivers/pinctrl/nomadik/pinctrl-nomadik-stn8815.c +++ b/drivers/pinctrl/nomadik/pinctrl-nomadik-stn8815.c @@ -264,20 +264,6 @@ static const struct pinctrl_pin_desc nmk_stn8815_pins[] = { PINCTRL_PIN(STN8815_PIN_J22, "GPIO123_J22"), }; -#define STN8815_GPIO_RANGE(a, b, c) { .name = "STN8815", .id = a, .base = b, \ - .pin_base = b, .npins = c } - -/* - * This matches the 32-pin gpio chips registered by the GPIO portion. This - * cannot be const since we assign the struct gpio_chip * pointer at runtime. - */ -static struct pinctrl_gpio_range nmk_stn8815_ranges[] = { - STN8815_GPIO_RANGE(0, 0, 32), - STN8815_GPIO_RANGE(1, 32, 32), - STN8815_GPIO_RANGE(2, 64, 32), - STN8815_GPIO_RANGE(3, 96, 28), -}; - /* * Read the pin group names like this: * u0_a_1 = first groups of pins for uart0 on alt function a @@ -285,9 +271,11 @@ static struct pinctrl_gpio_range nmk_stn8815_ranges[] = { */ /* Altfunction A */ -static const unsigned u0_a_1_pins[] = { STN8815_PIN_B4, STN8815_PIN_D5, - STN8815_PIN_C5, STN8815_PIN_A4, STN8815_PIN_B5, STN8815_PIN_D6, - STN8815_PIN_C6, STN8815_PIN_B6 }; +static const unsigned u0txrx_a_1_pins[] = { STN8815_PIN_B4, STN8815_PIN_D5 }; +static const unsigned u0ctsrts_a_1_pins[] = { STN8815_PIN_C5, STN8815_PIN_B6 }; +/* Modem pins: DCD, DSR, RI, DTR */ +static const unsigned u0modem_a_1_pins[] = { STN8815_PIN_A4, STN8815_PIN_B5, + STN8815_PIN_D6, STN8815_PIN_C6 }; static const unsigned mmcsd_a_1_pins[] = { STN8815_PIN_B10, STN8815_PIN_A10, STN8815_PIN_C11, STN8815_PIN_B11, STN8815_PIN_A11, STN8815_PIN_C12, STN8815_PIN_B12, STN8815_PIN_A12, STN8815_PIN_C13, STN8815_PIN_C15 }; @@ -304,7 +292,9 @@ static const unsigned i2cusb_b_1_pins[] = { STN8815_PIN_C21, STN8815_PIN_C20 }; .npins = ARRAY_SIZE(a##_pins), .altsetting = b } static const struct nmk_pingroup nmk_stn8815_groups[] = { - STN8815_PIN_GROUP(u0_a_1, NMK_GPIO_ALT_A), + STN8815_PIN_GROUP(u0txrx_a_1, NMK_GPIO_ALT_A), + STN8815_PIN_GROUP(u0ctsrts_a_1, NMK_GPIO_ALT_A), + STN8815_PIN_GROUP(u0modem_a_1, NMK_GPIO_ALT_A), STN8815_PIN_GROUP(mmcsd_a_1, NMK_GPIO_ALT_A), STN8815_PIN_GROUP(mmcsd_b_1, NMK_GPIO_ALT_B), STN8815_PIN_GROUP(u1_a_1, NMK_GPIO_ALT_A), @@ -318,7 +308,7 @@ static const struct nmk_pingroup nmk_stn8815_groups[] = { #define STN8815_FUNC_GROUPS(a, b...) \ static const char * const a##_groups[] = { b }; -STN8815_FUNC_GROUPS(u0, "u0_a_1"); +STN8815_FUNC_GROUPS(u0, "u0txrx_a_1", "u0ctsrts_a_1", "u0modem_a_1"); STN8815_FUNC_GROUPS(mmcsd, "mmcsd_a_1", "mmcsd_b_1"); STN8815_FUNC_GROUPS(u1, "u1_a_1", "u1_b_1"); STN8815_FUNC_GROUPS(i2c1, "i2c1_a_1"); @@ -342,8 +332,6 @@ static const struct nmk_function nmk_stn8815_functions[] = { }; static const struct nmk_pinctrl_soc_data nmk_stn8815_soc = { - .gpio_ranges = nmk_stn8815_ranges, - .gpio_num_ranges = ARRAY_SIZE(nmk_stn8815_ranges), .pins = nmk_stn8815_pins, .npins = ARRAY_SIZE(nmk_stn8815_pins), .functions = nmk_stn8815_functions, diff --git a/drivers/pinctrl/nomadik/pinctrl-nomadik.c b/drivers/pinctrl/nomadik/pinctrl-nomadik.c index 809d88445db5aa..352ede13a9e9a3 100644 --- a/drivers/pinctrl/nomadik/pinctrl-nomadik.c +++ b/drivers/pinctrl/nomadik/pinctrl-nomadik.c @@ -203,6 +203,7 @@ typedef unsigned long pin_cfg_t; #define GPIO_BLOCK_SHIFT 5 #define NMK_GPIO_PER_CHIP (1 << GPIO_BLOCK_SHIFT) +#define NMK_MAX_BANKS DIV_ROUND_UP(ARCH_NR_GPIOS, NMK_GPIO_PER_CHIP) /* Register in the logic block */ #define NMK_GPIO_DAT 0x00 @@ -282,8 +283,7 @@ struct nmk_pinctrl { void __iomem *prcm_base; }; -static struct nmk_gpio_chip * -nmk_gpio_chips[DIV_ROUND_UP(ARCH_NR_GPIOS, NMK_GPIO_PER_CHIP)]; +static struct nmk_gpio_chip *nmk_gpio_chips[NMK_MAX_BANKS]; static DEFINE_SPINLOCK(nmk_gpio_slpm_lock); @@ -843,10 +843,9 @@ static void nmk_gpio_irq_shutdown(struct irq_data *d) clk_disable(nmk_chip->clk); } -static void __nmk_gpio_irq_handler(unsigned int irq, struct irq_desc *desc, - u32 status) +static void __nmk_gpio_irq_handler(struct irq_desc *desc, u32 status) { - struct irq_chip *host_chip = irq_get_chip(irq); + struct irq_chip *host_chip = irq_desc_get_chip(desc); struct gpio_chip *chip = irq_desc_get_handler_data(desc); chained_irq_enter(host_chip, desc); @@ -871,17 +870,16 @@ static void nmk_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) status = readl(nmk_chip->addr + NMK_GPIO_IS); clk_disable(nmk_chip->clk); - __nmk_gpio_irq_handler(irq, desc, status); + __nmk_gpio_irq_handler(desc, status); } -static void nmk_gpio_latent_irq_handler(unsigned int irq, - struct irq_desc *desc) +static void nmk_gpio_latent_irq_handler(unsigned int irq, struct irq_desc *desc) { struct gpio_chip *chip = irq_desc_get_handler_data(desc); struct nmk_gpio_chip *nmk_chip = container_of(chip, struct nmk_gpio_chip, chip); u32 status = nmk_chip->get_latent_status(nmk_chip->bank); - __nmk_gpio_irq_handler(irq, desc, status); + __nmk_gpio_irq_handler(desc, status); } /* I/O Functions */ @@ -1012,6 +1010,7 @@ static void nmk_gpio_dbg_show_one(struct seq_file *s, int irq = gpio_to_irq(gpio); struct irq_desc *desc = irq_to_desc(irq); int pullidx = 0; + int val; if (pull) pullidx = data_out ? 1 : 2; @@ -1021,6 +1020,10 @@ static void nmk_gpio_dbg_show_one(struct seq_file *s, label ?: "(none)", pulls[pullidx], (mode < 0) ? "unknown" : modes[mode]); + + val = nmk_gpio_get_input(chip, offset); + seq_printf(s, " VAL %d", val); + /* * This races with request_irq(), set_irq_type(), * and set_irq_wake() ... but those are "rare". @@ -1162,29 +1165,90 @@ void nmk_gpio_read_pull(int gpio_bank, u32 *pull_up) } } +/* + * We will allocate memory for the state container using devm* allocators + * binding to the first device reaching this point, it doesn't matter if + * it is the pin controller or GPIO driver. However we need to use the right + * platform device when looking up resources so pay attention to pdev. + */ +static struct nmk_gpio_chip *nmk_gpio_populate_chip(struct device_node *np, + struct platform_device *pdev) +{ + struct nmk_gpio_chip *nmk_chip; + struct platform_device *gpio_pdev; + struct gpio_chip *chip; + struct resource *res; + struct clk *clk; + void __iomem *base; + u32 id; + + gpio_pdev = of_find_device_by_node(np); + if (!gpio_pdev) { + pr_err("populate \"%s\": device not found\n", np->name); + return ERR_PTR(-ENODEV); + } + if (of_property_read_u32(np, "gpio-bank", &id)) { + dev_err(&pdev->dev, "populate: gpio-bank property not found\n"); + return ERR_PTR(-EINVAL); + } + + /* Already populated? */ + nmk_chip = nmk_gpio_chips[id]; + if (nmk_chip) + return nmk_chip; + + nmk_chip = devm_kzalloc(&pdev->dev, sizeof(*nmk_chip), GFP_KERNEL); + if (!nmk_chip) + return ERR_PTR(-ENOMEM); + + nmk_chip->bank = id; + chip = &nmk_chip->chip; + chip->base = id * NMK_GPIO_PER_CHIP; + chip->ngpio = NMK_GPIO_PER_CHIP; + chip->label = dev_name(&gpio_pdev->dev); + chip->dev = &gpio_pdev->dev; + + res = platform_get_resource(gpio_pdev, IORESOURCE_MEM, 0); + base = devm_ioremap_resource(&pdev->dev, res); + if (IS_ERR(base)) + return base; + nmk_chip->addr = base; + + clk = clk_get(&gpio_pdev->dev, NULL); + if (IS_ERR(clk)) + return (void *) clk; + clk_prepare(clk); + nmk_chip->clk = clk; + + BUG_ON(nmk_chip->bank >= ARRAY_SIZE(nmk_gpio_chips)); + nmk_gpio_chips[id] = nmk_chip; + return nmk_chip; +} + static int nmk_gpio_probe(struct platform_device *dev) { struct device_node *np = dev->dev.of_node; struct nmk_gpio_chip *nmk_chip; struct gpio_chip *chip; struct irq_chip *irqchip; - struct resource *res; - struct clk *clk; int latent_irq; bool supports_sleepmode; - void __iomem *base; int irq; int ret; + nmk_chip = nmk_gpio_populate_chip(np, dev); + if (IS_ERR(nmk_chip)) { + dev_err(&dev->dev, "could not populate nmk chip struct\n"); + return PTR_ERR(nmk_chip); + } + if (of_get_property(np, "st,supports-sleepmode", NULL)) supports_sleepmode = true; else supports_sleepmode = false; - if (of_property_read_u32(np, "gpio-bank", &dev->id)) { - dev_err(&dev->dev, "gpio-bank property not found\n"); - return -EINVAL; - } + /* Correct platform device ID */ + dev->id = nmk_chip->bank; irq = platform_get_irq(dev, 0); if (irq < 0) @@ -1193,27 +1257,10 @@ static int nmk_gpio_probe(struct platform_device *dev) /* It's OK for this IRQ not to be present */ latent_irq = platform_get_irq(dev, 1); - res = platform_get_resource(dev, IORESOURCE_MEM, 0); - base = devm_ioremap_resource(&dev->dev, res); - if (IS_ERR(base)) - return PTR_ERR(base); - - clk = devm_clk_get(&dev->dev, NULL); - if (IS_ERR(clk)) - return PTR_ERR(clk); - clk_prepare(clk); - - nmk_chip = devm_kzalloc(&dev->dev, sizeof(*nmk_chip), GFP_KERNEL); - if (!nmk_chip) - return -ENOMEM; - /* * The virt address in nmk_chip->addr is in the nomadik register space, * so we can simply convert the resource address, without remapping */ - nmk_chip->bank = dev->id; - nmk_chip->clk = clk; - nmk_chip->addr = base; nmk_chip->parent_irq = irq; nmk_chip->latent_parent_irq = latent_irq; nmk_chip->sleepmode = supports_sleepmode; @@ -1228,10 +1275,6 @@ static int nmk_gpio_probe(struct platform_device *dev) chip->set = nmk_gpio_set_output; chip->dbg_show = nmk_gpio_dbg_show; chip->can_sleep = false; - chip->base = dev->id * NMK_GPIO_PER_CHIP; - chip->ngpio = NMK_GPIO_PER_CHIP; - chip->label = dev_name(&dev->dev); - chip->dev = &dev->dev; chip->owner = THIS_MODULE; irqchip = &nmk_chip->irqchip; @@ -1253,14 +1296,10 @@ static int nmk_gpio_probe(struct platform_device *dev) clk_disable(nmk_chip->clk); chip->of_node = np; - ret = gpiochip_add(&nmk_chip->chip); + ret = gpiochip_add(chip); if (ret) return ret; - BUG_ON(nmk_chip->bank >= ARRAY_SIZE(nmk_gpio_chips)); - - nmk_gpio_chips[nmk_chip->bank] = nmk_chip; - platform_set_drvdata(dev, nmk_chip); /* @@ -1320,35 +1359,40 @@ static int nmk_get_group_pins(struct pinctrl_dev *pctldev, unsigned selector, return 0; } -static struct pinctrl_gpio_range * -nmk_match_gpio_range(struct pinctrl_dev *pctldev, unsigned offset) +static struct nmk_gpio_chip *find_nmk_gpio_from_pin(unsigned pin) { - struct nmk_pinctrl *npct = pinctrl_dev_get_drvdata(pctldev); int i; + struct nmk_gpio_chip *nmk_gpio; - for (i = 0; i < npct->soc->gpio_num_ranges; i++) { - struct pinctrl_gpio_range *range; - - range = &npct->soc->gpio_ranges[i]; - if (offset >= range->pin_base && - offset <= (range->pin_base + range->npins - 1)) - return range; + for(i = 0; i < NMK_MAX_BANKS; i++) { + nmk_gpio = nmk_gpio_chips[i]; + if (!nmk_gpio) + continue; + if (pin >= nmk_gpio->chip.base && + pin < nmk_gpio->chip.base + nmk_gpio->chip.ngpio) + return nmk_gpio; } return NULL; } +static struct gpio_chip *find_gc_from_pin(unsigned pin) +{ + struct nmk_gpio_chip *nmk_gpio = find_nmk_gpio_from_pin(pin); + + if (nmk_gpio) + return &nmk_gpio->chip; + return NULL; +} + static void nmk_pin_dbg_show(struct pinctrl_dev *pctldev, struct seq_file *s, unsigned offset) { - struct pinctrl_gpio_range *range; - struct gpio_chip *chip; + struct gpio_chip *chip = find_gc_from_pin(offset); - range = nmk_match_gpio_range(pctldev, offset); - if (!range || !range->gc) { + if (!chip) { seq_printf(s, "invalid pin offset"); return; } - chip = range->gc; nmk_gpio_dbg_show_one(s, pctldev, chip, offset - chip->base, offset); } @@ -1693,25 +1737,16 @@ static int nmk_pmx_set(struct pinctrl_dev *pctldev, unsigned function, } for (i = 0; i < g->npins; i++) { - struct pinctrl_gpio_range *range; struct nmk_gpio_chip *nmk_chip; - struct gpio_chip *chip; unsigned bit; - range = nmk_match_gpio_range(pctldev, g->pins[i]); - if (!range) { + nmk_chip = find_nmk_gpio_from_pin(g->pins[i]); + if (!nmk_chip) { dev_err(npct->dev, "invalid pin offset %d in group %s at index %d\n", g->pins[i], g->name, i); goto out_glitch; } - if (!range->gc) { - dev_err(npct->dev, "GPIO chip missing in range for pin offset %d in group %s at index %d\n", - g->pins[i], g->name, i); - goto out_glitch; - } - chip = range->gc; - nmk_chip = container_of(chip, struct nmk_gpio_chip, chip); dev_dbg(npct->dev, "setting pin %d to altsetting %d\n", g->pins[i], g->altsetting); clk_enable(nmk_chip->clk); @@ -1827,25 +1862,17 @@ static int nmk_pin_config_set(struct pinctrl_dev *pctldev, unsigned pin, }; struct nmk_pinctrl *npct = pinctrl_dev_get_drvdata(pctldev); struct nmk_gpio_chip *nmk_chip; - struct pinctrl_gpio_range *range; - struct gpio_chip *chip; unsigned bit; pin_cfg_t cfg; int pull, slpm, output, val, i; bool lowemi, gpiomode, sleep; - range = nmk_match_gpio_range(pctldev, pin); - if (!range) { - dev_err(npct->dev, "invalid pin offset %d\n", pin); - return -EINVAL; - } - if (!range->gc) { - dev_err(npct->dev, "GPIO chip missing in range for pin %d\n", - pin); + nmk_chip = find_nmk_gpio_from_pin(pin); + if (!nmk_chip) { + dev_err(npct->dev, + "invalid pin offset %d\n", pin); return -EINVAL; } - chip = range->gc; - nmk_chip = container_of(chip, struct nmk_gpio_chip, chip); for (i = 0; i < num_configs; i++) { /* @@ -1997,6 +2024,31 @@ static int nmk_pinctrl_probe(struct platform_device *pdev) if (version == PINCTRL_NMK_DB8540) nmk_pinctrl_db8540_init(&npct->soc); + /* + * Since we depend on the GPIO chips to provide clock and register base + * for the pin control operations, make sure that we have these + * populated before we continue. Follow the phandles to instantiate + * them. The GPIO portion of the actual hardware may be probed before + * or after this point: it shouldn't matter as the APIs are orthogonal. + */ + for (i = 0; i < NMK_MAX_BANKS; i++) { + struct device_node *gpio_np; + struct nmk_gpio_chip *nmk_chip; + + gpio_np = of_parse_phandle(np, "nomadik-gpio-chips", i); + if (gpio_np) { + dev_info(&pdev->dev, + "populate NMK GPIO %d \"%s\"\n", + i, gpio_np->name); + nmk_chip = nmk_gpio_populate_chip(gpio_np, pdev); + if (IS_ERR(nmk_chip)) + dev_err(&pdev->dev, + "could not populate nmk chip struct " + "- continue anyway\n"); + of_node_put(gpio_np); + } + } + prcm_np = of_parse_phandle(np, "prcm", 0); if (prcm_np) npct->prcm_base = of_iomap(prcm_np, 0); @@ -2011,19 +2063,6 @@ static int nmk_pinctrl_probe(struct platform_device *pdev) } } - /* - * We need all the GPIO drivers to probe FIRST, or we will not be able - * to obtain references to the struct gpio_chip * for them, and we - * need this to proceed. - */ - for (i = 0; i < npct->soc->gpio_num_ranges; i++) { - if (!nmk_gpio_chips[npct->soc->gpio_ranges[i].id]) { - dev_warn(&pdev->dev, "GPIO chip %d not registered yet\n", i); - return -EPROBE_DEFER; - } - npct->soc->gpio_ranges[i].gc = &nmk_gpio_chips[npct->soc->gpio_ranges[i].id]->chip; - } - nmk_pinctrl_desc.pins = npct->soc->pins; nmk_pinctrl_desc.npins = npct->soc->npins; npct->dev = &pdev->dev; @@ -2034,10 +2073,6 @@ static int nmk_pinctrl_probe(struct platform_device *pdev) return PTR_ERR(npct->pctl); } - /* We will handle a range of GPIO pins */ - for (i = 0; i < npct->soc->gpio_num_ranges; i++) - pinctrl_add_gpio_range(npct->pctl, &npct->soc->gpio_ranges[i]); - platform_set_drvdata(pdev, npct); dev_info(&pdev->dev, "initialized Nomadik pin control driver\n"); @@ -2072,15 +2107,15 @@ static struct platform_driver nmk_pinctrl_driver = { static int __init nmk_gpio_init(void) { - int ret; + return platform_driver_register(&nmk_gpio_driver); +} +subsys_initcall(nmk_gpio_init); - ret = platform_driver_register(&nmk_gpio_driver); - if (ret) - return ret; +static int __init nmk_pinctrl_init(void) +{ return platform_driver_register(&nmk_pinctrl_driver); } - -core_initcall(nmk_gpio_init); +core_initcall(nmk_pinctrl_init); MODULE_AUTHOR("Prafulla WADASKAR and Alessandro Rubini"); MODULE_DESCRIPTION("Nomadik GPIO Driver"); diff --git a/drivers/pinctrl/nomadik/pinctrl-nomadik.h b/drivers/pinctrl/nomadik/pinctrl-nomadik.h index d8215f1e70c747..30bba2a75a58aa 100644 --- a/drivers/pinctrl/nomadik/pinctrl-nomadik.h +++ b/drivers/pinctrl/nomadik/pinctrl-nomadik.h @@ -121,8 +121,6 @@ struct nmk_pingroup { /** * struct nmk_pinctrl_soc_data - Nomadik pin controller per-SoC configuration - * @gpio_ranges: An array of GPIO ranges for this SoC - * @gpio_num_ranges: The number of GPIO ranges for this SoC * @pins: An array describing all pins the pin controller affects. * All pins which are also GPIOs must be listed first within the * array, and be numbered identically to the GPIO controller's @@ -137,8 +135,6 @@ struct nmk_pingroup { * @prcm_gpiocr_registers: The array of PRCM GPIOCR registers on this SoC */ struct nmk_pinctrl_soc_data { - struct pinctrl_gpio_range *gpio_ranges; - unsigned gpio_num_ranges; const struct pinctrl_pin_desc *pins; unsigned npins; const struct nmk_function *functions; diff --git a/drivers/pinctrl/pinconf.c b/drivers/pinctrl/pinconf.c index 1fc09dc2019905..29a7bb17a42f52 100644 --- a/drivers/pinctrl/pinconf.c +++ b/drivers/pinctrl/pinconf.c @@ -61,8 +61,8 @@ int pin_config_get_for_pin(struct pinctrl_dev *pctldev, unsigned pin, const struct pinconf_ops *ops = pctldev->desc->confops; if (!ops || !ops->pin_config_get) { - dev_dbg(pctldev->dev, "cannot get pin configuration, missing " - "pin_config_get() function in driver\n"); + dev_dbg(pctldev->dev, + "cannot get pin configuration, .pin_config_get missing in driver\n"); return -ENOTSUPP; } @@ -202,18 +202,34 @@ int pinconf_apply_setting(struct pinctrl_setting const *setting) #ifdef CONFIG_DEBUG_FS -void pinconf_show_map(struct seq_file *s, struct pinctrl_map const *map) +static void pinconf_show_config(struct seq_file *s, struct pinctrl_dev *pctldev, + unsigned long *configs, unsigned num_configs) { - struct pinctrl_dev *pctldev; const struct pinconf_ops *confops; int i; - pctldev = get_pinctrl_dev_from_devname(map->ctrl_dev_name); if (pctldev) confops = pctldev->desc->confops; else confops = NULL; + for (i = 0; i < num_configs; i++) { + seq_puts(s, "config "); + if (confops && confops->pin_config_config_dbg_show) + confops->pin_config_config_dbg_show(pctldev, s, + configs[i]); + else + seq_printf(s, "%08lx", configs[i]); + seq_puts(s, "\n"); + } +} + +void pinconf_show_map(struct seq_file *s, struct pinctrl_map const *map) +{ + struct pinctrl_dev *pctldev; + + pctldev = get_pinctrl_dev_from_devname(map->ctrl_dev_name); + switch (map->type) { case PIN_MAP_TYPE_CONFIGS_PIN: seq_printf(s, "pin "); @@ -227,15 +243,8 @@ void pinconf_show_map(struct seq_file *s, struct pinctrl_map const *map) seq_printf(s, "%s\n", map->data.configs.group_or_pin); - for (i = 0; i < map->data.configs.num_configs; i++) { - seq_printf(s, "config "); - if (confops && confops->pin_config_config_dbg_show) - confops->pin_config_config_dbg_show(pctldev, s, - map->data.configs.configs[i]); - else - seq_printf(s, "%08lx", map->data.configs.configs[i]); - seq_printf(s, "\n"); - } + pinconf_show_config(s, pctldev, map->data.configs.configs, + map->data.configs.num_configs); } void pinconf_show_setting(struct seq_file *s, @@ -243,9 +252,7 @@ void pinconf_show_setting(struct seq_file *s, { struct pinctrl_dev *pctldev = setting->pctldev; const struct pinctrl_ops *pctlops = pctldev->desc->pctlops; - const struct pinconf_ops *confops = pctldev->desc->confops; struct pin_desc *desc; - int i; switch (setting->type) { case PIN_MAP_TYPE_CONFIGS_PIN: @@ -269,17 +276,8 @@ void pinconf_show_setting(struct seq_file *s, * FIXME: We should really get the pin controler to dump the config * values, so they can be decoded to something meaningful. */ - for (i = 0; i < setting->data.configs.num_configs; i++) { - seq_printf(s, " "); - if (confops && confops->pin_config_config_dbg_show) - confops->pin_config_config_dbg_show(pctldev, s, - setting->data.configs.configs[i]); - else - seq_printf(s, "%08lx", - setting->data.configs.configs[i]); - } - - seq_printf(s, "\n"); + pinconf_show_config(s, pctldev, setting->data.configs.configs, + setting->data.configs.num_configs); } static void pinconf_dump_pin(struct pinctrl_dev *pctldev, @@ -412,10 +410,8 @@ static int pinconf_dbg_config_print(struct seq_file *s, void *d) const struct pinctrl_map *map; const struct pinctrl_map *found = NULL; struct pinctrl_dev *pctldev; - const struct pinconf_ops *confops = NULL; struct dbg_cfg *dbg = &pinconf_dbg_conf; int i, j; - unsigned long config; mutex_lock(&pinctrl_maps_mutex); @@ -449,16 +445,10 @@ static int pinconf_dbg_config_print(struct seq_file *s, void *d) } pctldev = get_pinctrl_dev_from_devname(found->ctrl_dev_name); - config = *found->data.configs.configs; - seq_printf(s, "Dev %s has config of %s in state %s: 0x%08lX\n", - dbg->dev_name, dbg->pin_name, - dbg->state_name, config); - - if (pctldev) - confops = pctldev->desc->confops; - - if (confops && confops->pin_config_config_dbg_show) - confops->pin_config_config_dbg_show(pctldev, s, config); + seq_printf(s, "Dev %s has config of %s in state %s:\n", + dbg->dev_name, dbg->pin_name, dbg->state_name); + pinconf_show_config(s, pctldev, found->data.configs.configs, + found->data.configs.num_configs); exit: mutex_unlock(&pinctrl_maps_mutex); @@ -470,10 +460,12 @@ static int pinconf_dbg_config_print(struct seq_file *s, void *d) * pinconf_dbg_config_write() - modify the pinctrl config in the pinctrl * map, of a dev/pin/state entry based on user entries to pinconf-config * @user_buf: contains the modification request with expected format: - * modify config_pin + * modify * modify is literal string, alternatives like add/delete not supported yet - * config_pin is literal, alternatives like config_mux not supported yet - * are values that should match the pinctrl-maps + * is the configuration to be changed. Supported configs are + * "config_pin" or "config_group", alternatives like config_mux are not + * supported yet. + * are values that should match the pinctrl-maps * reflects the new config and is driver dependant */ static ssize_t pinconf_dbg_config_write(struct file *file, @@ -511,13 +503,19 @@ static ssize_t pinconf_dbg_config_write(struct file *file, if (strcmp(token, "modify")) return -EINVAL; - /* Get arg type: "config_pin" type supported so far */ + /* + * Get arg type: "config_pin" and "config_group" + * types are supported so far + */ token = strsep(&b, " "); if (!token) return -EINVAL; - if (strcmp(token, "config_pin")) + if (!strcmp(token, "config_pin")) + dbg->map_type = PIN_MAP_TYPE_CONFIGS_PIN; + else if (!strcmp(token, "config_group")) + dbg->map_type = PIN_MAP_TYPE_CONFIGS_GROUP; + else return -EINVAL; - dbg->map_type = PIN_MAP_TYPE_CONFIGS_PIN; /* get arg 'device_name' */ token = strsep(&b, " "); diff --git a/drivers/pinctrl/pinctrl-adi2-bf60x.c b/drivers/pinctrl/pinctrl-adi2-bf60x.c index 4cb59fe9be7039..fcfa00821f1221 100644 --- a/drivers/pinctrl/pinctrl-adi2-bf60x.c +++ b/drivers/pinctrl/pinctrl-adi2-bf60x.c @@ -394,25 +394,25 @@ static const unsigned short ppi2_16b_mux[] = { static const unsigned short lp0_mux[] = { P_LP0_CLK, P_LP0_ACK, P_LP0_D0, P_LP0_D1, P_LP0_D2, P_LP0_D3, P_LP0_D4, P_LP0_D5, P_LP0_D6, P_LP0_D7, - 0 + 0 }; static const unsigned short lp1_mux[] = { P_LP1_CLK, P_LP1_ACK, P_LP1_D0, P_LP1_D1, P_LP1_D2, P_LP1_D3, P_LP1_D4, P_LP1_D5, P_LP1_D6, P_LP1_D7, - 0 + 0 }; static const unsigned short lp2_mux[] = { P_LP2_CLK, P_LP2_ACK, P_LP2_D0, P_LP2_D1, P_LP2_D2, P_LP2_D3, P_LP2_D4, P_LP2_D5, P_LP2_D6, P_LP2_D7, - 0 + 0 }; static const unsigned short lp3_mux[] = { P_LP3_CLK, P_LP3_ACK, P_LP3_D0, P_LP3_D1, P_LP3_D2, P_LP3_D3, P_LP3_D4, P_LP3_D5, P_LP3_D6, P_LP3_D7, - 0 + 0 }; static const struct adi_pin_group adi_pin_groups[] = { diff --git a/drivers/pinctrl/pinctrl-adi2.c b/drivers/pinctrl/pinctrl-adi2.c index c3c3d2345fc6b2..a5976ebc448263 100644 --- a/drivers/pinctrl/pinctrl-adi2.c +++ b/drivers/pinctrl/pinctrl-adi2.c @@ -427,10 +427,10 @@ static int adi_gpio_irq_type(struct irq_data *d, unsigned int type) if (type & (IRQ_TYPE_EDGE_RISING | IRQ_TYPE_EDGE_FALLING)) { writel(pintmask, &pint_regs->edge_set); - __irq_set_handler_locked(irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); } else { writel(pintmask, &pint_regs->edge_clear); - __irq_set_handler_locked(irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); } out: diff --git a/drivers/pinctrl/pinctrl-amd.c b/drivers/pinctrl/pinctrl-amd.c index d8e3f7c7fea35c..5e86bb8ca80e68 100644 --- a/drivers/pinctrl/pinctrl-amd.c +++ b/drivers/pinctrl/pinctrl-amd.c @@ -420,7 +420,7 @@ static int amd_gpio_irq_set_type(struct irq_data *d, unsigned int type) pin_reg &= ~(ACTIVE_LEVEL_MASK << ACTIVE_LEVEL_OFF); pin_reg |= ACTIVE_HIGH << ACTIVE_LEVEL_OFF; pin_reg |= DB_TYPE_REMOVE_GLITCH << DB_CNTRL_OFF; - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); break; case IRQ_TYPE_EDGE_FALLING: @@ -428,7 +428,7 @@ static int amd_gpio_irq_set_type(struct irq_data *d, unsigned int type) pin_reg &= ~(ACTIVE_LEVEL_MASK << ACTIVE_LEVEL_OFF); pin_reg |= ACTIVE_LOW << ACTIVE_LEVEL_OFF; pin_reg |= DB_TYPE_REMOVE_GLITCH << DB_CNTRL_OFF; - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); break; case IRQ_TYPE_EDGE_BOTH: @@ -436,7 +436,7 @@ static int amd_gpio_irq_set_type(struct irq_data *d, unsigned int type) pin_reg &= ~(ACTIVE_LEVEL_MASK << ACTIVE_LEVEL_OFF); pin_reg |= BOTH_EADGE << ACTIVE_LEVEL_OFF; pin_reg |= DB_TYPE_REMOVE_GLITCH << DB_CNTRL_OFF; - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); break; case IRQ_TYPE_LEVEL_HIGH: @@ -445,7 +445,7 @@ static int amd_gpio_irq_set_type(struct irq_data *d, unsigned int type) pin_reg |= ACTIVE_HIGH << ACTIVE_LEVEL_OFF; pin_reg &= ~(DB_CNTRl_MASK << DB_CNTRL_OFF); pin_reg |= DB_TYPE_PRESERVE_LOW_GLITCH << DB_CNTRL_OFF; - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); break; case IRQ_TYPE_LEVEL_LOW: @@ -454,7 +454,7 @@ static int amd_gpio_irq_set_type(struct irq_data *d, unsigned int type) pin_reg |= ACTIVE_LOW << ACTIVE_LEVEL_OFF; pin_reg &= ~(DB_CNTRl_MASK << DB_CNTRL_OFF); pin_reg |= DB_TYPE_PRESERVE_HIGH_GLITCH << DB_CNTRL_OFF; - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); break; case IRQ_TYPE_NONE: @@ -492,8 +492,9 @@ static struct irq_chip amd_gpio_irqchip = { .irq_set_type = amd_gpio_irq_set_type, }; -static void amd_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) +static void amd_gpio_irq_handler(unsigned int __irq, struct irq_desc *desc) { + unsigned int irq = irq_desc_get_irq(desc); u32 i; u32 off; u32 reg; @@ -501,7 +502,7 @@ static void amd_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) u64 reg64; int handled = 0; unsigned long flags; - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); struct gpio_chip *gc = irq_desc_get_handler_data(desc); struct amd_gpio *gpio_dev = to_amd_gpio(gc); diff --git a/drivers/pinctrl/pinctrl-at91.c b/drivers/pinctrl/pinctrl-at91.c index a0824477072bd1..bae0012ee35601 100644 --- a/drivers/pinctrl/pinctrl-at91.c +++ b/drivers/pinctrl/pinctrl-at91.c @@ -320,6 +320,9 @@ static const struct pinctrl_ops at91_pctrl_ops = { static void __iomem *pin_to_controller(struct at91_pinctrl *info, unsigned int bank) { + if (!gpio_chips[bank]) + return NULL; + return gpio_chips[bank]->regbase; } @@ -729,6 +732,10 @@ static int at91_pmx_set(struct pinctrl_dev *pctldev, unsigned selector, pin = &pins_conf[i]; at91_pin_dbg(info->dev, pin); pio = pin_to_controller(info, pin->bank); + + if (!pio) + continue; + mask = pin_to_mask(pin->pin); at91_mux_disable_interrupt(pio, mask); switch (pin->mux) { @@ -848,6 +855,10 @@ static int at91_pinconf_get(struct pinctrl_dev *pctldev, *config = 0; dev_dbg(info->dev, "%s:%d, pin_id=%d", __func__, __LINE__, pin_id); pio = pin_to_controller(info, pin_to_bank(pin_id)); + + if (!pio) + return -EINVAL; + pin = pin_id % MAX_NB_GPIO_PER_BANK; if (at91_mux_get_multidrive(pio, pin)) @@ -889,6 +900,10 @@ static int at91_pinconf_set(struct pinctrl_dev *pctldev, "%s:%d, pin_id=%d, config=0x%lx", __func__, __LINE__, pin_id, config); pio = pin_to_controller(info, pin_to_bank(pin_id)); + + if (!pio) + return -EINVAL; + pin = pin_id % MAX_NB_GPIO_PER_BANK; mask = pin_to_mask(pin); @@ -1444,22 +1459,22 @@ static int alt_gpio_irq_type(struct irq_data *d, unsigned type) switch (type) { case IRQ_TYPE_EDGE_RISING: - __irq_set_handler_locked(d->irq, handle_simple_irq); + irq_set_handler_locked(d, handle_simple_irq); writel_relaxed(mask, pio + PIO_ESR); writel_relaxed(mask, pio + PIO_REHLSR); break; case IRQ_TYPE_EDGE_FALLING: - __irq_set_handler_locked(d->irq, handle_simple_irq); + irq_set_handler_locked(d, handle_simple_irq); writel_relaxed(mask, pio + PIO_ESR); writel_relaxed(mask, pio + PIO_FELLSR); break; case IRQ_TYPE_LEVEL_LOW: - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); writel_relaxed(mask, pio + PIO_LSR); writel_relaxed(mask, pio + PIO_FELLSR); break; case IRQ_TYPE_LEVEL_HIGH: - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); writel_relaxed(mask, pio + PIO_LSR); writel_relaxed(mask, pio + PIO_REHLSR); break; @@ -1468,7 +1483,7 @@ static int alt_gpio_irq_type(struct irq_data *d, unsigned type) * disable additional interrupt modes: * fall back to default behavior */ - __irq_set_handler_locked(d->irq, handle_simple_irq); + irq_set_handler_locked(d, handle_simple_irq); writel_relaxed(mask, pio + PIO_AIMDR); return 0; case IRQ_TYPE_NONE: @@ -1488,28 +1503,6 @@ static void gpio_irq_ack(struct irq_data *d) /* the interrupt is already cleared before by reading ISR */ } -static int gpio_irq_request_res(struct irq_data *d) -{ - struct at91_gpio_chip *at91_gpio = irq_data_get_irq_chip_data(d); - unsigned pin = d->hwirq; - int ret; - - ret = gpiochip_lock_as_irq(&at91_gpio->chip, pin); - if (ret) - dev_err(at91_gpio->chip.dev, "unable to lock pind %lu IRQ\n", - d->hwirq); - - return ret; -} - -static void gpio_irq_release_res(struct irq_data *d) -{ - struct at91_gpio_chip *at91_gpio = irq_data_get_irq_chip_data(d); - unsigned pin = d->hwirq; - - gpiochip_unlock_as_irq(&at91_gpio->chip, pin); -} - #ifdef CONFIG_PM static u32 wakeups[MAX_GPIO_BANKS]; @@ -1585,8 +1578,6 @@ void at91_pinctrl_gpio_resume(void) static struct irq_chip gpio_irqchip = { .name = "GPIO", .irq_ack = gpio_irq_ack, - .irq_request_resources = gpio_irq_request_res, - .irq_release_resources = gpio_irq_release_res, .irq_disable = gpio_irq_mask, .irq_mask = gpio_irq_mask, .irq_unmask = gpio_irq_unmask, @@ -1596,7 +1587,7 @@ static struct irq_chip gpio_irqchip = { static void gpio_irq_handler(unsigned irq, struct irq_desc *desc) { - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); struct gpio_chip *gpio_chip = irq_desc_get_handler_data(desc); struct at91_gpio_chip *at91_gpio = container_of(gpio_chip, struct at91_gpio_chip, chip); diff --git a/drivers/pinctrl/pinctrl-coh901.c b/drivers/pinctrl/pinctrl-coh901.c index 29cbbab8c3a673..3731cc67a88bf8 100644 --- a/drivers/pinctrl/pinctrl-coh901.c +++ b/drivers/pinctrl/pinctrl-coh901.c @@ -519,10 +519,11 @@ static struct irq_chip u300_gpio_irqchip = { .irq_set_type = u300_gpio_irq_type, }; -static void u300_gpio_irq_handler(unsigned irq, struct irq_desc *desc) +static void u300_gpio_irq_handler(unsigned __irq, struct irq_desc *desc) { - struct irq_chip *parent_chip = irq_get_chip(irq); - struct gpio_chip *chip = irq_get_handler_data(irq); + unsigned int irq = irq_desc_get_irq(desc); + struct irq_chip *parent_chip = irq_desc_get_chip(desc); + struct gpio_chip *chip = irq_desc_get_handler_data(desc); struct u300_gpio *gpio = to_u300_gpio(chip); struct u300_gpio_port *port = &gpio->ports[irq - chip->base]; int pinoffset = port->number << 3; /* get the right stride */ diff --git a/drivers/pinctrl/pinctrl-digicolor.c b/drivers/pinctrl/pinctrl-digicolor.c new file mode 100644 index 00000000000000..461fffc4c62ae8 --- /dev/null +++ b/drivers/pinctrl/pinctrl-digicolor.c @@ -0,0 +1,378 @@ +/* + * Driver for Conexant Digicolor General Purpose Pin Mapping + * + * Author: Baruch Siach + * + * Copyright (C) 2015 Paradox Innovation Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * TODO: + * - GPIO interrupt support + * - Pin pad configuration (pull up/down, strength) + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "pinctrl-utils.h" + +#define DRIVER_NAME "pinctrl-digicolor" + +#define GP_CLIENTSEL(clct) ((clct)*8 + 0x20) +#define GP_DRIVE0(clct) (GP_CLIENTSEL(clct) + 2) +#define GP_OUTPUT0(clct) (GP_CLIENTSEL(clct) + 3) +#define GP_INPUT(clct) (GP_CLIENTSEL(clct) + 6) + +#define PIN_COLLECTIONS ('R' - 'A' + 1) +#define PINS_PER_COLLECTION 8 +#define PINS_COUNT (PIN_COLLECTIONS * PINS_PER_COLLECTION) + +struct dc_pinmap { + void __iomem *regs; + struct device *dev; + struct pinctrl_dev *pctl; + + struct pinctrl_desc *desc; + const char *pin_names[PINS_COUNT]; + + struct gpio_chip chip; + spinlock_t lock; +}; + +static int dc_get_groups_count(struct pinctrl_dev *pctldev) +{ + return PINS_COUNT; +} + +static const char *dc_get_group_name(struct pinctrl_dev *pctldev, + unsigned selector) +{ + struct dc_pinmap *pmap = pinctrl_dev_get_drvdata(pctldev); + + /* Exactly one group per pin */ + return pmap->desc->pins[selector].name; +} + +static int dc_get_group_pins(struct pinctrl_dev *pctldev, unsigned selector, + const unsigned **pins, + unsigned *num_pins) +{ + struct dc_pinmap *pmap = pinctrl_dev_get_drvdata(pctldev); + + *pins = &pmap->desc->pins[selector].number; + *num_pins = 1; + + return 0; +} + +static struct pinctrl_ops dc_pinctrl_ops = { + .get_groups_count = dc_get_groups_count, + .get_group_name = dc_get_group_name, + .get_group_pins = dc_get_group_pins, + .dt_node_to_map = pinconf_generic_dt_node_to_map_pin, + .dt_free_map = pinctrl_utils_dt_free_map, +}; + +static const char *const dc_functions[] = { + "gpio", + "client_a", + "client_b", + "client_c", +}; + +static int dc_get_functions_count(struct pinctrl_dev *pctldev) +{ + return ARRAY_SIZE(dc_functions); +} + +static const char *dc_get_fname(struct pinctrl_dev *pctldev, unsigned selector) +{ + return dc_functions[selector]; +} + +static int dc_get_groups(struct pinctrl_dev *pctldev, unsigned selector, + const char * const **groups, + unsigned * const num_groups) +{ + struct dc_pinmap *pmap = pinctrl_dev_get_drvdata(pctldev); + + *groups = pmap->pin_names; + *num_groups = PINS_COUNT; + + return 0; +} + +static void dc_client_sel(int pin_num, int *reg, int *bit) +{ + *bit = (pin_num % PINS_PER_COLLECTION) * 2; + *reg = GP_CLIENTSEL(pin_num/PINS_PER_COLLECTION); + + if (*bit >= PINS_PER_COLLECTION) { + *bit -= PINS_PER_COLLECTION; + *reg += 1; + } +} + +static int dc_set_mux(struct pinctrl_dev *pctldev, unsigned selector, + unsigned group) +{ + struct dc_pinmap *pmap = pinctrl_dev_get_drvdata(pctldev); + int bit_off, reg_off; + u8 reg; + + dc_client_sel(group, ®_off, &bit_off); + + reg = readb_relaxed(pmap->regs + reg_off); + reg &= ~(3 << bit_off); + reg |= (selector << bit_off); + writeb_relaxed(reg, pmap->regs + reg_off); + + return 0; +} + +static int dc_pmx_request_gpio(struct pinctrl_dev *pcdev, + struct pinctrl_gpio_range *range, + unsigned offset) +{ + struct dc_pinmap *pmap = pinctrl_dev_get_drvdata(pcdev); + int bit_off, reg_off; + u8 reg; + + dc_client_sel(offset, ®_off, &bit_off); + + reg = readb_relaxed(pmap->regs + reg_off); + if ((reg & (3 << bit_off)) != 0) + return -EBUSY; + + return 0; +} + +static struct pinmux_ops dc_pmxops = { + .get_functions_count = dc_get_functions_count, + .get_function_name = dc_get_fname, + .get_function_groups = dc_get_groups, + .set_mux = dc_set_mux, + .gpio_request_enable = dc_pmx_request_gpio, +}; + +static int dc_gpio_request(struct gpio_chip *chip, unsigned gpio) +{ + return pinctrl_request_gpio(chip->base + gpio); +} + +static void dc_gpio_free(struct gpio_chip *chip, unsigned gpio) +{ + pinctrl_free_gpio(chip->base + gpio); +} + +static int dc_gpio_direction_input(struct gpio_chip *chip, unsigned gpio) +{ + struct dc_pinmap *pmap = container_of(chip, struct dc_pinmap, chip); + int reg_off = GP_DRIVE0(gpio/PINS_PER_COLLECTION); + int bit_off = gpio % PINS_PER_COLLECTION; + u8 drive; + unsigned long flags; + + spin_lock_irqsave(&pmap->lock, flags); + drive = readb_relaxed(pmap->regs + reg_off); + drive &= ~BIT(bit_off); + writeb_relaxed(drive, pmap->regs + reg_off); + spin_unlock_irqrestore(&pmap->lock, flags); + + return 0; +} + +static void dc_gpio_set(struct gpio_chip *chip, unsigned gpio, int value); + +static int dc_gpio_direction_output(struct gpio_chip *chip, unsigned gpio, + int value) +{ + struct dc_pinmap *pmap = container_of(chip, struct dc_pinmap, chip); + int reg_off = GP_DRIVE0(gpio/PINS_PER_COLLECTION); + int bit_off = gpio % PINS_PER_COLLECTION; + u8 drive; + unsigned long flags; + + dc_gpio_set(chip, gpio, value); + + spin_lock_irqsave(&pmap->lock, flags); + drive = readb_relaxed(pmap->regs + reg_off); + drive |= BIT(bit_off); + writeb_relaxed(drive, pmap->regs + reg_off); + spin_unlock_irqrestore(&pmap->lock, flags); + + return 0; +} + +static int dc_gpio_get(struct gpio_chip *chip, unsigned gpio) +{ + struct dc_pinmap *pmap = container_of(chip, struct dc_pinmap, chip); + int reg_off = GP_INPUT(gpio/PINS_PER_COLLECTION); + int bit_off = gpio % PINS_PER_COLLECTION; + u8 input; + + input = readb_relaxed(pmap->regs + reg_off); + + return !!(input & BIT(bit_off)); +} + +static void dc_gpio_set(struct gpio_chip *chip, unsigned gpio, int value) +{ + struct dc_pinmap *pmap = container_of(chip, struct dc_pinmap, chip); + int reg_off = GP_OUTPUT0(gpio/PINS_PER_COLLECTION); + int bit_off = gpio % PINS_PER_COLLECTION; + u8 output; + unsigned long flags; + + spin_lock_irqsave(&pmap->lock, flags); + output = readb_relaxed(pmap->regs + reg_off); + if (value) + output |= BIT(bit_off); + else + output &= ~BIT(bit_off); + writeb_relaxed(output, pmap->regs + reg_off); + spin_unlock_irqrestore(&pmap->lock, flags); +} + +static int dc_gpiochip_add(struct dc_pinmap *pmap, struct device_node *np) +{ + struct gpio_chip *chip = &pmap->chip; + int ret; + + chip->label = DRIVER_NAME; + chip->dev = pmap->dev; + chip->request = dc_gpio_request; + chip->free = dc_gpio_free; + chip->direction_input = dc_gpio_direction_input; + chip->direction_output = dc_gpio_direction_output; + chip->get = dc_gpio_get; + chip->set = dc_gpio_set; + chip->base = -1; + chip->ngpio = PINS_COUNT; + chip->of_node = np; + chip->of_gpio_n_cells = 2; + + spin_lock_init(&pmap->lock); + + ret = gpiochip_add(chip); + if (ret < 0) + return ret; + + ret = gpiochip_add_pin_range(chip, dev_name(pmap->dev), 0, 0, + PINS_COUNT); + if (ret < 0) { + gpiochip_remove(chip); + return ret; + } + + return 0; +} + +static int dc_pinctrl_probe(struct platform_device *pdev) +{ + struct dc_pinmap *pmap; + struct resource *r; + struct pinctrl_pin_desc *pins; + struct pinctrl_desc *pctl_desc; + char *pin_names; + int name_len = strlen("GP_xx") + 1; + int i, j, ret; + + pmap = devm_kzalloc(&pdev->dev, sizeof(*pmap), GFP_KERNEL); + if (!pmap) + return -ENOMEM; + + r = platform_get_resource(pdev, IORESOURCE_MEM, 0); + pmap->regs = devm_ioremap_resource(&pdev->dev, r); + if (IS_ERR(pmap->regs)) + return PTR_ERR(pmap->regs); + + pins = devm_kzalloc(&pdev->dev, sizeof(*pins)*PINS_COUNT, GFP_KERNEL); + if (!pins) + return -ENOMEM; + pin_names = devm_kzalloc(&pdev->dev, name_len * PINS_COUNT, + GFP_KERNEL); + if (!pin_names) + return -ENOMEM; + + for (i = 0; i < PIN_COLLECTIONS; i++) { + for (j = 0; j < PINS_PER_COLLECTION; j++) { + int pin_id = i*PINS_PER_COLLECTION + j; + char *name = &pin_names[pin_id * name_len]; + + snprintf(name, name_len, "GP_%c%c", 'A'+i, '0'+j); + + pins[pin_id].number = pin_id; + pins[pin_id].name = name; + pmap->pin_names[pin_id] = name; + } + } + + pctl_desc = devm_kzalloc(&pdev->dev, sizeof(*pctl_desc), GFP_KERNEL); + if (!pctl_desc) + return -ENOMEM; + + pctl_desc->name = DRIVER_NAME, + pctl_desc->owner = THIS_MODULE, + pctl_desc->pctlops = &dc_pinctrl_ops, + pctl_desc->pmxops = &dc_pmxops, + pctl_desc->npins = PINS_COUNT; + pctl_desc->pins = pins; + pmap->desc = pctl_desc; + + pmap->dev = &pdev->dev; + + pmap->pctl = pinctrl_register(pctl_desc, &pdev->dev, pmap); + if (!pmap->pctl) { + dev_err(&pdev->dev, "pinctrl driver registration failed\n"); + return -EINVAL; + } + + ret = dc_gpiochip_add(pmap, pdev->dev.of_node); + if (ret < 0) { + pinctrl_unregister(pmap->pctl); + return ret; + } + + return 0; +} + +static int dc_pinctrl_remove(struct platform_device *pdev) +{ + struct dc_pinmap *pmap = platform_get_drvdata(pdev); + + pinctrl_unregister(pmap->pctl); + gpiochip_remove(&pmap->chip); + + return 0; +} + +static const struct of_device_id dc_pinctrl_ids[] = { + { .compatible = "cnxt,cx92755-pinctrl" }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, dc_pinctrl_ids); + +static struct platform_driver dc_pinctrl_driver = { + .driver = { + .name = DRIVER_NAME, + .of_match_table = dc_pinctrl_ids, + }, + .probe = dc_pinctrl_probe, + .remove = dc_pinctrl_remove, +}; +module_platform_driver(dc_pinctrl_driver); diff --git a/drivers/pinctrl/pinctrl-lpc18xx.c b/drivers/pinctrl/pinctrl-lpc18xx.c index 347c763a6a7876..f0bebbe0682b9b 100644 --- a/drivers/pinctrl/pinctrl-lpc18xx.c +++ b/drivers/pinctrl/pinctrl-lpc18xx.c @@ -37,6 +37,9 @@ #define LPC18XX_SCU_PIN_EHD_MASK 0x300 #define LPC18XX_SCU_PIN_EHD_POS 8 +#define LPC18XX_SCU_USB1_EPD BIT(2) +#define LPC18XX_SCU_USB1_EPWR BIT(4) + #define LPC18XX_SCU_I2C0_EFP BIT(0) #define LPC18XX_SCU_I2C0_EHD BIT(2) #define LPC18XX_SCU_I2C0_EZI BIT(3) @@ -617,8 +620,31 @@ static const struct pinctrl_pin_desc lpc18xx_pins[] = { static int lpc18xx_pconf_get_usb1(enum pin_config_param param, int *arg, u32 reg) { - /* TODO */ - return -ENOTSUPP; + switch (param) { + case PIN_CONFIG_LOW_POWER_MODE: + if (reg & LPC18XX_SCU_USB1_EPWR) + *arg = 0; + else + *arg = 1; + break; + + case PIN_CONFIG_BIAS_DISABLE: + if (reg & LPC18XX_SCU_USB1_EPD) + return -EINVAL; + break; + + case PIN_CONFIG_BIAS_PULL_DOWN: + if (reg & LPC18XX_SCU_USB1_EPD) + *arg = 1; + else + return -EINVAL; + break; + + default: + return -ENOTSUPP; + } + + return 0; } static int lpc18xx_pconf_get_i2c0(enum pin_config_param param, int *arg, u32 reg, @@ -782,8 +808,28 @@ static int lpc18xx_pconf_set_usb1(struct pinctrl_dev *pctldev, enum pin_config_param param, u16 param_val, u32 *reg) { - /* TODO */ - return -ENOTSUPP; + switch (param) { + case PIN_CONFIG_LOW_POWER_MODE: + if (param_val) + *reg &= ~LPC18XX_SCU_USB1_EPWR; + else + *reg |= LPC18XX_SCU_USB1_EPWR; + break; + + case PIN_CONFIG_BIAS_DISABLE: + *reg &= ~LPC18XX_SCU_USB1_EPD; + break; + + case PIN_CONFIG_BIAS_PULL_DOWN: + *reg |= LPC18XX_SCU_USB1_EPD; + break; + + default: + dev_err(pctldev->dev, "Property not supported\n"); + return -ENOTSUPP; + } + + return 0; } static int lpc18xx_pconf_set_i2c0(struct pinctrl_dev *pctldev, diff --git a/drivers/pinctrl/pinctrl-pistachio.c b/drivers/pinctrl/pinctrl-pistachio.c index 63100be8101520..3dc2ae15f3a10f 100644 --- a/drivers/pinctrl/pinctrl-pistachio.c +++ b/drivers/pinctrl/pinctrl-pistachio.c @@ -1310,9 +1310,11 @@ static int pistachio_gpio_irq_set_type(struct irq_data *data, unsigned int type) return 0; } -static void pistachio_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) +static void pistachio_gpio_irq_handler(unsigned int __irq, + struct irq_desc *desc) { - struct gpio_chip *gc = irq_get_handler_data(irq); + unsigned int irq = irq_desc_get_irq(desc); + struct gpio_chip *gc = irq_desc_get_handler_data(desc); struct pistachio_gpio_bank *bank = gc_to_bank(gc); struct irq_chip *chip = irq_get_chip(irq); unsigned long pending; diff --git a/drivers/pinctrl/pinctrl-rockchip.c b/drivers/pinctrl/pinctrl-rockchip.c index 9affcd7257764d..c5246c05f70cc7 100644 --- a/drivers/pinctrl/pinctrl-rockchip.c +++ b/drivers/pinctrl/pinctrl-rockchip.c @@ -945,6 +945,7 @@ static int _rockchip_pmx_gpio_set_direction(struct gpio_chip *chip, if (ret < 0) return ret; + clk_enable(bank->clk); spin_lock_irqsave(&bank->slock, flags); data = readl_relaxed(bank->reg_base + GPIO_SWPORT_DDR); @@ -956,6 +957,7 @@ static int _rockchip_pmx_gpio_set_direction(struct gpio_chip *chip, writel_relaxed(data, bank->reg_base + GPIO_SWPORT_DDR); spin_unlock_irqrestore(&bank->slock, flags); + clk_disable(bank->clk); return 0; } @@ -1389,6 +1391,7 @@ static void rockchip_gpio_set(struct gpio_chip *gc, unsigned offset, int value) unsigned long flags; u32 data; + clk_enable(bank->clk); spin_lock_irqsave(&bank->slock, flags); data = readl(reg); @@ -1398,6 +1401,7 @@ static void rockchip_gpio_set(struct gpio_chip *gc, unsigned offset, int value) writel(data, reg); spin_unlock_irqrestore(&bank->slock, flags); + clk_disable(bank->clk); } /* @@ -1409,7 +1413,9 @@ static int rockchip_gpio_get(struct gpio_chip *gc, unsigned offset) struct rockchip_pin_bank *bank = gc_to_pin_bank(gc); u32 data; + clk_enable(bank->clk); data = readl(bank->reg_base + GPIO_EXT_PORT); + clk_disable(bank->clk); data >>= offset; data &= 1; return data; @@ -1469,10 +1475,10 @@ static const struct gpio_chip rockchip_gpiolib_chip = { * Interrupt handling */ -static void rockchip_irq_demux(unsigned int irq, struct irq_desc *desc) +static void rockchip_irq_demux(unsigned int __irq, struct irq_desc *desc) { - struct irq_chip *chip = irq_get_chip(irq); - struct rockchip_pin_bank *bank = irq_get_handler_data(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); + struct rockchip_pin_bank *bank = irq_desc_get_handler_data(desc); u32 pend; dev_dbg(bank->drvdata->dev, "got irq for bank %s\n", bank->name); @@ -1482,7 +1488,7 @@ static void rockchip_irq_demux(unsigned int irq, struct irq_desc *desc) pend = readl_relaxed(bank->reg_base + GPIO_INT_STATUS); while (pend) { - unsigned int virq; + unsigned int irq, virq; irq = __ffs(pend); pend &= ~BIT(irq); @@ -1546,6 +1552,7 @@ static int rockchip_irq_set_type(struct irq_data *d, unsigned int type) if (ret < 0) return ret; + clk_enable(bank->clk); spin_lock_irqsave(&bank->slock, flags); data = readl_relaxed(bank->reg_base + GPIO_SWPORT_DDR); @@ -1555,9 +1562,9 @@ static int rockchip_irq_set_type(struct irq_data *d, unsigned int type) spin_unlock_irqrestore(&bank->slock, flags); if (type & IRQ_TYPE_EDGE_BOTH) - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); else - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); spin_lock_irqsave(&bank->slock, flags); irq_gc_lock(gc); @@ -1603,6 +1610,7 @@ static int rockchip_irq_set_type(struct irq_data *d, unsigned int type) default: irq_gc_unlock(gc); spin_unlock_irqrestore(&bank->slock, flags); + clk_disable(bank->clk); return -EINVAL; } @@ -1611,6 +1619,7 @@ static int rockchip_irq_set_type(struct irq_data *d, unsigned int type) irq_gc_unlock(gc); spin_unlock_irqrestore(&bank->slock, flags); + clk_disable(bank->clk); return 0; } @@ -1620,8 +1629,10 @@ static void rockchip_irq_suspend(struct irq_data *d) struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d); struct rockchip_pin_bank *bank = gc->private; + clk_enable(bank->clk); bank->saved_masks = irq_reg_readl(gc, GPIO_INTMASK); irq_reg_writel(gc, ~gc->wake_active, GPIO_INTMASK); + clk_disable(bank->clk); } static void rockchip_irq_resume(struct irq_data *d) @@ -1629,7 +1640,27 @@ static void rockchip_irq_resume(struct irq_data *d) struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d); struct rockchip_pin_bank *bank = gc->private; + clk_enable(bank->clk); irq_reg_writel(gc, bank->saved_masks, GPIO_INTMASK); + clk_disable(bank->clk); +} + +static void rockchip_irq_gc_mask_clr_bit(struct irq_data *d) +{ + struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d); + struct rockchip_pin_bank *bank = gc->private; + + clk_enable(bank->clk); + irq_gc_mask_clr_bit(d); +} + +void rockchip_irq_gc_mask_set_bit(struct irq_data *d) +{ + struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d); + struct rockchip_pin_bank *bank = gc->private; + + irq_gc_mask_set_bit(d); + clk_disable(bank->clk); } static int rockchip_interrupts_register(struct platform_device *pdev, @@ -1640,7 +1671,7 @@ static int rockchip_interrupts_register(struct platform_device *pdev, unsigned int clr = IRQ_NOREQUEST | IRQ_NOPROBE | IRQ_NOAUTOEN; struct irq_chip_generic *gc; int ret; - int i; + int i, j; for (i = 0; i < ctrl->nr_banks; ++i, ++bank) { if (!bank->valid) { @@ -1649,11 +1680,19 @@ static int rockchip_interrupts_register(struct platform_device *pdev, continue; } + ret = clk_enable(bank->clk); + if (ret) { + dev_err(&pdev->dev, "failed to enable clock for bank %s\n", + bank->name); + continue; + } + bank->domain = irq_domain_add_linear(bank->of_node, 32, &irq_generic_chip_ops, NULL); if (!bank->domain) { dev_warn(&pdev->dev, "could not initialize irq domain for bank %s\n", bank->name); + clk_disable(bank->clk); continue; } @@ -1664,6 +1703,7 @@ static int rockchip_interrupts_register(struct platform_device *pdev, dev_err(&pdev->dev, "could not alloc generic chips for bank %s\n", bank->name); irq_domain_remove(bank->domain); + clk_disable(bank->clk); continue; } @@ -1681,16 +1721,23 @@ static int rockchip_interrupts_register(struct platform_device *pdev, gc->chip_types[0].regs.mask = GPIO_INTMASK; gc->chip_types[0].regs.ack = GPIO_PORTS_EOI; gc->chip_types[0].chip.irq_ack = irq_gc_ack_set_bit; - gc->chip_types[0].chip.irq_mask = irq_gc_mask_set_bit; - gc->chip_types[0].chip.irq_unmask = irq_gc_mask_clr_bit; + gc->chip_types[0].chip.irq_mask = rockchip_irq_gc_mask_set_bit; + gc->chip_types[0].chip.irq_unmask = + rockchip_irq_gc_mask_clr_bit; gc->chip_types[0].chip.irq_set_wake = irq_gc_set_wake; gc->chip_types[0].chip.irq_suspend = rockchip_irq_suspend; gc->chip_types[0].chip.irq_resume = rockchip_irq_resume; gc->chip_types[0].chip.irq_set_type = rockchip_irq_set_type; gc->wake_enabled = IRQ_MSK(bank->nr_pins); - irq_set_handler_data(bank->irq, bank); - irq_set_chained_handler(bank->irq, rockchip_irq_demux); + irq_set_chained_handler_and_data(bank->irq, + rockchip_irq_demux, bank); + + /* map the gpio irqs here, when the clock is still running */ + for (j = 0 ; j < 32 ; j++) + irq_create_mapping(bank->domain, j); + + clk_disable(bank->clk); } return 0; @@ -1808,7 +1855,7 @@ static int rockchip_get_bank_data(struct rockchip_pin_bank *bank, if (IS_ERR(bank->clk)) return PTR_ERR(bank->clk); - return clk_prepare_enable(bank->clk); + return clk_prepare(bank->clk); } static const struct of_device_id rockchip_pinctrl_dt_match[]; diff --git a/drivers/pinctrl/pinctrl-single.c b/drivers/pinctrl/pinctrl-single.c index 0b8d480171a3df..bf548c2a7a9d83 100644 --- a/drivers/pinctrl/pinctrl-single.c +++ b/drivers/pinctrl/pinctrl-single.c @@ -1684,7 +1684,7 @@ static void pcs_irq_chain_handler(unsigned int irq, struct irq_desc *desc) struct pcs_soc_data *pcs_soc = irq_desc_get_handler_data(desc); struct irq_chip *chip; - chip = irq_get_chip(irq); + chip = irq_desc_get_chip(desc); chained_irq_enter(chip, desc); pcs_irq_handle(pcs_soc); /* REVISIT: export and add handle_bad_irq(irq, desc)? */ @@ -1716,12 +1716,7 @@ static int pcs_irqdomain_map(struct irq_domain *d, unsigned int irq, irq_set_chip_data(irq, pcs_soc); irq_set_chip_and_handler(irq, &pcs->chip, handle_level_irq); - -#ifdef CONFIG_ARM - set_irq_flags(irq, IRQF_VALID); -#else irq_set_noprobe(irq); -#endif return 0; } @@ -1768,9 +1763,9 @@ static int pcs_irq_init_chained_handler(struct pcs_device *pcs, return res; } } else { - irq_set_handler_data(pcs_soc->irq, pcs_soc); - irq_set_chained_handler(pcs_soc->irq, - pcs_irq_chain_handler); + irq_set_chained_handler_and_data(pcs_soc->irq, + pcs_irq_chain_handler, + pcs_soc); } /* @@ -1983,7 +1978,6 @@ static const struct pcs_soc_data pinctrl_single_omap_wkup = { }; static const struct pcs_soc_data pinctrl_single_dra7 = { - .flags = PCS_QUIRK_SHARED_IRQ, .irq_enable_mask = (1 << 24), /* WAKEUPENABLE */ .irq_status_mask = (1 << 25), /* WAKEUPEVENT */ }; diff --git a/drivers/pinctrl/pinctrl-st.c b/drivers/pinctrl/pinctrl-st.c index c262e5f35c2808..f8338d2e6b6b91 100644 --- a/drivers/pinctrl/pinctrl-st.c +++ b/drivers/pinctrl/pinctrl-st.c @@ -1463,7 +1463,7 @@ static void __gpio_irq_handler(struct st_gpio_bank *bank) static void st_gpio_irq_handler(unsigned irq, struct irq_desc *desc) { /* interrupt dedicated per bank */ - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); struct gpio_chip *gc = irq_desc_get_handler_data(desc); struct st_gpio_bank *bank = gpio_chip_to_bank(gc); @@ -1474,8 +1474,8 @@ static void st_gpio_irq_handler(unsigned irq, struct irq_desc *desc) static void st_gpio_irqmux_handler(unsigned irq, struct irq_desc *desc) { - struct irq_chip *chip = irq_get_chip(irq); - struct st_pinctrl *info = irq_get_handler_data(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); + struct st_pinctrl *info = irq_desc_get_handler_data(desc); unsigned long status; int n; diff --git a/drivers/pinctrl/pinctrl-zynq.c b/drivers/pinctrl/pinctrl-zynq.c index 7ce23b6282adf1..5aafea8c6590a9 100644 --- a/drivers/pinctrl/pinctrl-zynq.c +++ b/drivers/pinctrl/pinctrl-zynq.c @@ -706,10 +706,10 @@ static const char * const sdio1_wp_groups[] = {"gpio0_0_grp", "gpio0_39_grp", "gpio0_41_grp", "gpio0_43_grp", "gpio0_45_grp", "gpio0_47_grp", "gpio0_49_grp", "gpio0_51_grp", "gpio0_53_grp", "sdio1_emio_wp_grp"}; -static const char * const smc0_nor_groups[] = {"smc0_nor"}; +static const char * const smc0_nor_groups[] = {"smc0_nor_grp"}; static const char * const smc0_nor_cs1_groups[] = {"smc0_nor_cs1_grp"}; static const char * const smc0_nor_addr25_groups[] = {"smc0_nor_addr25_grp"}; -static const char * const smc0_nand_groups[] = {"smc0_nand"}; +static const char * const smc0_nand_groups[] = {"smc0_nand_grp"}; static const char * const can0_groups[] = {"can0_0_grp", "can0_1_grp", "can0_2_grp", "can0_3_grp", "can0_4_grp", "can0_5_grp", "can0_6_grp", "can0_7_grp", "can0_8_grp", "can0_9_grp", diff --git a/drivers/pinctrl/pinmux.c b/drivers/pinctrl/pinmux.c index e7ae890dcf1aed..67e08cb315c47e 100644 --- a/drivers/pinctrl/pinmux.c +++ b/drivers/pinctrl/pinmux.c @@ -322,8 +322,7 @@ static int pinmux_func_name_to_selector(struct pinctrl_dev *pctldev, selector++; } - pr_err("%s does not support function %s\n", - pinctrl_dev_get_name(pctldev), function); + dev_err(pctldev->dev, "function '%s' not supported\n", function); return -EINVAL; } diff --git a/drivers/pinctrl/qcom/Kconfig b/drivers/pinctrl/qcom/Kconfig index 58f5632b27f450..383263a92e59a4 100644 --- a/drivers/pinctrl/qcom/Kconfig +++ b/drivers/pinctrl/qcom/Kconfig @@ -63,6 +63,14 @@ config PINCTRL_MSM8916 This is the pinctrl, pinmux, pinconf and gpiolib driver for the Qualcomm TLMM block found on the Qualcomm 8916 platform. +config PINCTRL_QDF2XXX + tristate "Qualcomm Technologies QDF2xxx pin controller driver" + depends on GPIOLIB && ACPI + select PINCTRL_MSM + help + This is the GPIO driver for the TLMM block found on the + Qualcomm Technologies QDF2xxx SOCs. + config PINCTRL_QCOM_SPMI_PMIC tristate "Qualcomm SPMI PMIC pin controller driver" depends on GPIOLIB && OF && SPMI @@ -76,4 +84,16 @@ config PINCTRL_QCOM_SPMI_PMIC which are using SPMI for communication with SoC. Example PMIC's devices are pm8841, pm8941 and pma8084. +config PINCTRL_QCOM_SSBI_PMIC + tristate "Qualcomm SSBI PMIC pin controller driver" + depends on GPIOLIB && OF + select PINMUX + select PINCONF + select GENERIC_PINCONF + help + This is the pinctrl, pinmux, pinconf and gpiolib driver for the + Qualcomm GPIO and MPP blocks found in the Qualcomm PMIC's chips, + which are using SSBI for communication with SoC. Example PMIC's + devices are pm8058 and pm8921. + endif diff --git a/drivers/pinctrl/qcom/Makefile b/drivers/pinctrl/qcom/Makefile index 3666c703ce8890..13b190e72c2191 100644 --- a/drivers/pinctrl/qcom/Makefile +++ b/drivers/pinctrl/qcom/Makefile @@ -7,5 +7,8 @@ obj-$(CONFIG_PINCTRL_MSM8660) += pinctrl-msm8660.o obj-$(CONFIG_PINCTRL_MSM8960) += pinctrl-msm8960.o obj-$(CONFIG_PINCTRL_MSM8X74) += pinctrl-msm8x74.o obj-$(CONFIG_PINCTRL_MSM8916) += pinctrl-msm8916.o +obj-$(CONFIG_PINCTRL_QDF2XXX) += pinctrl-qdf2xxx.o obj-$(CONFIG_PINCTRL_QCOM_SPMI_PMIC) += pinctrl-spmi-gpio.o obj-$(CONFIG_PINCTRL_QCOM_SPMI_PMIC) += pinctrl-spmi-mpp.o +obj-$(CONFIG_PINCTRL_QCOM_SSBI_PMIC) += pinctrl-ssbi-gpio.o +obj-$(CONFIG_PINCTRL_QCOM_SSBI_PMIC) += pinctrl-ssbi-mpp.o diff --git a/drivers/pinctrl/qcom/pinctrl-msm.c b/drivers/pinctrl/qcom/pinctrl-msm.c index e457d52302a202..492cdd51dc5c71 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm.c +++ b/drivers/pinctrl/qcom/pinctrl-msm.c @@ -28,6 +28,7 @@ #include #include #include +#include #include "../core.h" #include "../pinconf.h" @@ -733,9 +734,9 @@ static int msm_gpio_irq_set_type(struct irq_data *d, unsigned int type) spin_unlock_irqrestore(&pctrl->lock, flags); if (type & (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_LEVEL_HIGH)) - __irq_set_handler_locked(d->irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); else if (type & (IRQ_TYPE_EDGE_FALLING | IRQ_TYPE_EDGE_RISING)) - __irq_set_handler_locked(d->irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); return 0; } @@ -764,12 +765,13 @@ static struct irq_chip msm_gpio_irq_chip = { .irq_set_wake = msm_gpio_irq_set_wake, }; -static void msm_gpio_irq_handler(unsigned int irq, struct irq_desc *desc) +static void msm_gpio_irq_handler(unsigned int __irq, struct irq_desc *desc) { + unsigned int irq = irq_desc_get_irq(desc); struct gpio_chip *gc = irq_desc_get_handler_data(desc); const struct msm_pingroup *g; struct msm_pinctrl *pctrl = to_msm_pinctrl(gc); - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); int irq_pin; int handled = 0; u32 val; @@ -855,6 +857,13 @@ static int msm_ps_hold_restart(struct notifier_block *nb, unsigned long action, return NOTIFY_DONE; } +static struct msm_pinctrl *poweroff_pctrl; + +static void msm_ps_hold_poweroff(void) +{ + msm_ps_hold_restart(&poweroff_pctrl->restart_nb, 0, NULL); +} + static void msm_pinctrl_setup_pm_reset(struct msm_pinctrl *pctrl) { int i; @@ -867,6 +876,8 @@ static void msm_pinctrl_setup_pm_reset(struct msm_pinctrl *pctrl) if (register_restart_handler(&pctrl->restart_nb)) dev_err(pctrl->dev, "failed to setup restart handler.\n"); + poweroff_pctrl = pctrl; + pm_power_off = msm_ps_hold_poweroff; break; } } diff --git a/drivers/pinctrl/qcom/pinctrl-qdf2xxx.c b/drivers/pinctrl/qcom/pinctrl-qdf2xxx.c new file mode 100644 index 00000000000000..e9ff3bc150bbc7 --- /dev/null +++ b/drivers/pinctrl/qcom/pinctrl-qdf2xxx.c @@ -0,0 +1,122 @@ +/* + * Copyright (c) 2015, The Linux Foundation. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 and + * only version 2 as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * GPIO and pin control functions on this SOC are handled by the "TLMM" + * device. The driver which controls this device is pinctrl-msm.c. Each + * SOC with a TLMM is expected to create a client driver that registers + * with pinctrl-msm.c. This means that all TLMM drivers are pin control + * drivers. + * + * This pin control driver is intended to be used only an ACPI-enabled + * system. As such, UEFI will handle all pin control configuration, so + * this driver does not provide pin control functions. It is effectively + * a GPIO-only driver. The alternative is to duplicate the GPIO code of + * pinctrl-msm.c into another driver. + */ + +#include +#include +#include +#include + +#include "pinctrl-msm.h" + +static struct msm_pinctrl_soc_data qdf2xxx_pinctrl; + +static int qdf2xxx_pinctrl_probe(struct platform_device *pdev) +{ + struct pinctrl_pin_desc *pins; + struct msm_pingroup *groups; + unsigned int i; + u32 num_gpios; + int ret; + + /* Query the number of GPIOs from ACPI */ + ret = device_property_read_u32(&pdev->dev, "num-gpios", &num_gpios); + if (ret < 0) + return ret; + + if (!num_gpios) { + dev_warn(&pdev->dev, "missing num-gpios property\n"); + return -ENODEV; + } + + pins = devm_kcalloc(&pdev->dev, num_gpios, + sizeof(struct pinctrl_pin_desc), GFP_KERNEL); + groups = devm_kcalloc(&pdev->dev, num_gpios, + sizeof(struct msm_pingroup), GFP_KERNEL); + + for (i = 0; i < num_gpios; i++) { + pins[i].number = i; + + groups[i].npins = 1, + groups[i].pins = &pins[i].number; + groups[i].ctl_reg = 0x10000 * i; + groups[i].io_reg = 0x04 + 0x10000 * i; + groups[i].intr_cfg_reg = 0x08 + 0x10000 * i; + groups[i].intr_status_reg = 0x0c + 0x10000 * i; + groups[i].intr_target_reg = 0x08 + 0x10000 * i; + + groups[i].mux_bit = 2; + groups[i].pull_bit = 0; + groups[i].drv_bit = 6; + groups[i].oe_bit = 9; + groups[i].in_bit = 0; + groups[i].out_bit = 1; + groups[i].intr_enable_bit = 0; + groups[i].intr_status_bit = 0; + groups[i].intr_target_bit = 5; + groups[i].intr_target_kpss_val = 1; + groups[i].intr_raw_status_bit = 4; + groups[i].intr_polarity_bit = 1; + groups[i].intr_detection_bit = 2; + groups[i].intr_detection_width = 2; + } + + qdf2xxx_pinctrl.pins = pins; + qdf2xxx_pinctrl.groups = groups; + qdf2xxx_pinctrl.npins = num_gpios; + qdf2xxx_pinctrl.ngroups = num_gpios; + qdf2xxx_pinctrl.ngpios = num_gpios; + + return msm_pinctrl_probe(pdev, &qdf2xxx_pinctrl); +} + +static const struct acpi_device_id qdf2xxx_acpi_ids[] = { + {"QCOM8001"}, + {}, +}; +MODULE_DEVICE_TABLE(acpi, qdf2xxx_acpi_ids); + +static struct platform_driver qdf2xxx_pinctrl_driver = { + .driver = { + .name = "qdf2xxx-pinctrl", + .acpi_match_table = ACPI_PTR(qdf2xxx_acpi_ids), + }, + .probe = qdf2xxx_pinctrl_probe, + .remove = msm_pinctrl_remove, +}; + +static int __init qdf2xxx_pinctrl_init(void) +{ + return platform_driver_register(&qdf2xxx_pinctrl_driver); +} +arch_initcall(qdf2xxx_pinctrl_init); + +static void __exit qdf2xxx_pinctrl_exit(void) +{ + platform_driver_unregister(&qdf2xxx_pinctrl_driver); +} +module_exit(qdf2xxx_pinctrl_exit); + +MODULE_DESCRIPTION("Qualcomm Technologies QDF2xxx pin control driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pinctrl/qcom/pinctrl-spmi-mpp.c b/drivers/pinctrl/qcom/pinctrl-spmi-mpp.c index 3121de9b63310c..e3be3ce2cada2e 100644 --- a/drivers/pinctrl/qcom/pinctrl-spmi-mpp.c +++ b/drivers/pinctrl/qcom/pinctrl-spmi-mpp.c @@ -61,7 +61,9 @@ #define PMIC_MPP_REG_DIG_PULL_CTL 0x42 #define PMIC_MPP_REG_DIG_IN_CTL 0x43 #define PMIC_MPP_REG_EN_CTL 0x46 +#define PMIC_MPP_REG_AOUT_CTL 0x48 #define PMIC_MPP_REG_AIN_CTL 0x4a +#define PMIC_MPP_REG_SINK_CTL 0x4c /* PMIC_MPP_REG_MODE_CTL */ #define PMIC_MPP_REG_MODE_VALUE_MASK 0x1 @@ -85,11 +87,25 @@ #define PMIC_MPP_REG_AIN_ROUTE_SHIFT 0 #define PMIC_MPP_REG_AIN_ROUTE_MASK 0x7 +#define PMIC_MPP_MODE_DIGITAL_INPUT 0 +#define PMIC_MPP_MODE_DIGITAL_OUTPUT 1 +#define PMIC_MPP_MODE_DIGITAL_BIDIR 2 +#define PMIC_MPP_MODE_ANALOG_BIDIR 3 +#define PMIC_MPP_MODE_ANALOG_INPUT 4 +#define PMIC_MPP_MODE_ANALOG_OUTPUT 5 +#define PMIC_MPP_MODE_CURRENT_SINK 6 + +#define PMIC_MPP_SELECTOR_NORMAL 0 +#define PMIC_MPP_SELECTOR_PAIRED 1 +#define PMIC_MPP_SELECTOR_DTEST_FIRST 4 + #define PMIC_MPP_PHYSICAL_OFFSET 1 /* Qualcomm specific pin configurations */ #define PMIC_MPP_CONF_AMUX_ROUTE (PIN_CONFIG_END + 1) -#define PMIC_MPP_CONF_ANALOG_MODE (PIN_CONFIG_END + 2) +#define PMIC_MPP_CONF_ANALOG_LEVEL (PIN_CONFIG_END + 2) +#define PMIC_MPP_CONF_DTEST_SELECTOR (PIN_CONFIG_END + 3) +#define PMIC_MPP_CONF_PAIRED (PIN_CONFIG_END + 4) /** * struct pmic_mpp_pad - keep current MPP settings @@ -99,13 +115,15 @@ * @out_value: Cached pin output value. * @output_enabled: Set to true if MPP output logic is enabled. * @input_enabled: Set to true if MPP input buffer logic is enabled. - * @analog_mode: Set to true when MPP should operate in Analog Input, Analog - * Output or Bidirectional Analog mode. + * @paired: Pin operates in paired mode * @num_sources: Number of power-sources supported by this MPP. * @power_source: Current power-source used. * @amux_input: Set the source for analog input. + * @aout_level: Analog output level * @pullup: Pullup resistor value. Valid in Bidirectional mode only. * @function: See pmic_mpp_functions[]. + * @drive_strength: Amount of current in sink mode + * @dtest: DTEST route selector */ struct pmic_mpp_pad { u16 base; @@ -114,12 +132,15 @@ struct pmic_mpp_pad { bool out_value; bool output_enabled; bool input_enabled; - bool analog_mode; + bool paired; unsigned int num_sources; unsigned int power_source; unsigned int amux_input; + unsigned int aout_level; unsigned int pullup; unsigned int function; + unsigned int drive_strength; + unsigned int dtest; }; struct pmic_mpp_state { @@ -129,25 +150,32 @@ struct pmic_mpp_state { struct gpio_chip chip; }; -struct pmic_mpp_bindings { - const char *property; - unsigned param; +static const struct pinconf_generic_params pmic_mpp_bindings[] = { + {"qcom,amux-route", PMIC_MPP_CONF_AMUX_ROUTE, 0}, + {"qcom,analog-level", PMIC_MPP_CONF_ANALOG_LEVEL, 0}, + {"qcom,dtest", PMIC_MPP_CONF_DTEST_SELECTOR, 0}, + {"qcom,paired", PMIC_MPP_CONF_PAIRED, 0}, }; -static struct pmic_mpp_bindings pmic_mpp_bindings[] = { - {"qcom,amux-route", PMIC_MPP_CONF_AMUX_ROUTE}, - {"qcom,analog-mode", PMIC_MPP_CONF_ANALOG_MODE}, +#ifdef CONFIG_DEBUG_FS +static const struct pin_config_item pmic_conf_items[] = { + PCONFDUMP(PMIC_MPP_CONF_AMUX_ROUTE, "analog mux", NULL, true), + PCONFDUMP(PMIC_MPP_CONF_ANALOG_LEVEL, "analog level", NULL, true), + PCONFDUMP(PMIC_MPP_CONF_DTEST_SELECTOR, "dtest", NULL, true), + PCONFDUMP(PMIC_MPP_CONF_PAIRED, "paired", NULL, false), }; +#endif static const char *const pmic_mpp_groups[] = { "mpp1", "mpp2", "mpp3", "mpp4", "mpp5", "mpp6", "mpp7", "mpp8", }; +#define PMIC_MPP_DIGITAL 0 +#define PMIC_MPP_ANALOG 1 +#define PMIC_MPP_SINK 2 + static const char *const pmic_mpp_functions[] = { - PMIC_MPP_FUNC_NORMAL, PMIC_MPP_FUNC_PAIRED, - "reserved1", "reserved2", - PMIC_MPP_FUNC_DTEST1, PMIC_MPP_FUNC_DTEST2, - PMIC_MPP_FUNC_DTEST3, PMIC_MPP_FUNC_DTEST4, + "digital", "analog", "sink" }; static inline struct pmic_mpp_state *to_mpp_state(struct gpio_chip *chip) @@ -204,118 +232,11 @@ static int pmic_mpp_get_group_pins(struct pinctrl_dev *pctldev, return 0; } -static int pmic_mpp_parse_dt_config(struct device_node *np, - struct pinctrl_dev *pctldev, - unsigned long **configs, - unsigned int *nconfs) -{ - struct pmic_mpp_bindings *par; - unsigned long cfg; - int ret, i; - u32 val; - - for (i = 0; i < ARRAY_SIZE(pmic_mpp_bindings); i++) { - par = &pmic_mpp_bindings[i]; - ret = of_property_read_u32(np, par->property, &val); - - /* property not found */ - if (ret == -EINVAL) - continue; - - /* use zero as default value, when no value is specified */ - if (ret) - val = 0; - - dev_dbg(pctldev->dev, "found %s with value %u\n", - par->property, val); - - cfg = pinconf_to_config_packed(par->param, val); - - ret = pinctrl_utils_add_config(pctldev, configs, nconfs, cfg); - if (ret) - return ret; - } - - return 0; -} - -static int pmic_mpp_dt_subnode_to_map(struct pinctrl_dev *pctldev, - struct device_node *np, - struct pinctrl_map **map, - unsigned *reserv, unsigned *nmaps, - enum pinctrl_map_type type) -{ - unsigned long *configs = NULL; - unsigned nconfs = 0; - struct property *prop; - const char *group; - int ret; - - ret = pmic_mpp_parse_dt_config(np, pctldev, &configs, &nconfs); - if (ret < 0) - return ret; - - if (!nconfs) - return 0; - - ret = of_property_count_strings(np, "pins"); - if (ret < 0) - goto exit; - - ret = pinctrl_utils_reserve_map(pctldev, map, reserv, nmaps, ret); - if (ret < 0) - goto exit; - - of_property_for_each_string(np, "pins", prop, group) { - ret = pinctrl_utils_add_map_configs(pctldev, map, - reserv, nmaps, group, - configs, nconfs, type); - if (ret < 0) - break; - } -exit: - kfree(configs); - return ret; -} - -static int pmic_mpp_dt_node_to_map(struct pinctrl_dev *pctldev, - struct device_node *np_config, - struct pinctrl_map **map, unsigned *nmaps) -{ - struct device_node *np; - enum pinctrl_map_type type; - unsigned reserv; - int ret; - - ret = 0; - *map = NULL; - *nmaps = 0; - reserv = 0; - type = PIN_MAP_TYPE_CONFIGS_GROUP; - - for_each_child_of_node(np_config, np) { - ret = pinconf_generic_dt_subnode_to_map(pctldev, np, map, - &reserv, nmaps, type); - if (ret) - break; - - ret = pmic_mpp_dt_subnode_to_map(pctldev, np, map, &reserv, - nmaps, type); - if (ret) - break; - } - - if (ret < 0) - pinctrl_utils_dt_free_map(pctldev, *map, *nmaps); - - return ret; -} - static const struct pinctrl_ops pmic_mpp_pinctrl_ops = { .get_groups_count = pmic_mpp_get_groups_count, .get_group_name = pmic_mpp_get_group_name, .get_group_pins = pmic_mpp_get_group_pins, - .dt_node_to_map = pmic_mpp_dt_node_to_map, + .dt_node_to_map = pinconf_generic_dt_node_to_map_group, .dt_free_map = pinctrl_utils_dt_free_map, }; @@ -340,6 +261,53 @@ static int pmic_mpp_get_function_groups(struct pinctrl_dev *pctldev, return 0; } +static int pmic_mpp_write_mode_ctl(struct pmic_mpp_state *state, + struct pmic_mpp_pad *pad) +{ + unsigned int mode; + unsigned int sel; + unsigned int val; + unsigned int en; + + switch (pad->function) { + case PMIC_MPP_ANALOG: + if (pad->input_enabled && pad->output_enabled) + mode = PMIC_MPP_MODE_ANALOG_BIDIR; + else if (pad->input_enabled) + mode = PMIC_MPP_MODE_ANALOG_INPUT; + else + mode = PMIC_MPP_MODE_ANALOG_OUTPUT; + break; + case PMIC_MPP_DIGITAL: + if (pad->input_enabled && pad->output_enabled) + mode = PMIC_MPP_MODE_DIGITAL_BIDIR; + else if (pad->input_enabled) + mode = PMIC_MPP_MODE_DIGITAL_INPUT; + else + mode = PMIC_MPP_MODE_DIGITAL_OUTPUT; + break; + case PMIC_MPP_SINK: + default: + mode = PMIC_MPP_MODE_CURRENT_SINK; + break; + } + + if (pad->dtest) + sel = PMIC_MPP_SELECTOR_DTEST_FIRST + pad->dtest - 1; + else if (pad->paired) + sel = PMIC_MPP_SELECTOR_PAIRED; + else + sel = PMIC_MPP_SELECTOR_NORMAL; + + en = !!pad->out_value; + + val = mode << PMIC_MPP_REG_MODE_DIR_SHIFT | + sel << PMIC_MPP_REG_MODE_FUNCTION_SHIFT | + en; + + return pmic_mpp_write(state, pad, PMIC_MPP_REG_MODE_CTL, val); +} + static int pmic_mpp_set_mux(struct pinctrl_dev *pctldev, unsigned function, unsigned pin) { @@ -352,31 +320,7 @@ static int pmic_mpp_set_mux(struct pinctrl_dev *pctldev, unsigned function, pad->function = function; - if (!pad->analog_mode) { - val = 0; /* just digital input */ - if (pad->output_enabled) { - if (pad->input_enabled) - val = 2; /* digital input and output */ - else - val = 1; /* just digital output */ - } - } else { - val = 4; /* just analog input */ - if (pad->output_enabled) { - if (pad->input_enabled) - val = 3; /* analog input and output */ - else - val = 5; /* just analog output */ - } - } - - val = val << PMIC_MPP_REG_MODE_DIR_SHIFT; - val |= pad->function << PMIC_MPP_REG_MODE_FUNCTION_SHIFT; - val |= pad->out_value & PMIC_MPP_REG_MODE_VALUE_MASK; - - ret = pmic_mpp_write(state, pad, PMIC_MPP_REG_MODE_CTL, val); - if (ret < 0) - return ret; + ret = pmic_mpp_write_mode_ctl(state, pad); val = pad->is_enabled << PMIC_MPP_REG_MASTER_EN_SHIFT; @@ -433,11 +377,20 @@ static int pmic_mpp_config_get(struct pinctrl_dev *pctldev, case PIN_CONFIG_OUTPUT: arg = pad->out_value; break; + case PMIC_MPP_CONF_DTEST_SELECTOR: + arg = pad->dtest; + break; case PMIC_MPP_CONF_AMUX_ROUTE: arg = pad->amux_input; break; - case PMIC_MPP_CONF_ANALOG_MODE: - arg = pad->analog_mode; + case PMIC_MPP_CONF_PAIRED: + arg = pad->paired; + break; + case PIN_CONFIG_DRIVE_STRENGTH: + arg = pad->drive_strength; + break; + case PMIC_MPP_CONF_ANALOG_LEVEL: + arg = pad->aout_level; break; default: return -EINVAL; @@ -459,6 +412,9 @@ static int pmic_mpp_config_set(struct pinctrl_dev *pctldev, unsigned int pin, pad = pctldev->desc->pins[pin].drv_data; + /* Make it possible to enable the pin, by not setting high impedance */ + pad->is_enabled = true; + for (i = 0; i < nconfs; i++) { param = pinconf_to_config_param(configs[i]); arg = pinconf_to_config_argument(configs[i]); @@ -497,13 +453,22 @@ static int pmic_mpp_config_set(struct pinctrl_dev *pctldev, unsigned int pin, pad->output_enabled = true; pad->out_value = arg; break; + case PMIC_MPP_CONF_DTEST_SELECTOR: + pad->dtest = arg; + break; + case PIN_CONFIG_DRIVE_STRENGTH: + arg = pad->drive_strength; + break; case PMIC_MPP_CONF_AMUX_ROUTE: if (arg >= PMIC_MPP_AMUX_ROUTE_ABUS4) return -EINVAL; pad->amux_input = arg; break; - case PMIC_MPP_CONF_ANALOG_MODE: - pad->analog_mode = true; + case PMIC_MPP_CONF_ANALOG_LEVEL: + pad->aout_level = arg; + break; + case PMIC_MPP_CONF_PAIRED: + pad->paired = !!arg; break; default: return -EINVAL; @@ -528,29 +493,17 @@ static int pmic_mpp_config_set(struct pinctrl_dev *pctldev, unsigned int pin, if (ret < 0) return ret; - if (!pad->analog_mode) { - val = 0; /* just digital input */ - if (pad->output_enabled) { - if (pad->input_enabled) - val = 2; /* digital input and output */ - else - val = 1; /* just digital output */ - } - } else { - val = 4; /* just analog input */ - if (pad->output_enabled) { - if (pad->input_enabled) - val = 3; /* analog input and output */ - else - val = 5; /* just analog output */ - } - } + ret = pmic_mpp_write(state, pad, PMIC_MPP_REG_AOUT_CTL, pad->aout_level); + if (ret < 0) + return ret; + + ret = pmic_mpp_write_mode_ctl(state, pad); + if (ret < 0) + return ret; - val = val << PMIC_MPP_REG_MODE_DIR_SHIFT; - val |= pad->function << PMIC_MPP_REG_MODE_FUNCTION_SHIFT; - val |= pad->out_value & PMIC_MPP_REG_MODE_VALUE_MASK; + val = pad->is_enabled << PMIC_MPP_REG_MASTER_EN_SHIFT; - return pmic_mpp_write(state, pad, PMIC_MPP_REG_MODE_CTL, val); + return pmic_mpp_write(state, pad, PMIC_MPP_REG_EN_CTL, val); } static void pmic_mpp_config_dbg_show(struct pinctrl_dev *pctldev, @@ -558,20 +511,17 @@ static void pmic_mpp_config_dbg_show(struct pinctrl_dev *pctldev, { struct pmic_mpp_state *state = pinctrl_dev_get_drvdata(pctldev); struct pmic_mpp_pad *pad; - int ret, val; + int ret; static const char *const biases[] = { "0.6kOhm", "10kOhm", "30kOhm", "Disabled" }; - pad = pctldev->desc->pins[pin].drv_data; seq_printf(s, " mpp%-2d:", pin + PMIC_MPP_PHYSICAL_OFFSET); - val = pmic_mpp_read(state, pad, PMIC_MPP_REG_EN_CTL); - - if (val < 0 || !(val >> PMIC_MPP_REG_MASTER_EN_SHIFT)) { + if (!pad->is_enabled) { seq_puts(s, " ---"); } else { @@ -585,15 +535,20 @@ static void pmic_mpp_config_dbg_show(struct pinctrl_dev *pctldev, } seq_printf(s, " %-4s", pad->output_enabled ? "out" : "in"); - seq_printf(s, " %-4s", pad->analog_mode ? "ana" : "dig"); seq_printf(s, " %-7s", pmic_mpp_functions[pad->function]); seq_printf(s, " vin-%d", pad->power_source); + seq_printf(s, " %d", pad->aout_level); seq_printf(s, " %-8s", biases[pad->pullup]); seq_printf(s, " %-4s", pad->out_value ? "high" : "low"); + if (pad->dtest) + seq_printf(s, " dtest%d", pad->dtest); + if (pad->paired) + seq_puts(s, " paired"); } } static const struct pinconf_ops pmic_mpp_pinconf_ops = { + .is_generic = true, .pin_config_group_get = pmic_mpp_config_get, .pin_config_group_set = pmic_mpp_config_set, .pin_config_group_dbg_show = pmic_mpp_config_dbg_show, @@ -709,6 +664,7 @@ static int pmic_mpp_populate(struct pmic_mpp_state *state, struct pmic_mpp_pad *pad) { int type, subtype, val, dir; + unsigned int sel; type = pmic_mpp_read(state, pad, PMIC_MPP_REG_TYPE); if (type < 0) @@ -751,43 +707,53 @@ static int pmic_mpp_populate(struct pmic_mpp_state *state, dir &= PMIC_MPP_REG_MODE_DIR_MASK; switch (dir) { - case 0: + case PMIC_MPP_MODE_DIGITAL_INPUT: pad->input_enabled = true; pad->output_enabled = false; - pad->analog_mode = false; + pad->function = PMIC_MPP_DIGITAL; break; - case 1: + case PMIC_MPP_MODE_DIGITAL_OUTPUT: pad->input_enabled = false; pad->output_enabled = true; - pad->analog_mode = false; + pad->function = PMIC_MPP_DIGITAL; break; - case 2: + case PMIC_MPP_MODE_DIGITAL_BIDIR: pad->input_enabled = true; pad->output_enabled = true; - pad->analog_mode = false; + pad->function = PMIC_MPP_DIGITAL; break; - case 3: + case PMIC_MPP_MODE_ANALOG_BIDIR: pad->input_enabled = true; pad->output_enabled = true; - pad->analog_mode = true; + pad->function = PMIC_MPP_ANALOG; break; - case 4: + case PMIC_MPP_MODE_ANALOG_INPUT: pad->input_enabled = true; pad->output_enabled = false; - pad->analog_mode = true; + pad->function = PMIC_MPP_ANALOG; break; - case 5: + case PMIC_MPP_MODE_ANALOG_OUTPUT: pad->input_enabled = false; pad->output_enabled = true; - pad->analog_mode = true; + pad->function = PMIC_MPP_ANALOG; + break; + case PMIC_MPP_MODE_CURRENT_SINK: + pad->input_enabled = false; + pad->output_enabled = true; + pad->function = PMIC_MPP_SINK; break; default: dev_err(state->dev, "unknown MPP direction\n"); return -ENODEV; } - pad->function = val >> PMIC_MPP_REG_MODE_FUNCTION_SHIFT; - pad->function &= PMIC_MPP_REG_MODE_FUNCTION_MASK; + sel = val >> PMIC_MPP_REG_MODE_FUNCTION_SHIFT; + sel &= PMIC_MPP_REG_MODE_FUNCTION_MASK; + + if (sel >= PMIC_MPP_SELECTOR_DTEST_FIRST) + pad->dtest = sel + 1; + else if (sel == PMIC_MPP_SELECTOR_PAIRED) + pad->paired = true; val = pmic_mpp_read(state, pad, PMIC_MPP_REG_DIG_VIN_CTL); if (val < 0) @@ -810,8 +776,24 @@ static int pmic_mpp_populate(struct pmic_mpp_state *state, pad->amux_input = val >> PMIC_MPP_REG_AIN_ROUTE_SHIFT; pad->amux_input &= PMIC_MPP_REG_AIN_ROUTE_MASK; - /* Pin could be disabled with PIN_CONFIG_BIAS_HIGH_IMPEDANCE */ - pad->is_enabled = true; + val = pmic_mpp_read(state, pad, PMIC_MPP_REG_SINK_CTL); + if (val < 0) + return val; + + pad->drive_strength = val; + + val = pmic_mpp_read(state, pad, PMIC_MPP_REG_AOUT_CTL); + if (val < 0) + return val; + + pad->aout_level = val; + + val = pmic_mpp_read(state, pad, PMIC_MPP_REG_EN_CTL); + if (val < 0) + return val; + + pad->is_enabled = !!val; + return 0; } @@ -866,6 +848,12 @@ static int pmic_mpp_probe(struct platform_device *pdev) pctrldesc->pins = pindesc; pctrldesc->npins = npins; + pctrldesc->num_custom_params = ARRAY_SIZE(pmic_mpp_bindings); + pctrldesc->custom_params = pmic_mpp_bindings; +#ifdef CONFIG_DEBUG_FS + pctrldesc->custom_conf_items = pmic_conf_items; +#endif + for (i = 0; i < npins; i++, pindesc++) { pad = &pads[i]; pindesc->drv_data = pad; diff --git a/drivers/pinctrl/qcom/pinctrl-ssbi-gpio.c b/drivers/pinctrl/qcom/pinctrl-ssbi-gpio.c new file mode 100644 index 00000000000000..c978b311031b52 --- /dev/null +++ b/drivers/pinctrl/qcom/pinctrl-ssbi-gpio.c @@ -0,0 +1,791 @@ +/* + * Copyright (c) 2015, Sony Mobile Communications AB. + * Copyright (c) 2013, The Linux Foundation. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 and + * only version 2 as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "../core.h" +#include "../pinctrl-utils.h" + +/* mode */ +#define PM8XXX_GPIO_MODE_ENABLED BIT(0) +#define PM8XXX_GPIO_MODE_INPUT 0 +#define PM8XXX_GPIO_MODE_OUTPUT 2 + +/* output buffer */ +#define PM8XXX_GPIO_PUSH_PULL 0 +#define PM8XXX_GPIO_OPEN_DRAIN 1 + +/* bias */ +#define PM8XXX_GPIO_BIAS_PU_30 0 +#define PM8XXX_GPIO_BIAS_PU_1P5 1 +#define PM8XXX_GPIO_BIAS_PU_31P5 2 +#define PM8XXX_GPIO_BIAS_PU_1P5_30 3 +#define PM8XXX_GPIO_BIAS_PD 4 +#define PM8XXX_GPIO_BIAS_NP 5 + +/* GPIO registers */ +#define SSBI_REG_ADDR_GPIO_BASE 0x150 +#define SSBI_REG_ADDR_GPIO(n) (SSBI_REG_ADDR_GPIO_BASE + n) + +#define PM8XXX_BANK_WRITE BIT(7) + +#define PM8XXX_MAX_GPIOS 44 + +/* custom pinconf parameters */ +#define PM8XXX_QCOM_DRIVE_STRENGH (PIN_CONFIG_END + 1) +#define PM8XXX_QCOM_PULL_UP_STRENGTH (PIN_CONFIG_END + 2) + +/** + * struct pm8xxx_pin_data - dynamic configuration for a pin + * @reg: address of the control register + * @irq: IRQ from the PMIC interrupt controller + * @power_source: logical selected voltage source, mapping in static data + * is used translate to register values + * @mode: operating mode for the pin (input/output) + * @open_drain: output buffer configured as open-drain (vs push-pull) + * @output_value: configured output value + * @bias: register view of configured bias + * @pull_up_strength: placeholder for selected pull up strength + * only used to configure bias when pull up is selected + * @output_strength: selector of output-strength + * @disable: pin disabled / configured as tristate + * @function: pinmux selector + * @inverted: pin logic is inverted + */ +struct pm8xxx_pin_data { + unsigned reg; + int irq; + u8 power_source; + u8 mode; + bool open_drain; + bool output_value; + u8 bias; + u8 pull_up_strength; + u8 output_strength; + bool disable; + u8 function; + bool inverted; +}; + +struct pm8xxx_gpio { + struct device *dev; + struct regmap *regmap; + struct pinctrl_dev *pctrl; + struct gpio_chip chip; + + struct pinctrl_desc desc; + unsigned npins; +}; + +static const struct pinconf_generic_params pm8xxx_gpio_bindings[] = { + {"qcom,drive-strength", PM8XXX_QCOM_DRIVE_STRENGH, 0}, + {"qcom,pull-up-strength", PM8XXX_QCOM_PULL_UP_STRENGTH, 0}, +}; + +#ifdef CONFIG_DEBUG_FS +static const struct pin_config_item pm8xxx_conf_items[ARRAY_SIZE(pm8xxx_gpio_bindings)] = { + PCONFDUMP(PM8XXX_QCOM_DRIVE_STRENGH, "drive-strength", NULL, true), + PCONFDUMP(PM8XXX_QCOM_PULL_UP_STRENGTH, "pull up strength", NULL, true), +}; +#endif + +static const char * const pm8xxx_groups[PM8XXX_MAX_GPIOS] = { + "gpio1", "gpio2", "gpio3", "gpio4", "gpio5", "gpio6", "gpio7", "gpio8", + "gpio9", "gpio10", "gpio11", "gpio12", "gpio13", "gpio14", "gpio15", + "gpio16", "gpio17", "gpio18", "gpio19", "gpio20", "gpio21", "gpio22", + "gpio23", "gpio24", "gpio25", "gpio26", "gpio27", "gpio28", "gpio29", + "gpio30", "gpio31", "gpio32", "gpio33", "gpio34", "gpio35", "gpio36", + "gpio37", "gpio38", "gpio39", "gpio40", "gpio41", "gpio42", "gpio43", + "gpio44", +}; + +static const char * const pm8xxx_gpio_functions[] = { + PMIC_GPIO_FUNC_NORMAL, PMIC_GPIO_FUNC_PAIRED, + PMIC_GPIO_FUNC_FUNC1, PMIC_GPIO_FUNC_FUNC2, + PMIC_GPIO_FUNC_DTEST1, PMIC_GPIO_FUNC_DTEST2, + PMIC_GPIO_FUNC_DTEST3, PMIC_GPIO_FUNC_DTEST4, +}; + +static int pm8xxx_read_bank(struct pm8xxx_gpio *pctrl, + struct pm8xxx_pin_data *pin, int bank) +{ + unsigned int val = bank << 4; + int ret; + + ret = regmap_write(pctrl->regmap, pin->reg, val); + if (ret) { + dev_err(pctrl->dev, "failed to select bank %d\n", bank); + return ret; + } + + ret = regmap_read(pctrl->regmap, pin->reg, &val); + if (ret) { + dev_err(pctrl->dev, "failed to read register %d\n", bank); + return ret; + } + + return val; +} + +static int pm8xxx_write_bank(struct pm8xxx_gpio *pctrl, + struct pm8xxx_pin_data *pin, + int bank, + u8 val) +{ + int ret; + + val |= PM8XXX_BANK_WRITE; + val |= bank << 4; + + ret = regmap_write(pctrl->regmap, pin->reg, val); + if (ret) + dev_err(pctrl->dev, "failed to write register\n"); + + return ret; +} + +static int pm8xxx_get_groups_count(struct pinctrl_dev *pctldev) +{ + struct pm8xxx_gpio *pctrl = pinctrl_dev_get_drvdata(pctldev); + + return pctrl->npins; +} + +static const char *pm8xxx_get_group_name(struct pinctrl_dev *pctldev, + unsigned group) +{ + return pm8xxx_groups[group]; +} + + +static int pm8xxx_get_group_pins(struct pinctrl_dev *pctldev, + unsigned group, + const unsigned **pins, + unsigned *num_pins) +{ + struct pm8xxx_gpio *pctrl = pinctrl_dev_get_drvdata(pctldev); + + *pins = &pctrl->desc.pins[group].number; + *num_pins = 1; + + return 0; +} + +static const struct pinctrl_ops pm8xxx_pinctrl_ops = { + .get_groups_count = pm8xxx_get_groups_count, + .get_group_name = pm8xxx_get_group_name, + .get_group_pins = pm8xxx_get_group_pins, + .dt_node_to_map = pinconf_generic_dt_node_to_map_group, + .dt_free_map = pinctrl_utils_dt_free_map, +}; + +static int pm8xxx_get_functions_count(struct pinctrl_dev *pctldev) +{ + return ARRAY_SIZE(pm8xxx_gpio_functions); +} + +static const char *pm8xxx_get_function_name(struct pinctrl_dev *pctldev, + unsigned function) +{ + return pm8xxx_gpio_functions[function]; +} + +static int pm8xxx_get_function_groups(struct pinctrl_dev *pctldev, + unsigned function, + const char * const **groups, + unsigned * const num_groups) +{ + struct pm8xxx_gpio *pctrl = pinctrl_dev_get_drvdata(pctldev); + + *groups = pm8xxx_groups; + *num_groups = pctrl->npins; + return 0; +} + +static int pm8xxx_pinmux_set_mux(struct pinctrl_dev *pctldev, + unsigned function, + unsigned group) +{ + struct pm8xxx_gpio *pctrl = pinctrl_dev_get_drvdata(pctldev); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[group].drv_data; + u8 val; + + pin->function = function; + val = pin->function << 1; + + pm8xxx_write_bank(pctrl, pin, 4, val); + + return 0; +} + +static const struct pinmux_ops pm8xxx_pinmux_ops = { + .get_functions_count = pm8xxx_get_functions_count, + .get_function_name = pm8xxx_get_function_name, + .get_function_groups = pm8xxx_get_function_groups, + .set_mux = pm8xxx_pinmux_set_mux, +}; + +static int pm8xxx_pin_config_get(struct pinctrl_dev *pctldev, + unsigned int offset, + unsigned long *config) +{ + struct pm8xxx_gpio *pctrl = pinctrl_dev_get_drvdata(pctldev); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + unsigned param = pinconf_to_config_param(*config); + unsigned arg; + + switch (param) { + case PIN_CONFIG_BIAS_DISABLE: + arg = pin->bias == PM8XXX_GPIO_BIAS_NP; + break; + case PIN_CONFIG_BIAS_PULL_DOWN: + arg = pin->bias == PM8XXX_GPIO_BIAS_PD; + break; + case PIN_CONFIG_BIAS_PULL_UP: + arg = pin->bias <= PM8XXX_GPIO_BIAS_PU_1P5_30; + break; + case PM8XXX_QCOM_PULL_UP_STRENGTH: + arg = pin->pull_up_strength; + break; + case PIN_CONFIG_BIAS_HIGH_IMPEDANCE: + arg = pin->disable; + break; + case PIN_CONFIG_INPUT_ENABLE: + arg = pin->mode == PM8XXX_GPIO_MODE_INPUT; + break; + case PIN_CONFIG_OUTPUT: + if (pin->mode & PM8XXX_GPIO_MODE_OUTPUT) + arg = pin->output_value; + else + arg = 0; + break; + case PIN_CONFIG_POWER_SOURCE: + arg = pin->power_source; + break; + case PM8XXX_QCOM_DRIVE_STRENGH: + arg = pin->output_strength; + break; + case PIN_CONFIG_DRIVE_PUSH_PULL: + arg = !pin->open_drain; + break; + case PIN_CONFIG_DRIVE_OPEN_DRAIN: + arg = pin->open_drain; + break; + default: + return -EINVAL; + } + + *config = pinconf_to_config_packed(param, arg); + + return 0; +} + +static int pm8xxx_pin_config_set(struct pinctrl_dev *pctldev, + unsigned int offset, + unsigned long *configs, + unsigned num_configs) +{ + struct pm8xxx_gpio *pctrl = pinctrl_dev_get_drvdata(pctldev); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + unsigned param; + unsigned arg; + unsigned i; + u8 banks = 0; + u8 val; + + for (i = 0; i < num_configs; i++) { + param = pinconf_to_config_param(configs[i]); + arg = pinconf_to_config_argument(configs[i]); + + switch (param) { + case PIN_CONFIG_BIAS_DISABLE: + pin->bias = PM8XXX_GPIO_BIAS_NP; + banks |= BIT(2); + pin->disable = 0; + banks |= BIT(3); + break; + case PIN_CONFIG_BIAS_PULL_DOWN: + pin->bias = PM8XXX_GPIO_BIAS_PD; + banks |= BIT(2); + pin->disable = 0; + banks |= BIT(3); + break; + case PM8XXX_QCOM_PULL_UP_STRENGTH: + if (arg > PM8XXX_GPIO_BIAS_PU_1P5_30) { + dev_err(pctrl->dev, "invalid pull-up strength\n"); + return -EINVAL; + } + pin->pull_up_strength = arg; + /* FALLTHROUGH */ + case PIN_CONFIG_BIAS_PULL_UP: + pin->bias = pin->pull_up_strength; + banks |= BIT(2); + pin->disable = 0; + banks |= BIT(3); + break; + case PIN_CONFIG_BIAS_HIGH_IMPEDANCE: + pin->disable = 1; + banks |= BIT(3); + break; + case PIN_CONFIG_INPUT_ENABLE: + pin->mode = PM8XXX_GPIO_MODE_INPUT; + banks |= BIT(0) | BIT(1); + break; + case PIN_CONFIG_OUTPUT: + pin->mode = PM8XXX_GPIO_MODE_OUTPUT; + pin->output_value = !!arg; + banks |= BIT(0) | BIT(1); + break; + case PIN_CONFIG_POWER_SOURCE: + pin->power_source = arg; + banks |= BIT(0); + break; + case PM8XXX_QCOM_DRIVE_STRENGH: + if (arg > PMIC_GPIO_STRENGTH_LOW) { + dev_err(pctrl->dev, "invalid drive strength\n"); + return -EINVAL; + } + pin->output_strength = arg; + banks |= BIT(3); + break; + case PIN_CONFIG_DRIVE_PUSH_PULL: + pin->open_drain = 0; + banks |= BIT(1); + break; + case PIN_CONFIG_DRIVE_OPEN_DRAIN: + pin->open_drain = 1; + banks |= BIT(1); + break; + default: + dev_err(pctrl->dev, + "unsupported config parameter: %x\n", + param); + return -EINVAL; + } + } + + if (banks & BIT(0)) { + val = pin->power_source << 1; + val |= PM8XXX_GPIO_MODE_ENABLED; + pm8xxx_write_bank(pctrl, pin, 0, val); + } + + if (banks & BIT(1)) { + val = pin->mode << 2; + val |= pin->open_drain << 1; + val |= pin->output_value; + pm8xxx_write_bank(pctrl, pin, 1, val); + } + + if (banks & BIT(2)) { + val = pin->bias << 1; + pm8xxx_write_bank(pctrl, pin, 2, val); + } + + if (banks & BIT(3)) { + val = pin->output_strength << 2; + val |= pin->disable; + pm8xxx_write_bank(pctrl, pin, 3, val); + } + + if (banks & BIT(4)) { + val = pin->function << 1; + pm8xxx_write_bank(pctrl, pin, 4, val); + } + + if (banks & BIT(5)) { + val = 0; + if (!pin->inverted) + val |= BIT(3); + pm8xxx_write_bank(pctrl, pin, 5, val); + } + + return 0; +} + +static const struct pinconf_ops pm8xxx_pinconf_ops = { + .is_generic = true, + .pin_config_group_get = pm8xxx_pin_config_get, + .pin_config_group_set = pm8xxx_pin_config_set, +}; + +static struct pinctrl_desc pm8xxx_pinctrl_desc = { + .name = "pm8xxx_gpio", + .pctlops = &pm8xxx_pinctrl_ops, + .pmxops = &pm8xxx_pinmux_ops, + .confops = &pm8xxx_pinconf_ops, + .owner = THIS_MODULE, +}; + +static int pm8xxx_gpio_direction_input(struct gpio_chip *chip, + unsigned offset) +{ + struct pm8xxx_gpio *pctrl = container_of(chip, struct pm8xxx_gpio, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + u8 val; + + pin->mode = PM8XXX_GPIO_MODE_INPUT; + val = pin->mode << 2; + + pm8xxx_write_bank(pctrl, pin, 1, val); + + return 0; +} + +static int pm8xxx_gpio_direction_output(struct gpio_chip *chip, + unsigned offset, + int value) +{ + struct pm8xxx_gpio *pctrl = container_of(chip, struct pm8xxx_gpio, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + u8 val; + + pin->mode = PM8XXX_GPIO_MODE_OUTPUT; + pin->output_value = !!value; + + val = pin->mode << 2; + val |= pin->open_drain << 1; + val |= pin->output_value; + + pm8xxx_write_bank(pctrl, pin, 1, val); + + return 0; +} + +static int pm8xxx_gpio_get(struct gpio_chip *chip, unsigned offset) +{ + struct pm8xxx_gpio *pctrl = container_of(chip, struct pm8xxx_gpio, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + bool state; + int ret; + + if (pin->mode == PM8XXX_GPIO_MODE_OUTPUT) { + ret = pin->output_value; + } else { + ret = irq_get_irqchip_state(pin->irq, IRQCHIP_STATE_LINE_LEVEL, &state); + if (!ret) + ret = !!state; + } + + return ret; +} + +static void pm8xxx_gpio_set(struct gpio_chip *chip, unsigned offset, int value) +{ + struct pm8xxx_gpio *pctrl = container_of(chip, struct pm8xxx_gpio, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + u8 val; + + pin->output_value = !!value; + + val = pin->mode << 2; + val |= pin->open_drain << 1; + val |= pin->output_value; + + pm8xxx_write_bank(pctrl, pin, 1, val); +} + +static int pm8xxx_gpio_of_xlate(struct gpio_chip *chip, + const struct of_phandle_args *gpio_desc, + u32 *flags) +{ + if (chip->of_gpio_n_cells < 2) + return -EINVAL; + + if (flags) + *flags = gpio_desc->args[1]; + + return gpio_desc->args[0] - 1; +} + + +static int pm8xxx_gpio_to_irq(struct gpio_chip *chip, unsigned offset) +{ + struct pm8xxx_gpio *pctrl = container_of(chip, struct pm8xxx_gpio, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + + return pin->irq; +} + +#ifdef CONFIG_DEBUG_FS +#include + +static void pm8xxx_gpio_dbg_show_one(struct seq_file *s, + struct pinctrl_dev *pctldev, + struct gpio_chip *chip, + unsigned offset, + unsigned gpio) +{ + struct pm8xxx_gpio *pctrl = container_of(chip, struct pm8xxx_gpio, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + + static const char * const modes[] = { + "in", "both", "out", "off" + }; + static const char * const biases[] = { + "pull-up 30uA", "pull-up 1.5uA", "pull-up 31.5uA", + "pull-up 1.5uA + 30uA boost", "pull-down 10uA", "no pull" + }; + static const char * const buffer_types[] = { + "push-pull", "open-drain" + }; + static const char * const strengths[] = { + "no", "high", "medium", "low" + }; + + seq_printf(s, " gpio%-2d:", offset + 1); + if (pin->disable) { + seq_puts(s, " ---"); + } else { + seq_printf(s, " %-4s", modes[pin->mode]); + seq_printf(s, " %-7s", pm8xxx_gpio_functions[pin->function]); + seq_printf(s, " VIN%d", pin->power_source); + seq_printf(s, " %-27s", biases[pin->bias]); + seq_printf(s, " %-10s", buffer_types[pin->open_drain]); + seq_printf(s, " %-4s", pin->output_value ? "high" : "low"); + seq_printf(s, " %-7s", strengths[pin->output_strength]); + if (pin->inverted) + seq_puts(s, " inverted"); + } +} + +static void pm8xxx_gpio_dbg_show(struct seq_file *s, struct gpio_chip *chip) +{ + unsigned gpio = chip->base; + unsigned i; + + for (i = 0; i < chip->ngpio; i++, gpio++) { + pm8xxx_gpio_dbg_show_one(s, NULL, chip, i, gpio); + seq_puts(s, "\n"); + } +} + +#else +#define msm_gpio_dbg_show NULL +#endif + +static struct gpio_chip pm8xxx_gpio_template = { + .direction_input = pm8xxx_gpio_direction_input, + .direction_output = pm8xxx_gpio_direction_output, + .get = pm8xxx_gpio_get, + .set = pm8xxx_gpio_set, + .of_xlate = pm8xxx_gpio_of_xlate, + .to_irq = pm8xxx_gpio_to_irq, + .dbg_show = pm8xxx_gpio_dbg_show, + .owner = THIS_MODULE, +}; + +static int pm8xxx_pin_populate(struct pm8xxx_gpio *pctrl, + struct pm8xxx_pin_data *pin) +{ + int val; + + val = pm8xxx_read_bank(pctrl, pin, 0); + if (val < 0) + return val; + + pin->power_source = (val >> 1) & 0x7; + + val = pm8xxx_read_bank(pctrl, pin, 1); + if (val < 0) + return val; + + pin->mode = (val >> 2) & 0x3; + pin->open_drain = !!(val & BIT(1)); + pin->output_value = val & BIT(0); + + val = pm8xxx_read_bank(pctrl, pin, 2); + if (val < 0) + return val; + + pin->bias = (val >> 1) & 0x7; + if (pin->bias <= PM8XXX_GPIO_BIAS_PU_1P5_30) + pin->pull_up_strength = pin->bias; + else + pin->pull_up_strength = PM8XXX_GPIO_BIAS_PU_30; + + val = pm8xxx_read_bank(pctrl, pin, 3); + if (val < 0) + return val; + + pin->output_strength = (val >> 2) & 0x3; + pin->disable = val & BIT(0); + + val = pm8xxx_read_bank(pctrl, pin, 4); + if (val < 0) + return val; + + pin->function = (val >> 1) & 0x7; + + val = pm8xxx_read_bank(pctrl, pin, 5); + if (val < 0) + return val; + + pin->inverted = !(val & BIT(3)); + + return 0; +} + +static const struct of_device_id pm8xxx_gpio_of_match[] = { + { .compatible = "qcom,pm8018-gpio", .data = (void *)6 }, + { .compatible = "qcom,pm8038-gpio", .data = (void *)12 }, + { .compatible = "qcom,pm8058-gpio", .data = (void *)40 }, + { .compatible = "qcom,pm8917-gpio", .data = (void *)38 }, + { .compatible = "qcom,pm8921-gpio", .data = (void *)44 }, + { }, +}; +MODULE_DEVICE_TABLE(of, pm8xxx_gpio_of_match); + +static int pm8xxx_gpio_probe(struct platform_device *pdev) +{ + struct pm8xxx_pin_data *pin_data; + struct pinctrl_pin_desc *pins; + struct pm8xxx_gpio *pctrl; + int ret; + int i; + + pctrl = devm_kzalloc(&pdev->dev, sizeof(*pctrl), GFP_KERNEL); + if (!pctrl) + return -ENOMEM; + + pctrl->dev = &pdev->dev; + pctrl->npins = (unsigned)of_device_get_match_data(&pdev->dev); + + pctrl->regmap = dev_get_regmap(pdev->dev.parent, NULL); + if (!pctrl->regmap) { + dev_err(&pdev->dev, "parent regmap unavailable\n"); + return -ENXIO; + } + + pctrl->desc = pm8xxx_pinctrl_desc; + pctrl->desc.npins = pctrl->npins; + + pins = devm_kcalloc(&pdev->dev, + pctrl->desc.npins, + sizeof(struct pinctrl_pin_desc), + GFP_KERNEL); + if (!pins) + return -ENOMEM; + + pin_data = devm_kcalloc(&pdev->dev, + pctrl->desc.npins, + sizeof(struct pm8xxx_pin_data), + GFP_KERNEL); + if (!pin_data) + return -ENOMEM; + + for (i = 0; i < pctrl->desc.npins; i++) { + pin_data[i].reg = SSBI_REG_ADDR_GPIO(i); + pin_data[i].irq = platform_get_irq(pdev, i); + if (pin_data[i].irq < 0) { + dev_err(&pdev->dev, + "missing interrupts for pin %d\n", i); + return pin_data[i].irq; + } + + ret = pm8xxx_pin_populate(pctrl, &pin_data[i]); + if (ret) + return ret; + + pins[i].number = i; + pins[i].name = pm8xxx_groups[i]; + pins[i].drv_data = &pin_data[i]; + } + pctrl->desc.pins = pins; + + pctrl->desc.num_custom_params = ARRAY_SIZE(pm8xxx_gpio_bindings); + pctrl->desc.custom_params = pm8xxx_gpio_bindings; +#ifdef CONFIG_DEBUG_FS + pctrl->desc.custom_conf_items = pm8xxx_conf_items; +#endif + + pctrl->pctrl = pinctrl_register(&pctrl->desc, &pdev->dev, pctrl); + if (!pctrl->pctrl) { + dev_err(&pdev->dev, "couldn't register pm8xxx gpio driver\n"); + return -ENODEV; + } + + pctrl->chip = pm8xxx_gpio_template; + pctrl->chip.base = -1; + pctrl->chip.dev = &pdev->dev; + pctrl->chip.of_node = pdev->dev.of_node; + pctrl->chip.of_gpio_n_cells = 2; + pctrl->chip.label = dev_name(pctrl->dev); + pctrl->chip.ngpio = pctrl->npins; + ret = gpiochip_add(&pctrl->chip); + if (ret) { + dev_err(&pdev->dev, "failed register gpiochip\n"); + goto unregister_pinctrl; + } + + ret = gpiochip_add_pin_range(&pctrl->chip, + dev_name(pctrl->dev), + 0, 0, pctrl->chip.ngpio); + if (ret) { + dev_err(pctrl->dev, "failed to add pin range\n"); + goto unregister_gpiochip; + } + + platform_set_drvdata(pdev, pctrl); + + dev_dbg(&pdev->dev, "Qualcomm pm8xxx gpio driver probed\n"); + + return 0; + +unregister_gpiochip: + gpiochip_remove(&pctrl->chip); + +unregister_pinctrl: + pinctrl_unregister(pctrl->pctrl); + + return ret; +} + +static int pm8xxx_gpio_remove(struct platform_device *pdev) +{ + struct pm8xxx_gpio *pctrl = platform_get_drvdata(pdev); + + gpiochip_remove(&pctrl->chip); + + pinctrl_unregister(pctrl->pctrl); + + return 0; +} + +static struct platform_driver pm8xxx_gpio_driver = { + .driver = { + .name = "qcom-ssbi-gpio", + .of_match_table = pm8xxx_gpio_of_match, + }, + .probe = pm8xxx_gpio_probe, + .remove = pm8xxx_gpio_remove, +}; + +module_platform_driver(pm8xxx_gpio_driver); + +MODULE_AUTHOR("Bjorn Andersson "); +MODULE_DESCRIPTION("Qualcomm PM8xxx GPIO driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pinctrl/qcom/pinctrl-ssbi-mpp.c b/drivers/pinctrl/qcom/pinctrl-ssbi-mpp.c new file mode 100644 index 00000000000000..2d1b69f171be7c --- /dev/null +++ b/drivers/pinctrl/qcom/pinctrl-ssbi-mpp.c @@ -0,0 +1,882 @@ +/* + * Copyright (c) 2015, Sony Mobile Communications AB. + * Copyright (c) 2013, The Linux Foundation. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 and + * only version 2 as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "../core.h" +#include "../pinctrl-utils.h" + +/* MPP registers */ +#define SSBI_REG_ADDR_MPP_BASE 0x50 +#define SSBI_REG_ADDR_MPP(n) (SSBI_REG_ADDR_MPP_BASE + n) + +/* MPP Type: type */ +#define PM8XXX_MPP_TYPE_D_INPUT 0 +#define PM8XXX_MPP_TYPE_D_OUTPUT 1 +#define PM8XXX_MPP_TYPE_D_BI_DIR 2 +#define PM8XXX_MPP_TYPE_A_INPUT 3 +#define PM8XXX_MPP_TYPE_A_OUTPUT 4 +#define PM8XXX_MPP_TYPE_SINK 5 +#define PM8XXX_MPP_TYPE_DTEST_SINK 6 +#define PM8XXX_MPP_TYPE_DTEST_OUTPUT 7 + +/* Digital Input: control */ +#define PM8XXX_MPP_DIN_TO_INT 0 +#define PM8XXX_MPP_DIN_TO_DBUS1 1 +#define PM8XXX_MPP_DIN_TO_DBUS2 2 +#define PM8XXX_MPP_DIN_TO_DBUS3 3 + +/* Digital Output: control */ +#define PM8XXX_MPP_DOUT_CTRL_LOW 0 +#define PM8XXX_MPP_DOUT_CTRL_HIGH 1 +#define PM8XXX_MPP_DOUT_CTRL_MPP 2 +#define PM8XXX_MPP_DOUT_CTRL_INV_MPP 3 + +/* Bidirectional: control */ +#define PM8XXX_MPP_BI_PULLUP_1KOHM 0 +#define PM8XXX_MPP_BI_PULLUP_OPEN 1 +#define PM8XXX_MPP_BI_PULLUP_10KOHM 2 +#define PM8XXX_MPP_BI_PULLUP_30KOHM 3 + +/* Analog Output: control */ +#define PM8XXX_MPP_AOUT_CTRL_DISABLE 0 +#define PM8XXX_MPP_AOUT_CTRL_ENABLE 1 +#define PM8XXX_MPP_AOUT_CTRL_MPP_HIGH_EN 2 +#define PM8XXX_MPP_AOUT_CTRL_MPP_LOW_EN 3 + +/* Current Sink: control */ +#define PM8XXX_MPP_CS_CTRL_DISABLE 0 +#define PM8XXX_MPP_CS_CTRL_ENABLE 1 +#define PM8XXX_MPP_CS_CTRL_MPP_HIGH_EN 2 +#define PM8XXX_MPP_CS_CTRL_MPP_LOW_EN 3 + +/* DTEST Current Sink: control */ +#define PM8XXX_MPP_DTEST_CS_CTRL_EN1 0 +#define PM8XXX_MPP_DTEST_CS_CTRL_EN2 1 +#define PM8XXX_MPP_DTEST_CS_CTRL_EN3 2 +#define PM8XXX_MPP_DTEST_CS_CTRL_EN4 3 + +/* DTEST Digital Output: control */ +#define PM8XXX_MPP_DTEST_DBUS1 0 +#define PM8XXX_MPP_DTEST_DBUS2 1 +#define PM8XXX_MPP_DTEST_DBUS3 2 +#define PM8XXX_MPP_DTEST_DBUS4 3 + +/* custom pinconf parameters */ +#define PM8XXX_CONFIG_AMUX (PIN_CONFIG_END + 1) +#define PM8XXX_CONFIG_DTEST_SELECTOR (PIN_CONFIG_END + 2) +#define PM8XXX_CONFIG_ALEVEL (PIN_CONFIG_END + 3) +#define PM8XXX_CONFIG_PAIRED (PIN_CONFIG_END + 4) + +/** + * struct pm8xxx_pin_data - dynamic configuration for a pin + * @reg: address of the control register + * @irq: IRQ from the PMIC interrupt controller + * @mode: operating mode for the pin (digital, analog or current sink) + * @input: pin is input + * @output: pin is output + * @high_z: pin is floating + * @paired: mpp operates in paired mode + * @output_value: logical output value of the mpp + * @power_source: selected power source + * @dtest: DTEST route selector + * @amux: input muxing in analog mode + * @aout_level: selector of the output in analog mode + * @drive_strength: drive strength of the current sink + * @pullup: pull up value, when in digital bidirectional mode + */ +struct pm8xxx_pin_data { + unsigned reg; + int irq; + + u8 mode; + + bool input; + bool output; + bool high_z; + bool paired; + bool output_value; + + u8 power_source; + u8 dtest; + u8 amux; + u8 aout_level; + u8 drive_strength; + unsigned pullup; +}; + +struct pm8xxx_mpp { + struct device *dev; + struct regmap *regmap; + struct pinctrl_dev *pctrl; + struct gpio_chip chip; + + struct pinctrl_desc desc; + unsigned npins; +}; + +static const struct pinconf_generic_params pm8xxx_mpp_bindings[] = { + {"qcom,amux-route", PM8XXX_CONFIG_AMUX, 0}, + {"qcom,analog-level", PM8XXX_CONFIG_ALEVEL, 0}, + {"qcom,dtest", PM8XXX_CONFIG_DTEST_SELECTOR, 0}, + {"qcom,paired", PM8XXX_CONFIG_PAIRED, 0}, +}; + +#ifdef CONFIG_DEBUG_FS +static const struct pin_config_item pm8xxx_conf_items[] = { + PCONFDUMP(PM8XXX_CONFIG_AMUX, "analog mux", NULL, true), + PCONFDUMP(PM8XXX_CONFIG_ALEVEL, "analog level", NULL, true), + PCONFDUMP(PM8XXX_CONFIG_DTEST_SELECTOR, "dtest", NULL, true), + PCONFDUMP(PM8XXX_CONFIG_PAIRED, "paired", NULL, false), +}; +#endif + +#define PM8XXX_MAX_MPPS 12 +static const char * const pm8xxx_groups[PM8XXX_MAX_MPPS] = { + "mpp1", "mpp2", "mpp3", "mpp4", "mpp5", "mpp6", "mpp7", "mpp8", + "mpp9", "mpp10", "mpp11", "mpp12", +}; + +#define PM8XXX_MPP_DIGITAL 0 +#define PM8XXX_MPP_ANALOG 1 +#define PM8XXX_MPP_SINK 2 + +static const char * const pm8xxx_mpp_functions[] = { + "digital", "analog", "sink", +}; + +static int pm8xxx_mpp_update(struct pm8xxx_mpp *pctrl, + struct pm8xxx_pin_data *pin) +{ + unsigned level; + unsigned ctrl; + unsigned type; + int ret; + u8 val; + + switch (pin->mode) { + case PM8XXX_MPP_DIGITAL: + if (pin->dtest) { + type = PM8XXX_MPP_TYPE_DTEST_OUTPUT; + ctrl = pin->dtest - 1; + } else if (pin->input && pin->output) { + type = PM8XXX_MPP_TYPE_D_BI_DIR; + if (pin->high_z) + ctrl = PM8XXX_MPP_BI_PULLUP_OPEN; + else if (pin->pullup == 600) + ctrl = PM8XXX_MPP_BI_PULLUP_1KOHM; + else if (pin->pullup == 10000) + ctrl = PM8XXX_MPP_BI_PULLUP_10KOHM; + else + ctrl = PM8XXX_MPP_BI_PULLUP_30KOHM; + } else if (pin->input) { + type = PM8XXX_MPP_TYPE_D_INPUT; + if (pin->dtest) + ctrl = pin->dtest; + else + ctrl = PM8XXX_MPP_DIN_TO_INT; + } else { + type = PM8XXX_MPP_TYPE_D_OUTPUT; + ctrl = !!pin->output_value; + if (pin->paired) + ctrl |= BIT(1); + } + + level = pin->power_source; + break; + case PM8XXX_MPP_ANALOG: + if (pin->output) { + type = PM8XXX_MPP_TYPE_A_OUTPUT; + level = pin->aout_level; + ctrl = pin->output_value; + if (pin->paired) + ctrl |= BIT(1); + } else { + type = PM8XXX_MPP_TYPE_A_INPUT; + level = pin->amux; + ctrl = 0; + } + break; + case PM8XXX_MPP_SINK: + level = (pin->drive_strength / 5) - 1; + if (pin->dtest) { + type = PM8XXX_MPP_TYPE_DTEST_SINK; + ctrl = pin->dtest - 1; + } else { + type = PM8XXX_MPP_TYPE_SINK; + ctrl = pin->output_value; + if (pin->paired) + ctrl |= BIT(1); + } + break; + default: + return -EINVAL; + } + + val = type << 5 | level << 2 | ctrl; + ret = regmap_write(pctrl->regmap, pin->reg, val); + if (ret) + dev_err(pctrl->dev, "failed to write register\n"); + + return ret; +} + +static int pm8xxx_get_groups_count(struct pinctrl_dev *pctldev) +{ + struct pm8xxx_mpp *pctrl = pinctrl_dev_get_drvdata(pctldev); + + return pctrl->npins; +} + +static const char *pm8xxx_get_group_name(struct pinctrl_dev *pctldev, + unsigned group) +{ + return pm8xxx_groups[group]; +} + + +static int pm8xxx_get_group_pins(struct pinctrl_dev *pctldev, + unsigned group, + const unsigned **pins, + unsigned *num_pins) +{ + struct pm8xxx_mpp *pctrl = pinctrl_dev_get_drvdata(pctldev); + + *pins = &pctrl->desc.pins[group].number; + *num_pins = 1; + + return 0; +} + +static const struct pinctrl_ops pm8xxx_pinctrl_ops = { + .get_groups_count = pm8xxx_get_groups_count, + .get_group_name = pm8xxx_get_group_name, + .get_group_pins = pm8xxx_get_group_pins, + .dt_node_to_map = pinconf_generic_dt_node_to_map_group, + .dt_free_map = pinctrl_utils_dt_free_map, +}; + +static int pm8xxx_get_functions_count(struct pinctrl_dev *pctldev) +{ + return ARRAY_SIZE(pm8xxx_mpp_functions); +} + +static const char *pm8xxx_get_function_name(struct pinctrl_dev *pctldev, + unsigned function) +{ + return pm8xxx_mpp_functions[function]; +} + +static int pm8xxx_get_function_groups(struct pinctrl_dev *pctldev, + unsigned function, + const char * const **groups, + unsigned * const num_groups) +{ + struct pm8xxx_mpp *pctrl = pinctrl_dev_get_drvdata(pctldev); + + *groups = pm8xxx_groups; + *num_groups = pctrl->npins; + return 0; +} + +static int pm8xxx_pinmux_set_mux(struct pinctrl_dev *pctldev, + unsigned function, + unsigned group) +{ + struct pm8xxx_mpp *pctrl = pinctrl_dev_get_drvdata(pctldev); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[group].drv_data; + + pin->mode = function; + pm8xxx_mpp_update(pctrl, pin); + + return 0; +} + +static const struct pinmux_ops pm8xxx_pinmux_ops = { + .get_functions_count = pm8xxx_get_functions_count, + .get_function_name = pm8xxx_get_function_name, + .get_function_groups = pm8xxx_get_function_groups, + .set_mux = pm8xxx_pinmux_set_mux, +}; + +static int pm8xxx_pin_config_get(struct pinctrl_dev *pctldev, + unsigned int offset, + unsigned long *config) +{ + struct pm8xxx_mpp *pctrl = pinctrl_dev_get_drvdata(pctldev); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + unsigned param = pinconf_to_config_param(*config); + unsigned arg; + + switch (param) { + case PIN_CONFIG_BIAS_PULL_UP: + arg = pin->pullup; + break; + case PIN_CONFIG_BIAS_HIGH_IMPEDANCE: + arg = pin->high_z; + break; + case PIN_CONFIG_INPUT_ENABLE: + arg = pin->input; + break; + case PIN_CONFIG_OUTPUT: + arg = pin->output_value; + break; + case PIN_CONFIG_POWER_SOURCE: + arg = pin->power_source; + break; + case PIN_CONFIG_DRIVE_STRENGTH: + arg = pin->drive_strength; + break; + case PM8XXX_CONFIG_DTEST_SELECTOR: + arg = pin->dtest; + break; + case PM8XXX_CONFIG_AMUX: + arg = pin->amux; + break; + case PM8XXX_CONFIG_ALEVEL: + arg = pin->aout_level; + break; + case PM8XXX_CONFIG_PAIRED: + arg = pin->paired; + break; + default: + return -EINVAL; + } + + *config = pinconf_to_config_packed(param, arg); + + return 0; +} + +static int pm8xxx_pin_config_set(struct pinctrl_dev *pctldev, + unsigned int offset, + unsigned long *configs, + unsigned num_configs) +{ + struct pm8xxx_mpp *pctrl = pinctrl_dev_get_drvdata(pctldev); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + unsigned param; + unsigned arg; + unsigned i; + + for (i = 0; i < num_configs; i++) { + param = pinconf_to_config_param(configs[i]); + arg = pinconf_to_config_argument(configs[i]); + + switch (param) { + case PIN_CONFIG_BIAS_PULL_UP: + pin->pullup = arg; + break; + case PIN_CONFIG_BIAS_HIGH_IMPEDANCE: + pin->high_z = true; + break; + case PIN_CONFIG_INPUT_ENABLE: + pin->input = true; + break; + case PIN_CONFIG_OUTPUT: + pin->output = true; + pin->output_value = !!arg; + break; + case PIN_CONFIG_POWER_SOURCE: + pin->power_source = arg; + break; + case PIN_CONFIG_DRIVE_STRENGTH: + pin->drive_strength = arg; + break; + case PM8XXX_CONFIG_DTEST_SELECTOR: + pin->dtest = arg; + break; + case PM8XXX_CONFIG_AMUX: + pin->amux = arg; + break; + case PM8XXX_CONFIG_ALEVEL: + pin->aout_level = arg; + break; + case PM8XXX_CONFIG_PAIRED: + pin->paired = !!arg; + break; + default: + dev_err(pctrl->dev, + "unsupported config parameter: %x\n", + param); + return -EINVAL; + } + } + + pm8xxx_mpp_update(pctrl, pin); + + return 0; +} + +static const struct pinconf_ops pm8xxx_pinconf_ops = { + .is_generic = true, + .pin_config_group_get = pm8xxx_pin_config_get, + .pin_config_group_set = pm8xxx_pin_config_set, +}; + +static struct pinctrl_desc pm8xxx_pinctrl_desc = { + .name = "pm8xxx_mpp", + .pctlops = &pm8xxx_pinctrl_ops, + .pmxops = &pm8xxx_pinmux_ops, + .confops = &pm8xxx_pinconf_ops, + .owner = THIS_MODULE, +}; + +static int pm8xxx_mpp_direction_input(struct gpio_chip *chip, + unsigned offset) +{ + struct pm8xxx_mpp *pctrl = container_of(chip, struct pm8xxx_mpp, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + + switch (pin->mode) { + case PM8XXX_MPP_DIGITAL: + pin->input = true; + break; + case PM8XXX_MPP_ANALOG: + pin->input = true; + pin->output = true; + break; + case PM8XXX_MPP_SINK: + return -EINVAL; + } + + pm8xxx_mpp_update(pctrl, pin); + + return 0; +} + +static int pm8xxx_mpp_direction_output(struct gpio_chip *chip, + unsigned offset, + int value) +{ + struct pm8xxx_mpp *pctrl = container_of(chip, struct pm8xxx_mpp, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + + switch (pin->mode) { + case PM8XXX_MPP_DIGITAL: + pin->output = true; + break; + case PM8XXX_MPP_ANALOG: + pin->input = false; + pin->output = true; + break; + case PM8XXX_MPP_SINK: + pin->input = false; + pin->output = true; + break; + } + + pm8xxx_mpp_update(pctrl, pin); + + return 0; +} + +static int pm8xxx_mpp_get(struct gpio_chip *chip, unsigned offset) +{ + struct pm8xxx_mpp *pctrl = container_of(chip, struct pm8xxx_mpp, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + bool state; + int ret; + + if (!pin->input) + return pin->output_value; + + ret = irq_get_irqchip_state(pin->irq, IRQCHIP_STATE_LINE_LEVEL, &state); + if (!ret) + ret = !!state; + + return ret; +} + +static void pm8xxx_mpp_set(struct gpio_chip *chip, unsigned offset, int value) +{ + struct pm8xxx_mpp *pctrl = container_of(chip, struct pm8xxx_mpp, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + + pin->output_value = !!value; + + pm8xxx_mpp_update(pctrl, pin); +} + +static int pm8xxx_mpp_of_xlate(struct gpio_chip *chip, + const struct of_phandle_args *gpio_desc, + u32 *flags) +{ + if (chip->of_gpio_n_cells < 2) + return -EINVAL; + + if (flags) + *flags = gpio_desc->args[1]; + + return gpio_desc->args[0] - 1; +} + + +static int pm8xxx_mpp_to_irq(struct gpio_chip *chip, unsigned offset) +{ + struct pm8xxx_mpp *pctrl = container_of(chip, struct pm8xxx_mpp, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + + return pin->irq; +} + +#ifdef CONFIG_DEBUG_FS +#include + +static void pm8xxx_mpp_dbg_show_one(struct seq_file *s, + struct pinctrl_dev *pctldev, + struct gpio_chip *chip, + unsigned offset, + unsigned gpio) +{ + struct pm8xxx_mpp *pctrl = container_of(chip, struct pm8xxx_mpp, chip); + struct pm8xxx_pin_data *pin = pctrl->desc.pins[offset].drv_data; + + static const char * const aout_lvls[] = { + "1v25", "1v25_2", "0v625", "0v3125", "mpp", "abus1", "abus2", + "abus3" + }; + + static const char * const amuxs[] = { + "amux5", "amux6", "amux7", "amux8", "amux9", "abus1", "abus2", + "abus3", + }; + + seq_printf(s, " mpp%-2d:", offset + 1); + + switch (pin->mode) { + case PM8XXX_MPP_DIGITAL: + seq_puts(s, " digital "); + if (pin->dtest) { + seq_printf(s, "dtest%d\n", pin->dtest); + } else if (pin->input && pin->output) { + if (pin->high_z) + seq_puts(s, "bi-dir high-z"); + else + seq_printf(s, "bi-dir %dOhm", pin->pullup); + } else if (pin->input) { + if (pin->dtest) + seq_printf(s, "in dtest%d", pin->dtest); + else + seq_puts(s, "in gpio"); + } else if (pin->output) { + seq_puts(s, "out "); + + if (!pin->paired) { + seq_puts(s, pin->output_value ? + "high" : "low"); + } else { + seq_puts(s, pin->output_value ? + "inverted" : "follow"); + } + } + break; + case PM8XXX_MPP_ANALOG: + seq_puts(s, " analog "); + if (pin->output) { + seq_printf(s, "out %s ", aout_lvls[pin->aout_level]); + if (!pin->paired) { + seq_puts(s, pin->output_value ? + "high" : "low"); + } else { + seq_puts(s, pin->output_value ? + "inverted" : "follow"); + } + } else { + seq_printf(s, "input mux %s", amuxs[pin->amux]); + } + break; + case PM8XXX_MPP_SINK: + seq_printf(s, " sink %dmA ", pin->drive_strength); + if (pin->dtest) { + seq_printf(s, "dtest%d", pin->dtest); + } else { + if (!pin->paired) { + seq_puts(s, pin->output_value ? + "high" : "low"); + } else { + seq_puts(s, pin->output_value ? + "inverted" : "follow"); + } + } + break; + } + +} + +static void pm8xxx_mpp_dbg_show(struct seq_file *s, struct gpio_chip *chip) +{ + unsigned gpio = chip->base; + unsigned i; + + for (i = 0; i < chip->ngpio; i++, gpio++) { + pm8xxx_mpp_dbg_show_one(s, NULL, chip, i, gpio); + seq_puts(s, "\n"); + } +} + +#else +#define msm_mpp_dbg_show NULL +#endif + +static struct gpio_chip pm8xxx_mpp_template = { + .direction_input = pm8xxx_mpp_direction_input, + .direction_output = pm8xxx_mpp_direction_output, + .get = pm8xxx_mpp_get, + .set = pm8xxx_mpp_set, + .of_xlate = pm8xxx_mpp_of_xlate, + .to_irq = pm8xxx_mpp_to_irq, + .dbg_show = pm8xxx_mpp_dbg_show, + .owner = THIS_MODULE, +}; + +static int pm8xxx_pin_populate(struct pm8xxx_mpp *pctrl, + struct pm8xxx_pin_data *pin) +{ + unsigned int val; + unsigned level; + unsigned ctrl; + unsigned type; + int ret; + + ret = regmap_read(pctrl->regmap, pin->reg, &val); + if (ret) { + dev_err(pctrl->dev, "failed to read register\n"); + return ret; + } + + type = (val >> 5) & 7; + level = (val >> 2) & 7; + ctrl = (val) & 3; + + switch (type) { + case PM8XXX_MPP_TYPE_D_INPUT: + pin->mode = PM8XXX_MPP_DIGITAL; + pin->input = true; + pin->power_source = level; + pin->dtest = ctrl; + break; + case PM8XXX_MPP_TYPE_D_OUTPUT: + pin->mode = PM8XXX_MPP_DIGITAL; + pin->output = true; + pin->power_source = level; + pin->output_value = !!(ctrl & BIT(0)); + pin->paired = !!(ctrl & BIT(1)); + break; + case PM8XXX_MPP_TYPE_D_BI_DIR: + pin->mode = PM8XXX_MPP_DIGITAL; + pin->input = true; + pin->output = true; + pin->power_source = level; + switch (ctrl) { + case PM8XXX_MPP_BI_PULLUP_1KOHM: + pin->pullup = 600; + break; + case PM8XXX_MPP_BI_PULLUP_OPEN: + pin->high_z = true; + break; + case PM8XXX_MPP_BI_PULLUP_10KOHM: + pin->pullup = 10000; + break; + case PM8XXX_MPP_BI_PULLUP_30KOHM: + pin->pullup = 30000; + break; + } + break; + case PM8XXX_MPP_TYPE_A_INPUT: + pin->mode = PM8XXX_MPP_ANALOG; + pin->input = true; + pin->amux = level; + break; + case PM8XXX_MPP_TYPE_A_OUTPUT: + pin->mode = PM8XXX_MPP_ANALOG; + pin->output = true; + pin->aout_level = level; + pin->output_value = !!(ctrl & BIT(0)); + pin->paired = !!(ctrl & BIT(1)); + break; + case PM8XXX_MPP_TYPE_SINK: + pin->mode = PM8XXX_MPP_SINK; + pin->drive_strength = 5 * (level + 1); + pin->output_value = !!(ctrl & BIT(0)); + pin->paired = !!(ctrl & BIT(1)); + break; + case PM8XXX_MPP_TYPE_DTEST_SINK: + pin->mode = PM8XXX_MPP_SINK; + pin->dtest = ctrl + 1; + pin->drive_strength = 5 * (level + 1); + break; + case PM8XXX_MPP_TYPE_DTEST_OUTPUT: + pin->mode = PM8XXX_MPP_DIGITAL; + pin->power_source = level; + if (ctrl >= 1) + pin->dtest = ctrl; + break; + } + + return 0; +} + +static const struct of_device_id pm8xxx_mpp_of_match[] = { + { .compatible = "qcom,pm8018-mpp", .data = (void *)6 }, + { .compatible = "qcom,pm8038-mpp", .data = (void *)6 }, + { .compatible = "qcom,pm8917-mpp", .data = (void *)10 }, + { .compatible = "qcom,pm8821-mpp", .data = (void *)4 }, + { .compatible = "qcom,pm8921-mpp", .data = (void *)12 }, + { }, +}; +MODULE_DEVICE_TABLE(of, pm8xxx_mpp_of_match); + +static int pm8xxx_mpp_probe(struct platform_device *pdev) +{ + struct pm8xxx_pin_data *pin_data; + struct pinctrl_pin_desc *pins; + struct pm8xxx_mpp *pctrl; + int ret; + int i; + + pctrl = devm_kzalloc(&pdev->dev, sizeof(*pctrl), GFP_KERNEL); + if (!pctrl) + return -ENOMEM; + + pctrl->dev = &pdev->dev; + pctrl->npins = (unsigned)of_device_get_match_data(&pdev->dev); + + pctrl->regmap = dev_get_regmap(pdev->dev.parent, NULL); + if (!pctrl->regmap) { + dev_err(&pdev->dev, "parent regmap unavailable\n"); + return -ENXIO; + } + + pctrl->desc = pm8xxx_pinctrl_desc; + pctrl->desc.npins = pctrl->npins; + + pins = devm_kcalloc(&pdev->dev, + pctrl->desc.npins, + sizeof(struct pinctrl_pin_desc), + GFP_KERNEL); + if (!pins) + return -ENOMEM; + + pin_data = devm_kcalloc(&pdev->dev, + pctrl->desc.npins, + sizeof(struct pm8xxx_pin_data), + GFP_KERNEL); + if (!pin_data) + return -ENOMEM; + + for (i = 0; i < pctrl->desc.npins; i++) { + pin_data[i].reg = SSBI_REG_ADDR_MPP(i); + pin_data[i].irq = platform_get_irq(pdev, i); + if (pin_data[i].irq < 0) { + dev_err(&pdev->dev, + "missing interrupts for pin %d\n", i); + return pin_data[i].irq; + } + + ret = pm8xxx_pin_populate(pctrl, &pin_data[i]); + if (ret) + return ret; + + pins[i].number = i; + pins[i].name = pm8xxx_groups[i]; + pins[i].drv_data = &pin_data[i]; + } + pctrl->desc.pins = pins; + + pctrl->desc.num_custom_params = ARRAY_SIZE(pm8xxx_mpp_bindings); + pctrl->desc.custom_params = pm8xxx_mpp_bindings; +#ifdef CONFIG_DEBUG_FS + pctrl->desc.custom_conf_items = pm8xxx_conf_items; +#endif + + pctrl->pctrl = pinctrl_register(&pctrl->desc, &pdev->dev, pctrl); + if (!pctrl->pctrl) { + dev_err(&pdev->dev, "couldn't register pm8xxx mpp driver\n"); + return -ENODEV; + } + + pctrl->chip = pm8xxx_mpp_template; + pctrl->chip.base = -1; + pctrl->chip.dev = &pdev->dev; + pctrl->chip.of_node = pdev->dev.of_node; + pctrl->chip.of_gpio_n_cells = 2; + pctrl->chip.label = dev_name(pctrl->dev); + pctrl->chip.ngpio = pctrl->npins; + ret = gpiochip_add(&pctrl->chip); + if (ret) { + dev_err(&pdev->dev, "failed register gpiochip\n"); + goto unregister_pinctrl; + } + + ret = gpiochip_add_pin_range(&pctrl->chip, + dev_name(pctrl->dev), + 0, 0, pctrl->chip.ngpio); + if (ret) { + dev_err(pctrl->dev, "failed to add pin range\n"); + goto unregister_gpiochip; + } + + platform_set_drvdata(pdev, pctrl); + + dev_dbg(&pdev->dev, "Qualcomm pm8xxx mpp driver probed\n"); + + return 0; + +unregister_gpiochip: + gpiochip_remove(&pctrl->chip); + +unregister_pinctrl: + pinctrl_unregister(pctrl->pctrl); + + return ret; +} + +static int pm8xxx_mpp_remove(struct platform_device *pdev) +{ + struct pm8xxx_mpp *pctrl = platform_get_drvdata(pdev); + + gpiochip_remove(&pctrl->chip); + + pinctrl_unregister(pctrl->pctrl); + + return 0; +} + +static struct platform_driver pm8xxx_mpp_driver = { + .driver = { + .name = "qcom-ssbi-mpp", + .of_match_table = pm8xxx_mpp_of_match, + }, + .probe = pm8xxx_mpp_probe, + .remove = pm8xxx_mpp_remove, +}; + +module_platform_driver(pm8xxx_mpp_driver); + +MODULE_AUTHOR("Bjorn Andersson "); +MODULE_DESCRIPTION("Qualcomm PM8xxx MPP driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pinctrl/samsung/pinctrl-exynos.c b/drivers/pinctrl/samsung/pinctrl-exynos.c index b18dabba03a480..5f45caaef46d68 100644 --- a/drivers/pinctrl/samsung/pinctrl-exynos.c +++ b/drivers/pinctrl/samsung/pinctrl-exynos.c @@ -148,9 +148,9 @@ static int exynos_irq_set_type(struct irq_data *irqd, unsigned int type) } if (type & IRQ_TYPE_EDGE_BOTH) - __irq_set_handler_locked(irqd->irq, handle_edge_irq); + irq_set_handler_locked(irqd, handle_edge_irq); else - __irq_set_handler_locked(irqd->irq, handle_level_irq); + irq_set_handler_locked(irqd, handle_level_irq); con = readl(d->virt_base + reg_con); con &= ~(EXYNOS_EINT_CON_MASK << shift); @@ -256,7 +256,6 @@ static int exynos_eint_irq_map(struct irq_domain *h, unsigned int virq, irq_set_chip_data(virq, b); irq_set_chip_and_handler(virq, &b->irq_chip->chip, handle_level_irq); - set_irq_flags(virq, IRQF_VALID); return 0; } @@ -422,9 +421,9 @@ static const struct of_device_id exynos_wkup_irq_ids[] = { /* interrupt handler for wakeup interrupts 0..15 */ static void exynos_irq_eint0_15(unsigned int irq, struct irq_desc *desc) { - struct exynos_weint_data *eintd = irq_get_handler_data(irq); + struct exynos_weint_data *eintd = irq_desc_get_handler_data(desc); struct samsung_pin_bank *bank = eintd->bank; - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); int eint_irq; chained_irq_enter(chip, desc); @@ -454,8 +453,8 @@ static inline void exynos_irq_demux_eint(unsigned long pend, /* interrupt handler for wakeup interrupt 16 */ static void exynos_irq_demux_eint16_31(unsigned int irq, struct irq_desc *desc) { - struct irq_chip *chip = irq_get_chip(irq); - struct exynos_muxed_weint_data *eintd = irq_get_handler_data(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); + struct exynos_muxed_weint_data *eintd = irq_desc_get_handler_data(desc); struct samsung_pinctrl_drv_data *d = eintd->banks[0]->drvdata; unsigned long pend; unsigned long mask; @@ -542,8 +541,9 @@ static int exynos_eint_wkup_init(struct samsung_pinctrl_drv_data *d) } weint_data[idx].irq = idx; weint_data[idx].bank = bank; - irq_set_handler_data(irq, &weint_data[idx]); - irq_set_chained_handler(irq, exynos_irq_eint0_15); + irq_set_chained_handler_and_data(irq, + exynos_irq_eint0_15, + &weint_data[idx]); } } diff --git a/drivers/pinctrl/samsung/pinctrl-exynos5440.c b/drivers/pinctrl/samsung/pinctrl-exynos5440.c index f5619fb504473a..9ce0b8619d4c6e 100644 --- a/drivers/pinctrl/samsung/pinctrl-exynos5440.c +++ b/drivers/pinctrl/samsung/pinctrl-exynos5440.c @@ -44,9 +44,7 @@ #define PIN_NAME_LENGTH 10 #define GROUP_SUFFIX "-grp" -#define GSUFFIX_LEN sizeof(GROUP_SUFFIX) #define FUNCTION_SUFFIX "-mux" -#define FSUFFIX_LEN sizeof(FUNCTION_SUFFIX) /* * pin configuration type and its value are packed together into a 16-bits. @@ -205,22 +203,17 @@ static int exynos5440_dt_node_to_map(struct pinctrl_dev *pctldev, /* Allocate memory for pin-map entries */ map = kzalloc(sizeof(*map) * map_cnt, GFP_KERNEL); - if (!map) { - dev_err(dev, "could not alloc memory for pin-maps\n"); + if (!map) return -ENOMEM; - } *nmaps = 0; /* * Allocate memory for pin group name. The pin group name is derived * from the node name from which these map entries are be created. */ - gname = kzalloc(strlen(np->name) + GSUFFIX_LEN, GFP_KERNEL); - if (!gname) { - dev_err(dev, "failed to alloc memory for group name\n"); + gname = kasprintf(GFP_KERNEL, "%s%s", np->name, GROUP_SUFFIX); + if (!gname) goto free_map; - } - snprintf(gname, strlen(np->name) + 4, "%s%s", np->name, GROUP_SUFFIX); /* * don't have config options? then skip over to creating function @@ -231,10 +224,8 @@ static int exynos5440_dt_node_to_map(struct pinctrl_dev *pctldev, /* Allocate memory for config entries */ cfg = kzalloc(sizeof(*cfg) * cfg_cnt, GFP_KERNEL); - if (!cfg) { - dev_err(dev, "failed to alloc memory for configs\n"); + if (!cfg) goto free_gname; - } /* Prepare a list of config settings */ for (idx = 0, cfg_cnt = 0; idx < ARRAY_SIZE(pcfgs); idx++) { @@ -254,13 +245,10 @@ static int exynos5440_dt_node_to_map(struct pinctrl_dev *pctldev, skip_cfgs: /* create the function map entry */ if (of_find_property(np, "samsung,exynos5440-pin-function", NULL)) { - fname = kzalloc(strlen(np->name) + FSUFFIX_LEN, GFP_KERNEL); - if (!fname) { - dev_err(dev, "failed to alloc memory for func name\n"); + fname = kasprintf(GFP_KERNEL, + "%s%s", np->name, FUNCTION_SUFFIX); + if (!fname) goto free_cfg; - } - snprintf(fname, strlen(np->name) + 4, "%s%s", np->name, - FUNCTION_SUFFIX); map[*nmaps].data.mux.group = gname; map[*nmaps].data.mux.function = fname; @@ -651,10 +639,8 @@ static int exynos5440_pinctrl_parse_dt_pins(struct platform_device *pdev, } *pin_list = devm_kzalloc(dev, *npins * sizeof(**pin_list), GFP_KERNEL); - if (!*pin_list) { - dev_err(dev, "failed to allocate memory for pin list\n"); + if (!*pin_list) return -ENOMEM; - } return of_property_read_u32_array(cfg_np, "samsung,exynos5440-pins", *pin_list, *npins); @@ -682,17 +668,15 @@ static int exynos5440_pinctrl_parse_dt(struct platform_device *pdev, return -EINVAL; groups = devm_kzalloc(dev, grp_cnt * sizeof(*groups), GFP_KERNEL); - if (!groups) { - dev_err(dev, "failed allocate memory for ping group list\n"); + if (!groups) return -EINVAL; - } + grp = groups; functions = devm_kzalloc(dev, grp_cnt * sizeof(*functions), GFP_KERNEL); - if (!functions) { - dev_err(dev, "failed to allocate memory for function list\n"); + if (!functions) return -EINVAL; - } + func = functions; /* @@ -710,14 +694,10 @@ static int exynos5440_pinctrl_parse_dt(struct platform_device *pdev, } /* derive pin group name from the node name */ - gname = devm_kzalloc(dev, strlen(cfg_np->name) + GSUFFIX_LEN, - GFP_KERNEL); - if (!gname) { - dev_err(dev, "failed to alloc memory for group name\n"); + gname = devm_kasprintf(dev, GFP_KERNEL, + "%s%s", cfg_np->name, GROUP_SUFFIX); + if (!gname) return -ENOMEM; - } - snprintf(gname, strlen(cfg_np->name) + 4, "%s%s", cfg_np->name, - GROUP_SUFFIX); grp->name = gname; grp->pins = pin_list; @@ -731,22 +711,15 @@ static int exynos5440_pinctrl_parse_dt(struct platform_device *pdev, continue; /* derive function name from the node name */ - fname = devm_kzalloc(dev, strlen(cfg_np->name) + FSUFFIX_LEN, - GFP_KERNEL); - if (!fname) { - dev_err(dev, "failed to alloc memory for func name\n"); + fname = devm_kasprintf(dev, GFP_KERNEL, + "%s%s", cfg_np->name, FUNCTION_SUFFIX); + if (!fname) return -ENOMEM; - } - snprintf(fname, strlen(cfg_np->name) + 4, "%s%s", cfg_np->name, - FUNCTION_SUFFIX); func->name = fname; func->groups = devm_kzalloc(dev, sizeof(char *), GFP_KERNEL); - if (!func->groups) { - dev_err(dev, "failed to alloc memory for group list " - "in pin function"); + if (!func->groups) return -ENOMEM; - } func->groups[0] = gname; func->num_groups = gname ? 1 : 0; func->function = function; @@ -774,10 +747,8 @@ static int exynos5440_pinctrl_register(struct platform_device *pdev, int pin, ret; ctrldesc = devm_kzalloc(dev, sizeof(*ctrldesc), GFP_KERNEL); - if (!ctrldesc) { - dev_err(dev, "could not allocate memory for pinctrl desc\n"); + if (!ctrldesc) return -ENOMEM; - } ctrldesc->name = "exynos5440-pinctrl"; ctrldesc->owner = THIS_MODULE; @@ -787,10 +758,8 @@ static int exynos5440_pinctrl_register(struct platform_device *pdev, pindesc = devm_kzalloc(&pdev->dev, sizeof(*pindesc) * EXYNOS5440_MAX_PINS, GFP_KERNEL); - if (!pindesc) { - dev_err(&pdev->dev, "mem alloc for pin descriptors failed\n"); + if (!pindesc) return -ENOMEM; - } ctrldesc->pins = pindesc; ctrldesc->npins = EXYNOS5440_MAX_PINS; @@ -804,10 +773,8 @@ static int exynos5440_pinctrl_register(struct platform_device *pdev, */ pin_names = devm_kzalloc(&pdev->dev, sizeof(char) * PIN_NAME_LENGTH * ctrldesc->npins, GFP_KERNEL); - if (!pin_names) { - dev_err(&pdev->dev, "mem alloc for pin names failed\n"); + if (!pin_names) return -ENOMEM; - } /* for each pin, set the name of the pin */ for (pin = 0; pin < ctrldesc->npins; pin++) { @@ -844,10 +811,8 @@ static int exynos5440_gpiolib_register(struct platform_device *pdev, int ret; gc = devm_kzalloc(&pdev->dev, sizeof(*gc), GFP_KERNEL); - if (!gc) { - dev_err(&pdev->dev, "mem alloc for gpio_chip failed\n"); + if (!gc) return -ENOMEM; - } priv->gc = gc; gc->base = 0; @@ -929,7 +894,6 @@ static int exynos5440_gpio_irq_map(struct irq_domain *h, unsigned int virq, irq_set_chip_data(virq, d); irq_set_chip_and_handler(virq, &exynos5440_gpio_irq_chip, handle_level_irq); - set_irq_flags(virq, IRQF_VALID); return 0; } @@ -949,10 +913,8 @@ static int exynos5440_gpio_irq_init(struct platform_device *pdev, intd = devm_kzalloc(dev, sizeof(*intd) * EXYNOS5440_MAX_GPIO_INT, GFP_KERNEL); - if (!intd) { - dev_err(dev, "failed to allocate memory for gpio intr data\n"); + if (!intd) return -ENOMEM; - } for (i = 0; i < EXYNOS5440_MAX_GPIO_INT; i++) { irq = irq_of_parse_and_map(dev->of_node, i); @@ -995,10 +957,8 @@ static int exynos5440_pinctrl_probe(struct platform_device *pdev) } priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL); - if (!priv) { - dev_err(dev, "could not allocate memory for private data\n"); + if (!priv) return -ENOMEM; - } res = platform_get_resource(pdev, IORESOURCE_MEM, 0); priv->reg_base = devm_ioremap_resource(&pdev->dev, res); diff --git a/drivers/pinctrl/samsung/pinctrl-s3c24xx.c b/drivers/pinctrl/samsung/pinctrl-s3c24xx.c index 01b43dbfb795b0..019844d479bb5c 100644 --- a/drivers/pinctrl/samsung/pinctrl-s3c24xx.c +++ b/drivers/pinctrl/samsung/pinctrl-s3c24xx.c @@ -131,13 +131,13 @@ static int s3c24xx_eint_get_trigger(unsigned int type) } } -static void s3c24xx_eint_set_handler(unsigned int irq, unsigned int type) +static void s3c24xx_eint_set_handler(struct irq_data *d, unsigned int type) { /* Edge- and level-triggered interrupts need different handlers */ if (type & IRQ_TYPE_EDGE_BOTH) - __irq_set_handler_locked(irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); else - __irq_set_handler_locked(irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); } static void s3c24xx_eint_set_function(struct samsung_pinctrl_drv_data *d, @@ -181,7 +181,7 @@ static int s3c24xx_eint_type(struct irq_data *data, unsigned int type) return -EINVAL; } - s3c24xx_eint_set_handler(data->irq, type); + s3c24xx_eint_set_handler(data, type); /* Set up interrupt trigger */ reg = d->virt_base + EINT_REG(index); @@ -243,7 +243,7 @@ static struct irq_chip s3c2410_eint0_3_chip = { static void s3c2410_demux_eint0_3(unsigned int irq, struct irq_desc *desc) { struct irq_data *data = irq_desc_get_irq_data(desc); - struct s3c24xx_eint_data *eint_data = irq_get_handler_data(irq); + struct s3c24xx_eint_data *eint_data = irq_desc_get_handler_data(desc); unsigned int virq; /* the first 4 eints have a simple 1 to 1 mapping */ @@ -297,9 +297,9 @@ static struct irq_chip s3c2412_eint0_3_chip = { static void s3c2412_demux_eint0_3(unsigned int irq, struct irq_desc *desc) { - struct irq_chip *chip = irq_get_chip(irq); + struct s3c24xx_eint_data *eint_data = irq_desc_get_handler_data(desc); struct irq_data *data = irq_desc_get_irq_data(desc); - struct s3c24xx_eint_data *eint_data = irq_get_handler_data(irq); + struct irq_chip *chip = irq_data_get_irq_chip(data); unsigned int virq; chained_irq_enter(chip, desc); @@ -357,11 +357,11 @@ static struct irq_chip s3c24xx_eint_chip = { .irq_set_type = s3c24xx_eint_type, }; -static inline void s3c24xx_demux_eint(unsigned int irq, struct irq_desc *desc, +static inline void s3c24xx_demux_eint(struct irq_desc *desc, u32 offset, u32 range) { - struct irq_chip *chip = irq_get_chip(irq); - struct s3c24xx_eint_data *data = irq_get_handler_data(irq); + struct s3c24xx_eint_data *data = irq_desc_get_handler_data(desc); + struct irq_chip *chip = irq_desc_get_irq_chip(desc); struct samsung_pinctrl_drv_data *d = data->drvdata; unsigned int pend, mask; @@ -374,7 +374,7 @@ static inline void s3c24xx_demux_eint(unsigned int irq, struct irq_desc *desc, pend &= range; while (pend) { - unsigned int virq; + unsigned int virq, irq; irq = __ffs(pend); pend &= ~(1 << irq); @@ -390,12 +390,12 @@ static inline void s3c24xx_demux_eint(unsigned int irq, struct irq_desc *desc, static void s3c24xx_demux_eint4_7(unsigned int irq, struct irq_desc *desc) { - s3c24xx_demux_eint(irq, desc, 0, 0xf0); + s3c24xx_demux_eint(desc, 0, 0xf0); } static void s3c24xx_demux_eint8_23(unsigned int irq, struct irq_desc *desc) { - s3c24xx_demux_eint(irq, desc, 8, 0xffff00); + s3c24xx_demux_eint(desc, 8, 0xffff00); } static irq_flow_handler_t s3c2410_eint_handlers[NUM_EINT_IRQ] = { @@ -437,7 +437,6 @@ static int s3c24xx_gpf_irq_map(struct irq_domain *h, unsigned int virq, handle_edge_irq); } irq_set_chip_data(virq, bank); - set_irq_flags(virq, IRQF_VALID); return 0; } @@ -457,7 +456,6 @@ static int s3c24xx_gpg_irq_map(struct irq_domain *h, unsigned int virq, irq_set_chip_and_handler(virq, &s3c24xx_eint_chip, handle_edge_irq); irq_set_chip_data(virq, bank); - set_irq_flags(virq, IRQF_VALID); return 0; } diff --git a/drivers/pinctrl/samsung/pinctrl-s3c64xx.c b/drivers/pinctrl/samsung/pinctrl-s3c64xx.c index ec8cc3b476213c..f5ea40a69711c0 100644 --- a/drivers/pinctrl/samsung/pinctrl-s3c64xx.c +++ b/drivers/pinctrl/samsung/pinctrl-s3c64xx.c @@ -260,13 +260,13 @@ static int s3c64xx_irq_get_trigger(unsigned int type) return trigger; } -static void s3c64xx_irq_set_handler(unsigned int irq, unsigned int type) +static void s3c64xx_irq_set_handler(struct irq_data *d, unsigned int type) { /* Edge- and level-triggered interrupts need different handlers */ if (type & IRQ_TYPE_EDGE_BOTH) - __irq_set_handler_locked(irq, handle_edge_irq); + irq_set_handler_locked(d, handle_edge_irq); else - __irq_set_handler_locked(irq, handle_level_irq); + irq_set_handler_locked(d, handle_level_irq); } static void s3c64xx_irq_set_function(struct samsung_pinctrl_drv_data *d, @@ -356,7 +356,7 @@ static int s3c64xx_gpio_irq_set_type(struct irq_data *irqd, unsigned int type) return -EINVAL; } - s3c64xx_irq_set_handler(irqd->irq, type); + s3c64xx_irq_set_handler(irqd, type); /* Set up interrupt trigger */ reg = d->virt_base + EINTCON_REG(bank->eint_offset); @@ -395,7 +395,6 @@ static int s3c64xx_gpio_irq_map(struct irq_domain *h, unsigned int virq, irq_set_chip_and_handler(virq, &s3c64xx_gpio_irq_chip, handle_level_irq); irq_set_chip_data(virq, bank); - set_irq_flags(virq, IRQF_VALID); return 0; } @@ -410,8 +409,8 @@ static const struct irq_domain_ops s3c64xx_gpio_irqd_ops = { static void s3c64xx_eint_gpio_irq(unsigned int irq, struct irq_desc *desc) { - struct irq_chip *chip = irq_get_chip(irq); - struct s3c64xx_eint_gpio_data *data = irq_get_handler_data(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); + struct s3c64xx_eint_gpio_data *data = irq_desc_get_handler_data(desc); struct samsung_pinctrl_drv_data *drvdata = data->drvdata; chained_irq_enter(chip, desc); @@ -567,7 +566,7 @@ static int s3c64xx_eint0_irq_set_type(struct irq_data *irqd, unsigned int type) return -EINVAL; } - s3c64xx_irq_set_handler(irqd->irq, type); + s3c64xx_irq_set_handler(irqd, type); /* Set up interrupt trigger */ reg = d->virt_base + EINT0CON0_REG; @@ -599,11 +598,10 @@ static struct irq_chip s3c64xx_eint0_irq_chip = { .irq_set_type = s3c64xx_eint0_irq_set_type, }; -static inline void s3c64xx_irq_demux_eint(unsigned int irq, - struct irq_desc *desc, u32 range) +static inline void s3c64xx_irq_demux_eint(struct irq_desc *desc, u32 range) { - struct irq_chip *chip = irq_get_chip(irq); - struct s3c64xx_eint0_data *data = irq_get_handler_data(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); + struct s3c64xx_eint0_data *data = irq_desc_get_handler_data(desc); struct samsung_pinctrl_drv_data *drvdata = data->drvdata; unsigned int pend, mask; @@ -616,11 +614,10 @@ static inline void s3c64xx_irq_demux_eint(unsigned int irq, pend &= range; while (pend) { - unsigned int virq; + unsigned int virq, irq; irq = fls(pend) - 1; pend &= ~(1 << irq); - virq = irq_linear_revmap(data->domains[irq], data->pins[irq]); /* * Something must be really wrong if an unmapped EINT @@ -636,22 +633,22 @@ static inline void s3c64xx_irq_demux_eint(unsigned int irq, static void s3c64xx_demux_eint0_3(unsigned int irq, struct irq_desc *desc) { - s3c64xx_irq_demux_eint(irq, desc, 0xf); + s3c64xx_irq_demux_eint(desc, 0xf); } static void s3c64xx_demux_eint4_11(unsigned int irq, struct irq_desc *desc) { - s3c64xx_irq_demux_eint(irq, desc, 0xff0); + s3c64xx_irq_demux_eint(desc, 0xff0); } static void s3c64xx_demux_eint12_19(unsigned int irq, struct irq_desc *desc) { - s3c64xx_irq_demux_eint(irq, desc, 0xff000); + s3c64xx_irq_demux_eint(desc, 0xff000); } static void s3c64xx_demux_eint20_27(unsigned int irq, struct irq_desc *desc) { - s3c64xx_irq_demux_eint(irq, desc, 0xff00000); + s3c64xx_irq_demux_eint(desc, 0xff00000); } static irq_flow_handler_t s3c64xx_eint0_handlers[NUM_EINT0_IRQ] = { @@ -673,7 +670,6 @@ static int s3c64xx_eint0_irq_map(struct irq_domain *h, unsigned int virq, irq_set_chip_and_handler(virq, &s3c64xx_eint0_irq_chip, handle_level_irq); irq_set_chip_data(virq, ddata); - set_irq_flags(virq, IRQF_VALID); return 0; } diff --git a/drivers/pinctrl/sh-pfc/core.c b/drivers/pinctrl/sh-pfc/core.c index 865d235612c520..fb9c4480523431 100644 --- a/drivers/pinctrl/sh-pfc/core.c +++ b/drivers/pinctrl/sh-pfc/core.c @@ -29,24 +29,25 @@ static int sh_pfc_map_resources(struct sh_pfc *pfc, struct platform_device *pdev) { - unsigned int num_windows = 0; - unsigned int num_irqs = 0; + unsigned int num_windows, num_irqs; struct sh_pfc_window *windows; unsigned int *irqs = NULL; struct resource *res; unsigned int i; + int irq; /* Count the MEM and IRQ resources. */ - for (i = 0; i < pdev->num_resources; ++i) { - switch (resource_type(&pdev->resource[i])) { - case IORESOURCE_MEM: - num_windows++; + for (num_windows = 0;; num_windows++) { + res = platform_get_resource(pdev, IORESOURCE_MEM, num_windows); + if (!res) break; - - case IORESOURCE_IRQ: - num_irqs++; + } + for (num_irqs = 0;; num_irqs++) { + irq = platform_get_irq(pdev, num_irqs); + if (irq == -EPROBE_DEFER) + return irq; + if (irq < 0) break; - } } if (num_windows == 0) @@ -72,22 +73,17 @@ static int sh_pfc_map_resources(struct sh_pfc *pfc, } /* Fill them. */ - for (i = 0, res = pdev->resource; i < pdev->num_resources; i++, res++) { - switch (resource_type(res)) { - case IORESOURCE_MEM: - windows->phys = res->start; - windows->size = resource_size(res); - windows->virt = devm_ioremap_resource(pfc->dev, res); - if (IS_ERR(windows->virt)) - return -ENOMEM; - windows++; - break; - - case IORESOURCE_IRQ: - *irqs++ = res->start; - break; - } + for (i = 0; i < num_windows; i++) { + res = platform_get_resource(pdev, IORESOURCE_MEM, i); + windows->phys = res->start; + windows->size = resource_size(res); + windows->virt = devm_ioremap_resource(pfc->dev, res); + if (IS_ERR(windows->virt)) + return -ENOMEM; + windows++; } + for (i = 0; i < num_irqs; i++) + *irqs++ = platform_get_irq(pdev, i); return 0; } @@ -591,9 +587,6 @@ static int sh_pfc_remove(struct platform_device *pdev) } static const struct platform_device_id sh_pfc_id_table[] = { -#ifdef CONFIG_PINCTRL_PFC_R8A7740 - { "pfc-r8a7740", (kernel_ulong_t)&r8a7740_pinmux_info }, -#endif #ifdef CONFIG_PINCTRL_PFC_R8A7778 { "pfc-r8a7778", (kernel_ulong_t)&r8a7778_pinmux_info }, #endif @@ -609,9 +602,6 @@ static const struct platform_device_id sh_pfc_id_table[] = { #ifdef CONFIG_PINCTRL_PFC_SH7269 { "pfc-sh7269", (kernel_ulong_t)&sh7269_pinmux_info }, #endif -#ifdef CONFIG_PINCTRL_PFC_SH73A0 - { "pfc-sh73a0", (kernel_ulong_t)&sh73a0_pinmux_info }, -#endif #ifdef CONFIG_PINCTRL_PFC_SH7720 { "pfc-sh7720", (kernel_ulong_t)&sh7720_pinmux_info }, #endif diff --git a/drivers/pinctrl/sh-pfc/pfc-r8a7740.c b/drivers/pinctrl/sh-pfc/pfc-r8a7740.c index d0bb1459783a16..82ef1862dd1ba8 100644 --- a/drivers/pinctrl/sh-pfc/pfc-r8a7740.c +++ b/drivers/pinctrl/sh-pfc/pfc-r8a7740.c @@ -22,10 +22,6 @@ #include #include -#ifndef CONFIG_ARCH_MULTIPLATFORM -#include -#endif - #include "core.h" #include "sh_pfc.h" diff --git a/drivers/pinctrl/sh-pfc/pfc-r8a7790.c b/drivers/pinctrl/sh-pfc/pfc-r8a7790.c index baab81ead9ff4b..fc344a7c2b5382 100644 --- a/drivers/pinctrl/sh-pfc/pfc-r8a7790.c +++ b/drivers/pinctrl/sh-pfc/pfc-r8a7790.c @@ -27,10 +27,27 @@ #include "core.h" #include "sh_pfc.h" +#define PORT_GP_30(bank, fn, sfx) \ + PORT_GP_1(bank, 0, fn, sfx), PORT_GP_1(bank, 1, fn, sfx), \ + PORT_GP_1(bank, 2, fn, sfx), PORT_GP_1(bank, 3, fn, sfx), \ + PORT_GP_1(bank, 4, fn, sfx), PORT_GP_1(bank, 5, fn, sfx), \ + PORT_GP_1(bank, 6, fn, sfx), PORT_GP_1(bank, 7, fn, sfx), \ + PORT_GP_1(bank, 8, fn, sfx), PORT_GP_1(bank, 9, fn, sfx), \ + PORT_GP_1(bank, 10, fn, sfx), PORT_GP_1(bank, 11, fn, sfx), \ + PORT_GP_1(bank, 12, fn, sfx), PORT_GP_1(bank, 13, fn, sfx), \ + PORT_GP_1(bank, 14, fn, sfx), PORT_GP_1(bank, 15, fn, sfx), \ + PORT_GP_1(bank, 16, fn, sfx), PORT_GP_1(bank, 17, fn, sfx), \ + PORT_GP_1(bank, 18, fn, sfx), PORT_GP_1(bank, 19, fn, sfx), \ + PORT_GP_1(bank, 20, fn, sfx), PORT_GP_1(bank, 21, fn, sfx), \ + PORT_GP_1(bank, 22, fn, sfx), PORT_GP_1(bank, 23, fn, sfx), \ + PORT_GP_1(bank, 24, fn, sfx), PORT_GP_1(bank, 25, fn, sfx), \ + PORT_GP_1(bank, 26, fn, sfx), PORT_GP_1(bank, 27, fn, sfx), \ + PORT_GP_1(bank, 28, fn, sfx), PORT_GP_1(bank, 29, fn, sfx) + #define CPU_ALL_PORT(fn, sfx) \ PORT_GP_32(0, fn, sfx), \ - PORT_GP_32(1, fn, sfx), \ - PORT_GP_32(2, fn, sfx), \ + PORT_GP_30(1, fn, sfx), \ + PORT_GP_30(2, fn, sfx), \ PORT_GP_32(3, fn, sfx), \ PORT_GP_32(4, fn, sfx), \ PORT_GP_32(5, fn, sfx) diff --git a/drivers/pinctrl/sh-pfc/pfc-r8a7791.c b/drivers/pinctrl/sh-pfc/pfc-r8a7791.c index 3ddf23ec9f0b24..25e8117f5a1ac0 100644 --- a/drivers/pinctrl/sh-pfc/pfc-r8a7791.c +++ b/drivers/pinctrl/sh-pfc/pfc-r8a7791.c @@ -14,15 +14,30 @@ #include "core.h" #include "sh_pfc.h" +#define PORT_GP_26(bank, fn, sfx) \ + PORT_GP_1(bank, 0, fn, sfx), PORT_GP_1(bank, 1, fn, sfx), \ + PORT_GP_1(bank, 2, fn, sfx), PORT_GP_1(bank, 3, fn, sfx), \ + PORT_GP_1(bank, 4, fn, sfx), PORT_GP_1(bank, 5, fn, sfx), \ + PORT_GP_1(bank, 6, fn, sfx), PORT_GP_1(bank, 7, fn, sfx), \ + PORT_GP_1(bank, 8, fn, sfx), PORT_GP_1(bank, 9, fn, sfx), \ + PORT_GP_1(bank, 10, fn, sfx), PORT_GP_1(bank, 11, fn, sfx), \ + PORT_GP_1(bank, 12, fn, sfx), PORT_GP_1(bank, 13, fn, sfx), \ + PORT_GP_1(bank, 14, fn, sfx), PORT_GP_1(bank, 15, fn, sfx), \ + PORT_GP_1(bank, 16, fn, sfx), PORT_GP_1(bank, 17, fn, sfx), \ + PORT_GP_1(bank, 18, fn, sfx), PORT_GP_1(bank, 19, fn, sfx), \ + PORT_GP_1(bank, 20, fn, sfx), PORT_GP_1(bank, 21, fn, sfx), \ + PORT_GP_1(bank, 22, fn, sfx), PORT_GP_1(bank, 23, fn, sfx), \ + PORT_GP_1(bank, 24, fn, sfx), PORT_GP_1(bank, 25, fn, sfx) + #define CPU_ALL_PORT(fn, sfx) \ PORT_GP_32(0, fn, sfx), \ - PORT_GP_32(1, fn, sfx), \ + PORT_GP_26(1, fn, sfx), \ PORT_GP_32(2, fn, sfx), \ PORT_GP_32(3, fn, sfx), \ PORT_GP_32(4, fn, sfx), \ PORT_GP_32(5, fn, sfx), \ PORT_GP_32(6, fn, sfx), \ - PORT_GP_32(7, fn, sfx) + PORT_GP_26(7, fn, sfx) enum { PINMUX_RESERVED = 0, diff --git a/drivers/pinctrl/sh-pfc/pfc-r8a7794.c b/drivers/pinctrl/sh-pfc/pfc-r8a7794.c index bfdcac4b3bc4a4..5248685dbb4e55 100644 --- a/drivers/pinctrl/sh-pfc/pfc-r8a7794.c +++ b/drivers/pinctrl/sh-pfc/pfc-r8a7794.c @@ -2770,6 +2770,24 @@ static const unsigned int sdhi2_wp_pins[] = { static const unsigned int sdhi2_wp_mux[] = { SD2_WP_MARK, }; +/* - USB0 ------------------------------------------------------------------- */ +static const unsigned int usb0_pins[] = { + RCAR_GP_PIN(5, 24), /* PWEN */ + RCAR_GP_PIN(5, 25), /* OVC */ +}; +static const unsigned int usb0_mux[] = { + USB0_PWEN_MARK, + USB0_OVC_MARK, +}; +/* - USB1 ------------------------------------------------------------------- */ +static const unsigned int usb1_pins[] = { + RCAR_GP_PIN(5, 26), /* PWEN */ + RCAR_GP_PIN(5, 27), /* OVC */ +}; +static const unsigned int usb1_mux[] = { + USB1_PWEN_MARK, + USB1_OVC_MARK, +}; static const struct sh_pfc_pin_group pinmux_groups[] = { SH_PFC_PIN_GROUP(eth_link), @@ -2945,6 +2963,8 @@ static const struct sh_pfc_pin_group pinmux_groups[] = { SH_PFC_PIN_GROUP(sdhi2_ctrl), SH_PFC_PIN_GROUP(sdhi2_cd), SH_PFC_PIN_GROUP(sdhi2_wp), + SH_PFC_PIN_GROUP(usb0), + SH_PFC_PIN_GROUP(usb1), }; static const char * const eth_groups[] = { @@ -3219,6 +3239,14 @@ static const char * const sdhi2_groups[] = { "sdhi2_wp", }; +static const char * const usb0_groups[] = { + "usb0", +}; + +static const char * const usb1_groups[] = { + "usb1", +}; + static const struct sh_pfc_function pinmux_functions[] = { SH_PFC_FUNCTION(eth), SH_PFC_FUNCTION(hscif0), @@ -3253,6 +3281,8 @@ static const struct sh_pfc_function pinmux_functions[] = { SH_PFC_FUNCTION(sdhi0), SH_PFC_FUNCTION(sdhi1), SH_PFC_FUNCTION(sdhi2), + SH_PFC_FUNCTION(usb0), + SH_PFC_FUNCTION(usb1), }; static const struct pinmux_cfg_reg pinmux_config_regs[] = { diff --git a/drivers/pinctrl/sh-pfc/pfc-sh73a0.c b/drivers/pinctrl/sh-pfc/pfc-sh73a0.c index d2efbfb776ac57..097526576f8818 100644 --- a/drivers/pinctrl/sh-pfc/pfc-sh73a0.c +++ b/drivers/pinctrl/sh-pfc/pfc-sh73a0.c @@ -26,10 +26,6 @@ #include #include -#ifndef CONFIG_ARCH_MULTIPLATFORM -#include -#endif - #include "core.h" #include "sh_pfc.h" diff --git a/drivers/pinctrl/sh-pfc/pinctrl.c b/drivers/pinctrl/sh-pfc/pinctrl.c index ff678966008b7c..863c3e30ce05f9 100644 --- a/drivers/pinctrl/sh-pfc/pinctrl.c +++ b/drivers/pinctrl/sh-pfc/pinctrl.c @@ -40,6 +40,10 @@ struct sh_pfc_pinctrl { struct pinctrl_pin_desc *pins; struct sh_pfc_pin_config *configs; + + const char *func_prop_name; + const char *groups_prop_name; + const char *pins_prop_name; }; static int sh_pfc_get_groups_count(struct pinctrl_dev *pctldev) @@ -96,10 +100,13 @@ static int sh_pfc_map_add_config(struct pinctrl_map *map, return 0; } -static int sh_pfc_dt_subnode_to_map(struct device *dev, struct device_node *np, +static int sh_pfc_dt_subnode_to_map(struct pinctrl_dev *pctldev, + struct device_node *np, struct pinctrl_map **map, unsigned int *num_maps, unsigned int *index) { + struct sh_pfc_pinctrl *pmx = pinctrl_dev_get_drvdata(pctldev); + struct device *dev = pmx->pfc->dev; struct pinctrl_map *maps = *map; unsigned int nmaps = *num_maps; unsigned int idx = *index; @@ -113,10 +120,27 @@ static int sh_pfc_dt_subnode_to_map(struct device *dev, struct device_node *np, const char *pin; int ret; + /* Support both the old Renesas-specific properties and the new standard + * properties. Mixing old and new properties isn't allowed, neither + * inside a subnode nor across subnodes. + */ + if (!pmx->func_prop_name) { + if (of_find_property(np, "groups", NULL) || + of_find_property(np, "pins", NULL)) { + pmx->func_prop_name = "function"; + pmx->groups_prop_name = "groups"; + pmx->pins_prop_name = "pins"; + } else { + pmx->func_prop_name = "renesas,function"; + pmx->groups_prop_name = "renesas,groups"; + pmx->pins_prop_name = "renesas,pins"; + } + } + /* Parse the function and configuration properties. At least a function * or one configuration must be specified. */ - ret = of_property_read_string(np, "renesas,function", &function); + ret = of_property_read_string(np, pmx->func_prop_name, &function); if (ret < 0 && ret != -EINVAL) { dev_err(dev, "Invalid function in DT\n"); return ret; @@ -129,11 +153,12 @@ static int sh_pfc_dt_subnode_to_map(struct device *dev, struct device_node *np, if (!function && num_configs == 0) { dev_err(dev, "DT node must contain at least a function or config\n"); + ret = -ENODEV; goto done; } /* Count the number of pins and groups and reallocate mappings. */ - ret = of_property_count_strings(np, "renesas,pins"); + ret = of_property_count_strings(np, pmx->pins_prop_name); if (ret == -EINVAL) { num_pins = 0; } else if (ret < 0) { @@ -143,7 +168,7 @@ static int sh_pfc_dt_subnode_to_map(struct device *dev, struct device_node *np, num_pins = ret; } - ret = of_property_count_strings(np, "renesas,groups"); + ret = of_property_count_strings(np, pmx->groups_prop_name); if (ret == -EINVAL) { num_groups = 0; } else if (ret < 0) { @@ -174,7 +199,7 @@ static int sh_pfc_dt_subnode_to_map(struct device *dev, struct device_node *np, *num_maps = nmaps; /* Iterate over pins and groups and create the mappings. */ - of_property_for_each_string(np, "renesas,groups", prop, group) { + of_property_for_each_string(np, pmx->groups_prop_name, prop, group) { if (function) { maps[idx].type = PIN_MAP_TYPE_MUX_GROUP; maps[idx].data.mux.group = group; @@ -198,7 +223,7 @@ static int sh_pfc_dt_subnode_to_map(struct device *dev, struct device_node *np, goto done; } - of_property_for_each_string(np, "renesas,pins", prop, pin) { + of_property_for_each_string(np, pmx->pins_prop_name, prop, pin) { ret = sh_pfc_map_add_config(&maps[idx], pin, PIN_MAP_TYPE_CONFIGS_PIN, configs, num_configs); @@ -246,7 +271,7 @@ static int sh_pfc_dt_node_to_map(struct pinctrl_dev *pctldev, index = 0; for_each_child_of_node(np, child) { - ret = sh_pfc_dt_subnode_to_map(dev, child, map, num_maps, + ret = sh_pfc_dt_subnode_to_map(pctldev, child, map, num_maps, &index); if (ret < 0) goto done; @@ -254,7 +279,8 @@ static int sh_pfc_dt_node_to_map(struct pinctrl_dev *pctldev, /* If no mapping has been found in child nodes try the config node. */ if (*num_maps == 0) { - ret = sh_pfc_dt_subnode_to_map(dev, np, map, num_maps, &index); + ret = sh_pfc_dt_subnode_to_map(pctldev, np, map, num_maps, + &index); if (ret < 0) goto done; } @@ -465,6 +491,9 @@ static bool sh_pfc_pinconf_validate(struct sh_pfc *pfc, unsigned int _pin, case PIN_CONFIG_BIAS_PULL_DOWN: return pin->configs & SH_PFC_PIN_CFG_PULL_DOWN; + case PIN_CONFIG_POWER_SOURCE: + return pin->configs & SH_PFC_PIN_CFG_IO_VOLTAGE; + default: return false; } @@ -477,7 +506,6 @@ static int sh_pfc_pinconf_get(struct pinctrl_dev *pctldev, unsigned _pin, struct sh_pfc *pfc = pmx->pfc; enum pin_config_param param = pinconf_to_config_param(*config); unsigned long flags; - unsigned int bias; if (!sh_pfc_pinconf_validate(pfc, _pin, param)) return -ENOTSUPP; @@ -485,7 +513,9 @@ static int sh_pfc_pinconf_get(struct pinctrl_dev *pctldev, unsigned _pin, switch (param) { case PIN_CONFIG_BIAS_DISABLE: case PIN_CONFIG_BIAS_PULL_UP: - case PIN_CONFIG_BIAS_PULL_DOWN: + case PIN_CONFIG_BIAS_PULL_DOWN: { + unsigned int bias; + if (!pfc->info->ops || !pfc->info->ops->get_bias) return -ENOTSUPP; @@ -498,6 +528,24 @@ static int sh_pfc_pinconf_get(struct pinctrl_dev *pctldev, unsigned _pin, *config = 0; break; + } + + case PIN_CONFIG_POWER_SOURCE: { + int ret; + + if (!pfc->info->ops || !pfc->info->ops->get_io_voltage) + return -ENOTSUPP; + + spin_lock_irqsave(&pfc->lock, flags); + ret = pfc->info->ops->get_io_voltage(pfc, _pin); + spin_unlock_irqrestore(&pfc->lock, flags); + + if (ret < 0) + return ret; + + *config = ret; + break; + } default: return -ENOTSUPP; @@ -534,6 +582,24 @@ static int sh_pfc_pinconf_set(struct pinctrl_dev *pctldev, unsigned _pin, break; + case PIN_CONFIG_POWER_SOURCE: { + unsigned int arg = + pinconf_to_config_argument(configs[i]); + int ret; + + if (!pfc->info->ops || !pfc->info->ops->set_io_voltage) + return -ENOTSUPP; + + spin_lock_irqsave(&pfc->lock, flags); + ret = pfc->info->ops->set_io_voltage(pfc, _pin, arg); + spin_unlock_irqrestore(&pfc->lock, flags); + + if (ret) + return ret; + + break; + } + default: return -ENOTSUPP; } diff --git a/drivers/pinctrl/sh-pfc/sh_pfc.h b/drivers/pinctrl/sh-pfc/sh_pfc.h index 0874cfee6889af..15afd49fd4e33b 100644 --- a/drivers/pinctrl/sh-pfc/sh_pfc.h +++ b/drivers/pinctrl/sh-pfc/sh_pfc.h @@ -12,6 +12,7 @@ #define __SH_PFC_H #include +#include #include enum { @@ -26,6 +27,7 @@ enum { #define SH_PFC_PIN_CFG_OUTPUT (1 << 1) #define SH_PFC_PIN_CFG_PULL_UP (1 << 2) #define SH_PFC_PIN_CFG_PULL_DOWN (1 << 3) +#define SH_PFC_PIN_CFG_IO_VOLTAGE (1 << 4) #define SH_PFC_PIN_CFG_NO_GPIO (1 << 31) struct sh_pfc_pin { @@ -121,6 +123,9 @@ struct sh_pfc_soc_operations { unsigned int (*get_bias)(struct sh_pfc *pfc, unsigned int pin); void (*set_bias)(struct sh_pfc *pfc, unsigned int pin, unsigned int bias); + int (*get_io_voltage)(struct sh_pfc *pfc, unsigned int pin); + int (*set_io_voltage)(struct sh_pfc *pfc, unsigned int pin, + u16 voltage_mV); }; struct sh_pfc_soc_info { diff --git a/drivers/pinctrl/sirf/pinctrl-atlas7.c b/drivers/pinctrl/sirf/pinctrl-atlas7.c index 9384e0aa399603..9df0c5f25824a2 100644 --- a/drivers/pinctrl/sirf/pinctrl-atlas7.c +++ b/drivers/pinctrl/sirf/pinctrl-atlas7.c @@ -148,6 +148,19 @@ enum altas7_pad_type { #define DIV_DISABLE 0x1 #define DIV_ENABLE 0x0 +/* Number of Function input disable registers */ +#define NUM_OF_IN_DISABLE_REG 0x2 + +/* Offset of Function input disable registers */ +#define IN_DISABLE_0_REG_SET 0x0A00 +#define IN_DISABLE_0_REG_CLR 0x0A04 +#define IN_DISABLE_1_REG_SET 0x0A08 +#define IN_DISABLE_1_REG_CLR 0x0A0C +#define IN_DISABLE_VAL_0_REG_SET 0x0A80 +#define IN_DISABLE_VAL_0_REG_CLR 0x0A84 +#define IN_DISABLE_VAL_1_REG_SET 0x0A88 +#define IN_DISABLE_VAL_1_REG_CLR 0x0A8C + struct dt_params { const char *property; int value; @@ -196,6 +209,16 @@ struct atlas7_pad_config { .ad_ctrl_bit = adb, \ } +/** + * struct atlas7_pad_status - Atlas7 Pad status + */ +struct atlas7_pad_status { + u8 func; + u8 pull; + u8 dstr; + u8 reserved; +}; + /** * struct atlas7_pad_mux - Atlas7 mux * @bank: The bank of this pad's registers on. @@ -285,6 +308,9 @@ struct atlas7_pinctrl_data { /* Platform info of atlas7 pinctrl */ #define ATLAS7_PINCTRL_REG_BANKS 2 #define ATLAS7_PINCTRL_BANK_0_PINS 18 +#define ATLAS7_PINCTRL_BANK_1_PINS 141 +#define ATLAS7_PINCTRL_TOTAL_PINS \ + (ATLAS7_PINCTRL_BANK_0_PINS + ATLAS7_PINCTRL_BANK_1_PINS) /** * Atlas7 GPIO Chip @@ -316,6 +342,7 @@ struct atlas7_gpio_bank { unsigned int gpio_offset; unsigned int ngpio; const unsigned int *gpio_pins; + u32 sleep_data[NGPIO_OF_BANK]; }; struct atlas7_gpio_chip { @@ -343,6 +370,9 @@ struct atlas7_pmx { struct pinctrl_desc pctl_desc; struct atlas7_pinctrl_data *pctl_data; void __iomem *regs[ATLAS7_PINCTRL_REG_BANKS]; + u32 status_ds[NUM_OF_IN_DISABLE_REG]; + u32 status_dsv[NUM_OF_IN_DISABLE_REG]; + struct atlas7_pad_status sleep_data[ATLAS7_PINCTRL_TOTAL_PINS]; }; /* @@ -3480,6 +3510,160 @@ struct atlas7_pinctrl_data atlas7_ioc_data = { .confs_cnt = ARRAY_SIZE(atlas7_ioc_pad_confs), }; +/* Simple map data structure */ +struct map_data { + u8 idx; + u8 data; +}; + +/** + * struct atlas7_pull_info - Atlas7 Pad pull info + * @type:The type of this Pad. + * @mask:The mas value of this pin's pull bits. + * @v2s: The map of pull register value to pull status. + * @s2v: The map of pull status to pull register value. + */ +struct atlas7_pull_info { + u8 pad_type; + u8 mask; + const struct map_data *v2s; + const struct map_data *s2v; +}; + +/* Pull Register value map to status */ +static const struct map_data p4we_pull_v2s[] = { + { P4WE_PULL_UP, PULL_UP }, + { P4WE_HIGH_HYSTERESIS, HIGH_HYSTERESIS }, + { P4WE_HIGH_Z, HIGH_Z }, + { P4WE_PULL_DOWN, PULL_DOWN }, +}; + +static const struct map_data p16st_pull_v2s[] = { + { P16ST_PULL_UP, PULL_UP }, + { PD, PULL_UNKNOWN }, + { P16ST_HIGH_Z, HIGH_Z }, + { P16ST_PULL_DOWN, PULL_DOWN }, +}; + +static const struct map_data pm31_pull_v2s[] = { + { PM31_PULL_DISABLED, PULL_DOWN }, + { PM31_PULL_ENABLED, PULL_UP }, +}; + +static const struct map_data pangd_pull_v2s[] = { + { PANGD_PULL_UP, PULL_UP }, + { PD, PULL_UNKNOWN }, + { PANGD_HIGH_Z, HIGH_Z }, + { PANGD_PULL_DOWN, PULL_DOWN }, +}; + +/* Pull status map to register value */ +static const struct map_data p4we_pull_s2v[] = { + { PULL_UP, P4WE_PULL_UP }, + { HIGH_HYSTERESIS, P4WE_HIGH_HYSTERESIS }, + { HIGH_Z, P4WE_HIGH_Z }, + { PULL_DOWN, P4WE_PULL_DOWN }, + { PULL_DISABLE, -1 }, + { PULL_ENABLE, -1 }, +}; + +static const struct map_data p16st_pull_s2v[] = { + { PULL_UP, P16ST_PULL_UP }, + { HIGH_HYSTERESIS, -1 }, + { HIGH_Z, P16ST_HIGH_Z }, + { PULL_DOWN, P16ST_PULL_DOWN }, + { PULL_DISABLE, -1 }, + { PULL_ENABLE, -1 }, +}; + +static const struct map_data pm31_pull_s2v[] = { + { PULL_UP, PM31_PULL_ENABLED }, + { HIGH_HYSTERESIS, -1 }, + { HIGH_Z, -1 }, + { PULL_DOWN, PM31_PULL_DISABLED }, + { PULL_DISABLE, -1 }, + { PULL_ENABLE, -1 }, +}; + +static const struct map_data pangd_pull_s2v[] = { + { PULL_UP, PANGD_PULL_UP }, + { HIGH_HYSTERESIS, -1 }, + { HIGH_Z, PANGD_HIGH_Z }, + { PULL_DOWN, PANGD_PULL_DOWN }, + { PULL_DISABLE, -1 }, + { PULL_ENABLE, -1 }, +}; + +static const struct atlas7_pull_info atlas7_pull_map[] = { + { PAD_T_4WE_PD, P4WE_PULL_MASK, p4we_pull_v2s, p4we_pull_s2v }, + { PAD_T_4WE_PU, P4WE_PULL_MASK, p4we_pull_v2s, p4we_pull_s2v }, + { PAD_T_16ST, P16ST_PULL_MASK, p16st_pull_v2s, p16st_pull_s2v }, + { PAD_T_M31_0204_PD, PM31_PULL_MASK, pm31_pull_v2s, pm31_pull_s2v }, + { PAD_T_M31_0204_PU, PM31_PULL_MASK, pm31_pull_v2s, pm31_pull_s2v }, + { PAD_T_M31_0610_PD, PM31_PULL_MASK, pm31_pull_v2s, pm31_pull_s2v }, + { PAD_T_M31_0610_PU, PM31_PULL_MASK, pm31_pull_v2s, pm31_pull_s2v }, + { PAD_T_AD, PANGD_PULL_MASK, pangd_pull_v2s, pangd_pull_s2v }, +}; + +/** + * struct atlas7_ds_ma_info - Atlas7 Pad DriveStrength & currents info + * @ma: The Drive Strength in current value . + * @ds_16st: The correspond raw value of 16st pad. + * @ds_4we: The correspond raw value of 4we pad. + * @ds_0204m31: The correspond raw value of 0204m31 pad. + * @ds_0610m31: The correspond raw value of 0610m31 pad. + */ +struct atlas7_ds_ma_info { + u32 ma; + u32 ds_16st; + u32 ds_4we; + u32 ds_0204m31; + u32 ds_0610m31; +}; + +static const struct atlas7_ds_ma_info atlas7_ma2ds_map[] = { + { 2, DS_16ST_0, DS_4WE_0, DS_M31_0, DS_NULL }, + { 4, DS_16ST_1, DS_NULL, DS_M31_1, DS_NULL }, + { 6, DS_16ST_2, DS_NULL, DS_NULL, DS_M31_0 }, + { 8, DS_16ST_3, DS_4WE_1, DS_NULL, DS_NULL }, + { 10, DS_16ST_4, DS_NULL, DS_NULL, DS_M31_1 }, + { 12, DS_16ST_5, DS_NULL, DS_NULL, DS_NULL }, + { 14, DS_16ST_6, DS_NULL, DS_NULL, DS_NULL }, + { 16, DS_16ST_7, DS_4WE_2, DS_NULL, DS_NULL }, + { 18, DS_16ST_8, DS_NULL, DS_NULL, DS_NULL }, + { 20, DS_16ST_9, DS_NULL, DS_NULL, DS_NULL }, + { 22, DS_16ST_10, DS_NULL, DS_NULL, DS_NULL }, + { 24, DS_16ST_11, DS_NULL, DS_NULL, DS_NULL }, + { 26, DS_16ST_12, DS_NULL, DS_NULL, DS_NULL }, + { 28, DS_16ST_13, DS_4WE_3, DS_NULL, DS_NULL }, + { 30, DS_16ST_14, DS_NULL, DS_NULL, DS_NULL }, + { 32, DS_16ST_15, DS_NULL, DS_NULL, DS_NULL }, +}; + +/** + * struct atlas7_ds_info - Atlas7 Pad DriveStrength info + * @type: The type of this Pad. + * @mask: The mask value of this pin's pull bits. + * @imval: The immediate value of drives trength register. + */ +struct atlas7_ds_info { + u8 type; + u8 mask; + u8 imval; + u8 reserved; +}; + +static const struct atlas7_ds_info atlas7_ds_map[] = { + { PAD_T_4WE_PD, DS_2BIT_MASK, DS_2BIT_IM_VAL }, + { PAD_T_4WE_PU, DS_2BIT_MASK, DS_2BIT_IM_VAL }, + { PAD_T_16ST, DS_4BIT_MASK, DS_4BIT_IM_VAL }, + { PAD_T_M31_0204_PD, DS_1BIT_MASK, DS_1BIT_IM_VAL }, + { PAD_T_M31_0204_PU, DS_1BIT_MASK, DS_1BIT_IM_VAL }, + { PAD_T_M31_0610_PD, DS_1BIT_MASK, DS_1BIT_IM_VAL }, + { PAD_T_M31_0610_PU, DS_1BIT_MASK, DS_1BIT_IM_VAL }, + { PAD_T_AD, DS_NULL, DS_NULL }, +}; + static inline u32 atlas7_pin_to_bank(u32 pin) { return (pin >= ATLAS7_PINCTRL_BANK_0_PINS) ? 1 : 0; @@ -3682,49 +3866,22 @@ static int atlas7_pmx_set_mux(struct pinctrl_dev *pctldev, return 0; } -struct atlas7_ds_info { - u32 ma; - u32 ds_16st; - u32 ds_4we; - u32 ds_0204m31; - u32 ds_0610m31; -}; - -const struct atlas7_ds_info atlas7_ds_map[] = { - { 2, DS_16ST_0, DS_4WE_0, DS_M31_0, DS_NULL}, - { 4, DS_16ST_1, DS_NULL, DS_M31_1, DS_NULL}, - { 6, DS_16ST_2, DS_NULL, DS_NULL, DS_M31_0}, - { 8, DS_16ST_3, DS_4WE_1, DS_NULL, DS_NULL}, - { 10, DS_16ST_4, DS_NULL, DS_NULL, DS_M31_1}, - { 12, DS_16ST_5, DS_NULL, DS_NULL, DS_NULL}, - { 14, DS_16ST_6, DS_NULL, DS_NULL, DS_NULL}, - { 16, DS_16ST_7, DS_4WE_2, DS_NULL, DS_NULL}, - { 18, DS_16ST_8, DS_NULL, DS_NULL, DS_NULL}, - { 20, DS_16ST_9, DS_NULL, DS_NULL, DS_NULL}, - { 22, DS_16ST_10, DS_NULL, DS_NULL, DS_NULL}, - { 24, DS_16ST_11, DS_NULL, DS_NULL, DS_NULL}, - { 26, DS_16ST_12, DS_NULL, DS_NULL, DS_NULL}, - { 28, DS_16ST_13, DS_4WE_3, DS_NULL, DS_NULL}, - { 30, DS_16ST_14, DS_NULL, DS_NULL, DS_NULL}, - { 32, DS_16ST_15, DS_NULL, DS_NULL, DS_NULL}, -}; - static u32 convert_current_to_drive_strength(u32 type, u32 ma) { int idx; - for (idx = 0; idx < ARRAY_SIZE(atlas7_ds_map); idx++) { - if (atlas7_ds_map[idx].ma != ma) + for (idx = 0; idx < ARRAY_SIZE(atlas7_ma2ds_map); idx++) { + if (atlas7_ma2ds_map[idx].ma != ma) continue; if (type == PAD_T_4WE_PD || type == PAD_T_4WE_PU) - return atlas7_ds_map[idx].ds_4we; + return atlas7_ma2ds_map[idx].ds_4we; else if (type == PAD_T_16ST) - return atlas7_ds_map[idx].ds_16st; + return atlas7_ma2ds_map[idx].ds_16st; else if (type == PAD_T_M31_0204_PD || type == PAD_T_M31_0204_PU) - return atlas7_ds_map[idx].ds_0204m31; + return atlas7_ma2ds_map[idx].ds_0204m31; else if (type == PAD_T_M31_0610_PD || type == PAD_T_M31_0610_PU) - return atlas7_ds_map[idx].ds_0610m31; + return atlas7_ma2ds_map[idx].ds_0610m31; } return DS_NULL; @@ -3735,78 +3892,21 @@ static int altas7_pinctrl_set_pull_sel(struct pinctrl_dev *pctldev, { struct atlas7_pmx *pmx = pinctrl_dev_get_drvdata(pctldev); struct atlas7_pad_config *conf = &pmx->pctl_data->confs[pin]; - u32 type = conf->type; - u32 shift = conf->pupd_bit; - u32 bank = atlas7_pin_to_bank(pin); - void __iomem *pull_sel_reg, *pull_clr_reg; + const struct atlas7_pull_info *pull_info; + u32 bank; + unsigned long regv; + void __iomem *pull_sel_reg; + bank = atlas7_pin_to_bank(pin); + pull_info = &atlas7_pull_map[conf->type]; pull_sel_reg = pmx->regs[bank] + conf->pupd_reg; - pull_clr_reg = CLR_REG(pull_sel_reg); - - if (type == PAD_T_4WE_PD || type == PAD_T_4WE_PU) { - writel(P4WE_PULL_MASK << shift, pull_clr_reg); - - if (sel == PULL_UP) - writel(P4WE_PULL_UP << shift, pull_sel_reg); - else if (sel == HIGH_HYSTERESIS) - writel(P4WE_HIGH_HYSTERESIS << shift, pull_sel_reg); - else if (sel == HIGH_Z) - writel(P4WE_HIGH_Z << shift, pull_sel_reg); - else if (sel == PULL_DOWN) - writel(P4WE_PULL_DOWN << shift, pull_sel_reg); - else { - pr_err("Unknown Pull select type for 4WEPAD#%d\n", - pin); - return -ENOTSUPP; - } - } else if (type == PAD_T_16ST) { - writel(P16ST_PULL_MASK << shift, pull_clr_reg); - - if (sel == PULL_UP) - writel(P16ST_PULL_UP << shift, pull_sel_reg); - else if (sel == HIGH_Z) - writel(P16ST_HIGH_Z << shift, pull_sel_reg); - else if (sel == PULL_DOWN) - writel(P16ST_PULL_DOWN << shift, pull_sel_reg); - else { - pr_err("Unknown Pull select type for 16STPAD#%d\n", - pin); - return -ENOTSUPP; - } - } else if (type == PAD_T_M31_0204_PD || - type == PAD_T_M31_0204_PU || - type == PAD_T_M31_0610_PD || - type == PAD_T_M31_0610_PU) { - writel(PM31_PULL_MASK << shift, pull_clr_reg); - - if (sel == PULL_UP) - writel(PM31_PULL_ENABLED << shift, pull_sel_reg); - else if (sel == PULL_DOWN) - writel(PM31_PULL_DISABLED << shift, pull_sel_reg); - else { - pr_err("Unknown Pull select type for M31PAD#%d\n", - pin); - return -ENOTSUPP; - } - } else if (type == PAD_T_AD) { - writel(PANGD_PULL_MASK << shift, pull_clr_reg); - - if (sel == PULL_UP) - writel(PANGD_PULL_UP << shift, pull_sel_reg); - else if (sel == HIGH_Z) - writel(PANGD_HIGH_Z << shift, pull_sel_reg); - else if (sel == PULL_DOWN) - writel(PANGD_PULL_DOWN << shift, pull_sel_reg); - else { - pr_err("Unknown Pull select type for A/D PAD#%d\n", - pin); - return -ENOTSUPP; - } - } else { - pr_err("Unknown Pad type[%d] for pull select PAD#%d\n", - type, pin); - return -ENOTSUPP; - } + + /* Retrieve correspond register value from table by sel */ + regv = pull_info->s2v[sel].data & pull_info->mask; + + /* Clear & Set new value to pull register */ + writel(pull_info->mask << conf->pupd_bit, CLR_REG(pull_sel_reg)); + writel(regv << conf->pupd_bit, pull_sel_reg); pr_debug("PIN_CFG ### SET PIN#%d PULL SELECTOR:%d == OK ####\n", pin, sel); @@ -3818,43 +3918,25 @@ static int __altas7_pinctrl_set_drive_strength_sel(struct pinctrl_dev *pctldev, { struct atlas7_pmx *pmx = pinctrl_dev_get_drvdata(pctldev); struct atlas7_pad_config *conf = &pmx->pctl_data->confs[pin]; - u32 type = conf->type; - u32 shift = conf->drvstr_bit; - u32 bank = atlas7_pin_to_bank(pin); - void __iomem *ds_sel_reg, *ds_clr_reg; - - ds_sel_reg = pmx->regs[bank] + conf->drvstr_reg; - ds_clr_reg = CLR_REG(ds_sel_reg); - if (type == PAD_T_4WE_PD || type == PAD_T_4WE_PU) { - if (sel & (~DS_2BIT_MASK)) - goto unsupport; - - writel(DS_2BIT_IM_VAL << shift, ds_clr_reg); - writel(sel << shift, ds_sel_reg); - - return 0; - } else if (type == PAD_T_16ST) { - if (sel & (~DS_4BIT_MASK)) - goto unsupport; + const struct atlas7_ds_info *ds_info; + u32 bank; + void __iomem *ds_sel_reg; - writel(DS_4BIT_IM_VAL << shift, ds_clr_reg); - writel(sel << shift, ds_sel_reg); + ds_info = &atlas7_ds_map[conf->type]; + if (sel & (~(ds_info->mask))) + goto unsupport; - return 0; - } else if (type == PAD_T_M31_0204_PD || type == PAD_T_M31_0204_PU || - type == PAD_T_M31_0610_PD || type == PAD_T_M31_0610_PU) { - if (sel & (~DS_1BIT_MASK)) - goto unsupport; + bank = atlas7_pin_to_bank(pin); + ds_sel_reg = pmx->regs[bank] + conf->drvstr_reg; - writel(DS_1BIT_IM_VAL << shift, ds_clr_reg); - writel(sel << shift, ds_sel_reg); + writel(ds_info->imval << conf->drvstr_bit, CLR_REG(ds_sel_reg)); + writel(sel << conf->drvstr_bit, ds_sel_reg); - return 0; - } + return 0; unsupport: pr_err("Pad#%d type[%d] doesn't support ds code[%d]!\n", - pin, type, sel); + pin, conf->type, sel); return -ENOTSUPP; } @@ -4101,14 +4183,135 @@ static int atlas7_pinmux_probe(struct platform_device *pdev) return ret; } +#ifdef CONFIG_PM_SLEEP +static int atlas7_pinmux_suspend_noirq(struct device *dev) +{ + struct atlas7_pmx *pmx = dev_get_drvdata(dev); + struct atlas7_pad_status *status; + struct atlas7_pad_config *conf; + const struct atlas7_ds_info *ds_info; + const struct atlas7_pull_info *pull_info; + int idx; + u32 bank; + unsigned long regv; + + for (idx = 0; idx < pmx->pctl_desc.npins; idx++) { + /* Get this Pad's descriptor from PINCTRL */ + conf = &pmx->pctl_data->confs[idx]; + bank = atlas7_pin_to_bank(idx); + status = &pmx->sleep_data[idx]; + + /* Save Function selector */ + regv = readl(pmx->regs[bank] + conf->mux_reg); + status->func = (regv >> conf->mux_bit) & FUNC_CLEAR_MASK; + + /* Check if Pad is in Analogue selector */ + if (conf->ad_ctrl_reg == -1) + goto save_ds_sel; + + regv = readl(pmx->regs[bank] + conf->ad_ctrl_reg); + if (!(regv & (conf->ad_ctrl_bit << ANA_CLEAR_MASK))) + status->func = FUNC_ANALOGUE; + +save_ds_sel: + if (conf->drvstr_reg == -1) + goto save_pull_sel; + + /* Save Drive Strength selector */ + ds_info = &atlas7_ds_map[conf->type]; + regv = readl(pmx->regs[bank] + conf->drvstr_reg); + status->dstr = (regv >> conf->drvstr_bit) & ds_info->mask; + +save_pull_sel: + /* Save Pull selector */ + pull_info = &atlas7_pull_map[conf->type]; + regv = readl(pmx->regs[bank] + conf->pupd_reg); + regv = (regv >> conf->pupd_bit) & pull_info->mask; + status->pull = pull_info->v2s[regv].data; + } + + /* + * Save disable input selector, this selector is not for Pin, + * but for Mux function. + */ + for (idx = 0; idx < NUM_OF_IN_DISABLE_REG; idx++) { + pmx->status_ds[idx] = readl(pmx->regs[BANK_DS] + + IN_DISABLE_0_REG_SET + 0x8 * idx); + pmx->status_dsv[idx] = readl(pmx->regs[BANK_DS] + + IN_DISABLE_VAL_0_REG_SET + 0x8 * idx); + } + + return 0; +} + +static int atlas7_pinmux_resume_noirq(struct device *dev) +{ + struct atlas7_pmx *pmx = dev_get_drvdata(dev); + struct atlas7_pad_status *status; + struct atlas7_pad_config *conf; + int idx; + u32 bank; + + for (idx = 0; idx < pmx->pctl_desc.npins; idx++) { + /* Get this Pad's descriptor from PINCTRL */ + conf = &pmx->pctl_data->confs[idx]; + bank = atlas7_pin_to_bank(idx); + status = &pmx->sleep_data[idx]; + + /* Restore Function selector */ + __atlas7_pmx_pin_enable(pmx, idx, (u32)status->func & 0xff); + + if (FUNC_ANALOGUE == status->func) + goto restore_pull_sel; + + /* Restore Drive Strength selector */ + __altas7_pinctrl_set_drive_strength_sel(pmx->pctl, idx, + (u32)status->dstr & 0xff); + +restore_pull_sel: + /* Restore Pull selector */ + altas7_pinctrl_set_pull_sel(pmx->pctl, idx, + (u32)status->pull & 0xff); + } + + /* + * Restore disable input selector, this selector is not for Pin, + * but for Mux function + */ + for (idx = 0; idx < NUM_OF_IN_DISABLE_REG; idx++) { + writel(~0, pmx->regs[BANK_DS] + + IN_DISABLE_0_REG_CLR + 0x8 * idx); + writel(pmx->status_ds[idx], pmx->regs[BANK_DS] + + IN_DISABLE_0_REG_SET + 0x8 * idx); + writel(~0, pmx->regs[BANK_DS] + + IN_DISABLE_VAL_0_REG_CLR + 0x8 * idx); + writel(pmx->status_dsv[idx], pmx->regs[BANK_DS] + + IN_DISABLE_VAL_0_REG_SET + 0x8 * idx); + } + + return 0; +} + +static const struct dev_pm_ops atlas7_pinmux_pm_ops = { + .suspend_noirq = atlas7_pinmux_suspend_noirq, + .resume_noirq = atlas7_pinmux_resume_noirq, + .freeze_noirq = atlas7_pinmux_suspend_noirq, + .restore_noirq = atlas7_pinmux_resume_noirq, +}; +#endif + static const struct of_device_id atlas7_pinmux_ids[] = { { .compatible = "sirf,atlas7-ioc",}, + {}, }; static struct platform_driver atlas7_pinmux_driver = { .driver = { .name = "atlas7-ioc", .of_match_table = atlas7_pinmux_ids, +#ifdef CONFIG_PM_SLEEP + .pm = &atlas7_pinmux_pm_ops, +#endif }, .probe = atlas7_pinmux_probe, }; @@ -4286,14 +4489,15 @@ static struct irq_chip atlas7_gpio_irq_chip = { .irq_set_type = atlas7_gpio_irq_type, }; -static void atlas7_gpio_handle_irq(unsigned int irq, struct irq_desc *desc) +static void atlas7_gpio_handle_irq(unsigned int __irq, struct irq_desc *desc) { struct gpio_chip *gc = irq_desc_get_handler_data(desc); struct atlas7_gpio_chip *a7gc = to_atlas7_gpio(gc); struct atlas7_gpio_bank *bank = NULL; u32 status, ctrl; int pin_in_bank = 0, idx; - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); + unsigned int irq = irq_desc_get_irq(desc); for (idx = 0; idx < a7gc->nbank; idx++) { bank = &a7gc->banks[idx]; @@ -4496,6 +4700,7 @@ static void atlas7_gpio_set_value(struct gpio_chip *chip, static const struct of_device_id atlas7_gpio_ids[] = { { .compatible = "sirf,atlas7-gpio", }, + {}, }; static int atlas7_gpio_probe(struct platform_device *pdev) @@ -4612,17 +4817,65 @@ static int atlas7_gpio_probe(struct platform_device *pdev) BUG_ON(!bank->pctldev); } + platform_set_drvdata(pdev, a7gc); dev_info(&pdev->dev, "add to system.\n"); return 0; failed: return ret; } +#ifdef CONFIG_PM_SLEEP +static int atlas7_gpio_suspend_noirq(struct device *dev) +{ + struct atlas7_gpio_chip *a7gc = dev_get_drvdata(dev); + struct atlas7_gpio_bank *bank; + void __iomem *ctrl_reg; + u32 idx, pin; + + for (idx = 0; idx < a7gc->nbank; idx++) { + bank = &a7gc->banks[idx]; + for (pin = 0; pin < bank->ngpio; pin++) { + ctrl_reg = ATLAS7_GPIO_CTRL(bank, pin); + bank->sleep_data[pin] = readl(ctrl_reg); + } + } + + return 0; +} + +static int atlas7_gpio_resume_noirq(struct device *dev) +{ + struct atlas7_gpio_chip *a7gc = dev_get_drvdata(dev); + struct atlas7_gpio_bank *bank; + void __iomem *ctrl_reg; + u32 idx, pin; + + for (idx = 0; idx < a7gc->nbank; idx++) { + bank = &a7gc->banks[idx]; + for (pin = 0; pin < bank->ngpio; pin++) { + ctrl_reg = ATLAS7_GPIO_CTRL(bank, pin); + writel(bank->sleep_data[pin], ctrl_reg); + } + } + + return 0; +} + +static const struct dev_pm_ops atlas7_gpio_pm_ops = { + .suspend_noirq = atlas7_gpio_suspend_noirq, + .resume_noirq = atlas7_gpio_resume_noirq, + .freeze_noirq = atlas7_gpio_suspend_noirq, + .restore_noirq = atlas7_gpio_resume_noirq, +}; +#endif + static struct platform_driver atlas7_gpio_driver = { .driver = { .name = "atlas7-gpio", - .owner = THIS_MODULE, .of_match_table = atlas7_gpio_ids, +#ifdef CONFIG_PM_SLEEP + .pm = &atlas7_gpio_pm_ops, +#endif }, .probe = atlas7_gpio_probe, }; diff --git a/drivers/pinctrl/sirf/pinctrl-sirf.c b/drivers/pinctrl/sirf/pinctrl-sirf.c index 8ba26e45499a35..f8bd9fb52033d7 100644 --- a/drivers/pinctrl/sirf/pinctrl-sirf.c +++ b/drivers/pinctrl/sirf/pinctrl-sirf.c @@ -545,14 +545,15 @@ static struct irq_chip sirfsoc_irq_chip = { .irq_set_type = sirfsoc_gpio_irq_type, }; -static void sirfsoc_gpio_handle_irq(unsigned int irq, struct irq_desc *desc) +static void sirfsoc_gpio_handle_irq(unsigned int __irq, struct irq_desc *desc) { + unsigned int irq = irq_desc_get_irq(desc); struct gpio_chip *gc = irq_desc_get_handler_data(desc); struct sirfsoc_gpio_chip *sgpio = to_sirfsoc_gpio(gc); struct sirfsoc_gpio_bank *bank; u32 status, ctrl; int idx = 0; - struct irq_chip *chip = irq_get_chip(irq); + struct irq_chip *chip = irq_desc_get_chip(desc); int i; for (i = 0; i < SIRFSOC_GPIO_NO_OF_BANKS; i++) { diff --git a/drivers/pinctrl/sunxi/pinctrl-sun4i-a10.c b/drivers/pinctrl/sunxi/pinctrl-sun4i-a10.c index 7376a97b5e65c3..862a096c5dbae3 100644 --- a/drivers/pinctrl/sunxi/pinctrl-sun4i-a10.c +++ b/drivers/pinctrl/sunxi/pinctrl-sun4i-a10.c @@ -135,7 +135,14 @@ static const struct sunxi_desc_pin sun4i_a10_pins[] = { SUNXI_PIN(SUNXI_PINCTRL_PIN(B, 3), SUNXI_FUNCTION(0x0, "gpio_in"), SUNXI_FUNCTION(0x1, "gpio_out"), - SUNXI_FUNCTION(0x2, "ir0")), /* TX */ + SUNXI_FUNCTION(0x2, "ir0"), /* TX */ + /* + * The SPDIF block is not referenced at all in the A10 user + * manual. However it is described in the code leaked and the + * pin descriptions are declared in the A20 user manual which + * is pin compatible with this device. + */ + SUNXI_FUNCTION(0x4, "spdif")), /* SPDIF MCLK */ SUNXI_PIN(SUNXI_PINCTRL_PIN(B, 4), SUNXI_FUNCTION(0x0, "gpio_in"), SUNXI_FUNCTION(0x1, "gpio_out"), @@ -176,11 +183,15 @@ static const struct sunxi_desc_pin sun4i_a10_pins[] = { SUNXI_FUNCTION(0x0, "gpio_in"), SUNXI_FUNCTION(0x1, "gpio_out"), SUNXI_FUNCTION(0x2, "i2s"), /* DI */ - SUNXI_FUNCTION(0x3, "ac97")), /* DI */ + SUNXI_FUNCTION(0x3, "ac97"), /* DI */ + /* Undocumented mux function - See SPDIF MCLK above */ + SUNXI_FUNCTION(0x4, "spdif")), /* SPDIF IN */ SUNXI_PIN(SUNXI_PINCTRL_PIN(B, 13), SUNXI_FUNCTION(0x0, "gpio_in"), SUNXI_FUNCTION(0x1, "gpio_out"), - SUNXI_FUNCTION(0x2, "spi2")), /* CS1 */ + SUNXI_FUNCTION(0x2, "spi2"), /* CS1 */ + /* Undocumented mux function - See SPDIF MCLK above */ + SUNXI_FUNCTION(0x4, "spdif")), /* SPDIF OUT */ SUNXI_PIN(SUNXI_PINCTRL_PIN(B, 14), SUNXI_FUNCTION(0x0, "gpio_in"), SUNXI_FUNCTION(0x1, "gpio_out"), diff --git a/drivers/pinctrl/sunxi/pinctrl-sunxi.c b/drivers/pinctrl/sunxi/pinctrl-sunxi.c index f09573e132035a..fb4669c0ce0e76 100644 --- a/drivers/pinctrl/sunxi/pinctrl-sunxi.c +++ b/drivers/pinctrl/sunxi/pinctrl-sunxi.c @@ -588,7 +588,6 @@ static void sunxi_pinctrl_irq_release_resources(struct irq_data *d) static int sunxi_pinctrl_irq_set_type(struct irq_data *d, unsigned int type) { struct sunxi_pinctrl *pctl = irq_data_get_irq_chip_data(d); - struct irq_desc *desc = container_of(d, struct irq_desc, irq_data); u32 reg = sunxi_irq_cfg_reg(d->hwirq); u8 index = sunxi_irq_cfg_offset(d->hwirq); unsigned long flags; @@ -615,16 +614,17 @@ static int sunxi_pinctrl_irq_set_type(struct irq_data *d, unsigned int type) return -EINVAL; } - if (type & IRQ_TYPE_LEVEL_MASK) { - d->chip = &sunxi_pinctrl_level_irq_chip; - desc->handle_irq = handle_fasteoi_irq; - } else { - d->chip = &sunxi_pinctrl_edge_irq_chip; - desc->handle_irq = handle_edge_irq; - } - spin_lock_irqsave(&pctl->lock, flags); + if (type & IRQ_TYPE_LEVEL_MASK) + __irq_set_chip_handler_name_locked(d->irq, + &sunxi_pinctrl_level_irq_chip, + handle_fasteoi_irq, NULL); + else + __irq_set_chip_handler_name_locked(d->irq, + &sunxi_pinctrl_edge_irq_chip, + handle_edge_irq, NULL); + regval = readl(pctl->membase + reg); regval &= ~(IRQ_CFG_IRQ_MASK << index); writel(regval | (mode << index), pctl->membase + reg); @@ -685,6 +685,7 @@ static void sunxi_pinctrl_irq_ack_unmask(struct irq_data *d) } static struct irq_chip sunxi_pinctrl_edge_irq_chip = { + .name = "sunxi_pio_edge", .irq_ack = sunxi_pinctrl_irq_ack, .irq_mask = sunxi_pinctrl_irq_mask, .irq_unmask = sunxi_pinctrl_irq_unmask, @@ -695,6 +696,7 @@ static struct irq_chip sunxi_pinctrl_edge_irq_chip = { }; static struct irq_chip sunxi_pinctrl_level_irq_chip = { + .name = "sunxi_pio_level", .irq_eoi = sunxi_pinctrl_irq_ack, .irq_mask = sunxi_pinctrl_irq_mask, .irq_unmask = sunxi_pinctrl_irq_unmask, @@ -709,10 +711,42 @@ static struct irq_chip sunxi_pinctrl_level_irq_chip = { IRQCHIP_EOI_IF_HANDLED, }; -static void sunxi_pinctrl_irq_handler(unsigned irq, struct irq_desc *desc) +static int sunxi_pinctrl_irq_of_xlate(struct irq_domain *d, + struct device_node *node, + const u32 *intspec, + unsigned int intsize, + unsigned long *out_hwirq, + unsigned int *out_type) +{ + struct sunxi_desc_function *desc; + int pin, base; + + if (intsize < 3) + return -EINVAL; + + base = PINS_PER_BANK * intspec[0]; + pin = base + intspec[1]; + + desc = sunxi_pinctrl_desc_find_function_by_pin(d->host_data, + pin, "irq"); + if (!desc) + return -EINVAL; + + *out_hwirq = desc->irqbank * PINS_PER_BANK + desc->irqnum; + *out_type = intspec[2]; + + return 0; +} + +static struct irq_domain_ops sunxi_pinctrl_irq_domain_ops = { + .xlate = sunxi_pinctrl_irq_of_xlate, +}; + +static void sunxi_pinctrl_irq_handler(unsigned __irq, struct irq_desc *desc) { - struct irq_chip *chip = irq_get_chip(irq); - struct sunxi_pinctrl *pctl = irq_get_handler_data(irq); + unsigned int irq = irq_desc_get_irq(desc); + struct irq_chip *chip = irq_desc_get_chip(desc); + struct sunxi_pinctrl *pctl = irq_desc_get_handler_data(desc); unsigned long bank, reg, val; for (bank = 0; bank < pctl->desc->irq_banks; bank++) @@ -983,8 +1017,8 @@ int sunxi_pinctrl_init(struct platform_device *pdev, pctl->domain = irq_domain_add_linear(node, pctl->desc->irq_banks * IRQ_PER_BANK, - &irq_domain_simple_ops, - NULL); + &sunxi_pinctrl_irq_domain_ops, + pctl); if (!pctl->domain) { dev_err(&pdev->dev, "Couldn't register IRQ domain\n"); ret = -ENOMEM; diff --git a/drivers/pinctrl/uniphier/Kconfig b/drivers/pinctrl/uniphier/Kconfig new file mode 100644 index 00000000000000..eab23ef9ddbf9a --- /dev/null +++ b/drivers/pinctrl/uniphier/Kconfig @@ -0,0 +1,32 @@ +if ARCH_UNIPHIER + +config PINCTRL_UNIPHIER_CORE + bool + select PINMUX + select GENERIC_PINCONF + +config PINCTRL_UNIPHIER_PH1_LD4 + tristate "UniPhier PH1-LD4 SoC pinctrl driver" + select PINCTRL_UNIPHIER_CORE + +config PINCTRL_UNIPHIER_PH1_PRO4 + tristate "UniPhier PH1-Pro4 SoC pinctrl driver" + select PINCTRL_UNIPHIER_CORE + +config PINCTRL_UNIPHIER_PH1_SLD8 + tristate "UniPhier PH1-sLD8 SoC pinctrl driver" + select PINCTRL_UNIPHIER_CORE + +config PINCTRL_UNIPHIER_PH1_PRO5 + tristate "UniPhier PH1-Pro5 SoC pinctrl driver" + select PINCTRL_UNIPHIER_CORE + +config PINCTRL_UNIPHIER_PROXSTREAM2 + tristate "UniPhier ProXstream2 SoC pinctrl driver" + select PINCTRL_UNIPHIER_CORE + +config PINCTRL_UNIPHIER_PH1_LD6B + tristate "UniPhier PH1-LD6b SoC pinctrl driver" + select PINCTRL_UNIPHIER_CORE + +endif diff --git a/drivers/pinctrl/uniphier/Makefile b/drivers/pinctrl/uniphier/Makefile new file mode 100644 index 00000000000000..e215b1097297a5 --- /dev/null +++ b/drivers/pinctrl/uniphier/Makefile @@ -0,0 +1,8 @@ +obj-$(CONFIG_PINCTRL_UNIPHIER_CORE) += pinctrl-uniphier-core.o + +obj-$(CONFIG_PINCTRL_UNIPHIER_PH1_LD4) += pinctrl-ph1-ld4.o +obj-$(CONFIG_PINCTRL_UNIPHIER_PH1_PRO4) += pinctrl-ph1-pro4.o +obj-$(CONFIG_PINCTRL_UNIPHIER_PH1_SLD8) += pinctrl-ph1-sld8.o +obj-$(CONFIG_PINCTRL_UNIPHIER_PH1_PRO5) += pinctrl-ph1-pro5.o +obj-$(CONFIG_PINCTRL_UNIPHIER_PROXSTREAM2) += pinctrl-proxstream2.o +obj-$(CONFIG_PINCTRL_UNIPHIER_PH1_LD6B) += pinctrl-ph1-ld6b.o diff --git a/drivers/pinctrl/uniphier/pinctrl-ph1-ld4.c b/drivers/pinctrl/uniphier/pinctrl-ph1-ld4.c new file mode 100644 index 00000000000000..7beb87e8f499af --- /dev/null +++ b/drivers/pinctrl/uniphier/pinctrl-ph1-ld4.c @@ -0,0 +1,886 @@ +/* + * Copyright (C) 2015 Masahiro Yamada + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include +#include +#include + +#include "pinctrl-uniphier.h" + +#define DRIVER_NAME "ph1-ld4-pinctrl" + +static const struct pinctrl_pin_desc ph1_ld4_pins[] = { + UNIPHIER_PINCTRL_PIN(0, "EA1", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_4_8, + 8, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(1, "EA2", UNIPHIER_PIN_IECTRL_NONE, + 9, UNIPHIER_PIN_DRV_4_8, + 9, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(2, "EA3", UNIPHIER_PIN_IECTRL_NONE, + 10, UNIPHIER_PIN_DRV_4_8, + 10, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(3, "EA4", UNIPHIER_PIN_IECTRL_NONE, + 11, UNIPHIER_PIN_DRV_4_8, + 11, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(4, "EA5", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_4_8, + 12, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(5, "EA6", UNIPHIER_PIN_IECTRL_NONE, + 13, UNIPHIER_PIN_DRV_4_8, + 13, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(6, "EA7", UNIPHIER_PIN_IECTRL_NONE, + 14, UNIPHIER_PIN_DRV_4_8, + 14, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(7, "EA8", 0, + 15, UNIPHIER_PIN_DRV_4_8, + 15, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(8, "EA9", 0, + 16, UNIPHIER_PIN_DRV_4_8, + 16, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(9, "EA10", 0, + 17, UNIPHIER_PIN_DRV_4_8, + 17, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(10, "EA11", 0, + 18, UNIPHIER_PIN_DRV_4_8, + 18, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(11, "EA12", 0, + 19, UNIPHIER_PIN_DRV_4_8, + 19, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(12, "EA13", 0, + 20, UNIPHIER_PIN_DRV_4_8, + 20, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(13, "EA14", 0, + 21, UNIPHIER_PIN_DRV_4_8, + 21, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(14, "EA15", 0, + 22, UNIPHIER_PIN_DRV_4_8, + 22, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(15, "ECLK", UNIPHIER_PIN_IECTRL_NONE, + 23, UNIPHIER_PIN_DRV_4_8, + 23, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(16, "XERWE0", UNIPHIER_PIN_IECTRL_NONE, + 24, UNIPHIER_PIN_DRV_4_8, + 24, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(17, "XERWE1", UNIPHIER_PIN_IECTRL_NONE, + 25, UNIPHIER_PIN_DRV_4_8, + 25, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(18, "ES0", UNIPHIER_PIN_IECTRL_NONE, + 27, UNIPHIER_PIN_DRV_4_8, + 27, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(19, "ES1", UNIPHIER_PIN_IECTRL_NONE, + 28, UNIPHIER_PIN_DRV_4_8, + 28, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(20, "ES2", UNIPHIER_PIN_IECTRL_NONE, + 29, UNIPHIER_PIN_DRV_4_8, + 29, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(21, "XERST", UNIPHIER_PIN_IECTRL_NONE, + 38, UNIPHIER_PIN_DRV_4_8, + 38, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(22, "MMCCLK", UNIPHIER_PIN_IECTRL_NONE, + 0, UNIPHIER_PIN_DRV_8_12_16_20, + 146, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(23, "MMCCMD", UNIPHIER_PIN_IECTRL_NONE, + 4, UNIPHIER_PIN_DRV_8_12_16_20, + 147, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(24, "MMCDAT0", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_8_12_16_20, + 148, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(25, "MMCDAT1", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_8_12_16_20, + 149, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(26, "MMCDAT2", UNIPHIER_PIN_IECTRL_NONE, + 16, UNIPHIER_PIN_DRV_8_12_16_20, + 150, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(27, "MMCDAT3", UNIPHIER_PIN_IECTRL_NONE, + 20, UNIPHIER_PIN_DRV_8_12_16_20, + 151, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(28, "MMCDAT4", UNIPHIER_PIN_IECTRL_NONE, + 24, UNIPHIER_PIN_DRV_8_12_16_20, + 152, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(29, "MMCDAT5", UNIPHIER_PIN_IECTRL_NONE, + 28, UNIPHIER_PIN_DRV_8_12_16_20, + 153, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(30, "MMCDAT6", UNIPHIER_PIN_IECTRL_NONE, + 32, UNIPHIER_PIN_DRV_8_12_16_20, + 154, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(31, "MMCDAT7", UNIPHIER_PIN_IECTRL_NONE, + 36, UNIPHIER_PIN_DRV_8_12_16_20, + 155, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(32, "RMII_RXD0", 6, + 39, UNIPHIER_PIN_DRV_4_8, + 39, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(33, "RMII_RXD1", 6, + 40, UNIPHIER_PIN_DRV_4_8, + 40, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(34, "RMII_CRS_DV", 6, + 41, UNIPHIER_PIN_DRV_4_8, + 41, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(35, "RMII_RXER", 6, + 42, UNIPHIER_PIN_DRV_4_8, + 42, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(36, "RMII_REFCLK", 6, + 43, UNIPHIER_PIN_DRV_4_8, + 43, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(37, "RMII_TXD0", 6, + 44, UNIPHIER_PIN_DRV_4_8, + 44, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(38, "RMII_TXD1", 6, + 45, UNIPHIER_PIN_DRV_4_8, + 45, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(39, "RMII_TXEN", 6, + 46, UNIPHIER_PIN_DRV_4_8, + 46, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(40, "MDC", 6, + 47, UNIPHIER_PIN_DRV_4_8, + 47, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(41, "MDIO", 6, + 48, UNIPHIER_PIN_DRV_4_8, + 48, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(42, "MDIO_INTL", 6, + 49, UNIPHIER_PIN_DRV_4_8, + 49, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(43, "PHYRSTL", 6, + 50, UNIPHIER_PIN_DRV_4_8, + 50, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(44, "SDCLK", UNIPHIER_PIN_IECTRL_NONE, + 40, UNIPHIER_PIN_DRV_8_12_16_20, + 156, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(45, "SDCMD", UNIPHIER_PIN_IECTRL_NONE, + 44, UNIPHIER_PIN_DRV_8_12_16_20, + 157, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(46, "SDDAT0", UNIPHIER_PIN_IECTRL_NONE, + 48, UNIPHIER_PIN_DRV_8_12_16_20, + 158, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(47, "SDDAT1", UNIPHIER_PIN_IECTRL_NONE, + 52, UNIPHIER_PIN_DRV_8_12_16_20, + 159, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(48, "SDDAT2", UNIPHIER_PIN_IECTRL_NONE, + 56, UNIPHIER_PIN_DRV_8_12_16_20, + 160, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(49, "SDDAT3", UNIPHIER_PIN_IECTRL_NONE, + 60, UNIPHIER_PIN_DRV_8_12_16_20, + 161, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(50, "SDCD", UNIPHIER_PIN_IECTRL_NONE, + 51, UNIPHIER_PIN_DRV_4_8, + 51, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(51, "SDWP", UNIPHIER_PIN_IECTRL_NONE, + 52, UNIPHIER_PIN_DRV_4_8, + 52, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(52, "SDVOLC", UNIPHIER_PIN_IECTRL_NONE, + 53, UNIPHIER_PIN_DRV_4_8, + 53, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(53, "USB0VBUS", 0, + 54, UNIPHIER_PIN_DRV_4_8, + 54, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(54, "USB0OD", 0, + 55, UNIPHIER_PIN_DRV_4_8, + 55, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(55, "USB1VBUS", 0, + 56, UNIPHIER_PIN_DRV_4_8, + 56, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(56, "USB1OD", 0, + 57, UNIPHIER_PIN_DRV_4_8, + 57, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(57, "PCRESET", 0, + 58, UNIPHIER_PIN_DRV_4_8, + 58, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(58, "PCREG", 0, + 59, UNIPHIER_PIN_DRV_4_8, + 59, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(59, "PCCE2", 0, + 60, UNIPHIER_PIN_DRV_4_8, + 60, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(60, "PCVS1", 0, + 61, UNIPHIER_PIN_DRV_4_8, + 61, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(61, "PCCD2", 0, + 62, UNIPHIER_PIN_DRV_4_8, + 62, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(62, "PCCD1", 0, + 63, UNIPHIER_PIN_DRV_4_8, + 63, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(63, "PCREADY", 0, + 64, UNIPHIER_PIN_DRV_4_8, + 64, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(64, "PCDOE", 0, + 65, UNIPHIER_PIN_DRV_4_8, + 65, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(65, "PCCE1", 0, + 66, UNIPHIER_PIN_DRV_4_8, + 66, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(66, "PCWE", 0, + 67, UNIPHIER_PIN_DRV_4_8, + 67, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(67, "PCOE", 0, + 68, UNIPHIER_PIN_DRV_4_8, + 68, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(68, "PCWAIT", 0, + 69, UNIPHIER_PIN_DRV_4_8, + 69, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(69, "PCIOWR", 0, + 70, UNIPHIER_PIN_DRV_4_8, + 70, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(70, "PCIORD", 0, + 71, UNIPHIER_PIN_DRV_4_8, + 71, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(71, "HS0DIN0", 0, + 72, UNIPHIER_PIN_DRV_4_8, + 72, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(72, "HS0DIN1", 0, + 73, UNIPHIER_PIN_DRV_4_8, + 73, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(73, "HS0DIN2", 0, + 74, UNIPHIER_PIN_DRV_4_8, + 74, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(74, "HS0DIN3", 0, + 75, UNIPHIER_PIN_DRV_4_8, + 75, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(75, "HS0DIN4", 0, + 76, UNIPHIER_PIN_DRV_4_8, + 76, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(76, "HS0DIN5", 0, + 77, UNIPHIER_PIN_DRV_4_8, + 77, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(77, "HS0DIN6", 0, + 78, UNIPHIER_PIN_DRV_4_8, + 78, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(78, "HS0DIN7", 0, + 79, UNIPHIER_PIN_DRV_4_8, + 79, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(79, "HS0BCLKIN", 0, + 80, UNIPHIER_PIN_DRV_4_8, + 80, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(80, "HS0VALIN", 0, + 81, UNIPHIER_PIN_DRV_4_8, + 81, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(81, "HS0SYNCIN", 0, + 82, UNIPHIER_PIN_DRV_4_8, + 82, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(82, "HSDOUT0", 0, + 83, UNIPHIER_PIN_DRV_4_8, + 83, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(83, "HSDOUT1", 0, + 84, UNIPHIER_PIN_DRV_4_8, + 84, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(84, "HSDOUT2", 0, + 85, UNIPHIER_PIN_DRV_4_8, + 85, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(85, "HSDOUT3", 0, + 86, UNIPHIER_PIN_DRV_4_8, + 86, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(86, "HSDOUT4", 0, + 87, UNIPHIER_PIN_DRV_4_8, + 87, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(87, "HSDOUT5", 0, + 88, UNIPHIER_PIN_DRV_4_8, + 88, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(88, "HSDOUT6", 0, + 89, UNIPHIER_PIN_DRV_4_8, + 89, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(89, "HSDOUT7", 0, + 90, UNIPHIER_PIN_DRV_4_8, + 90, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(90, "HSBCLKOUT", 0, + 91, UNIPHIER_PIN_DRV_4_8, + 91, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(91, "HSVALOUT", 0, + 92, UNIPHIER_PIN_DRV_4_8, + 92, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(92, "HSSYNCOUT", 0, + 93, UNIPHIER_PIN_DRV_4_8, + 93, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(93, "AGCI", 3, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 162, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(94, "AGCR", 4, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 163, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(95, "AGCBS", 5, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 164, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(96, "IECOUT", 0, + 94, UNIPHIER_PIN_DRV_4_8, + 94, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(97, "ASMCK", 0, + 95, UNIPHIER_PIN_DRV_4_8, + 95, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(98, "ABCKO", UNIPHIER_PIN_IECTRL_NONE, + 96, UNIPHIER_PIN_DRV_4_8, + 96, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(99, "ALRCKO", UNIPHIER_PIN_IECTRL_NONE, + 97, UNIPHIER_PIN_DRV_4_8, + 97, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(100, "ASDOUT0", UNIPHIER_PIN_IECTRL_NONE, + 98, UNIPHIER_PIN_DRV_4_8, + 98, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(101, "ARCOUT", 0, + 99, UNIPHIER_PIN_DRV_4_8, + 99, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(102, "SDA0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(103, "SCL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(104, "SDA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(105, "SCL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(106, "DMDSDA0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(107, "DMDSCL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(108, "DMDSDA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(109, "DMDSCL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(110, "SBO0", UNIPHIER_PIN_IECTRL_NONE, + 100, UNIPHIER_PIN_DRV_4_8, + 100, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(111, "SBI0", UNIPHIER_PIN_IECTRL_NONE, + 101, UNIPHIER_PIN_DRV_4_8, + 101, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(112, "HIN", 1, + -1, UNIPHIER_PIN_DRV_FIXED_5, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(113, "VIN", 2, + -1, UNIPHIER_PIN_DRV_FIXED_5, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(114, "TCON0", UNIPHIER_PIN_IECTRL_NONE, + 102, UNIPHIER_PIN_DRV_4_8, + 102, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(115, "TCON1", UNIPHIER_PIN_IECTRL_NONE, + 103, UNIPHIER_PIN_DRV_4_8, + 103, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(116, "TCON2", UNIPHIER_PIN_IECTRL_NONE, + 104, UNIPHIER_PIN_DRV_4_8, + 104, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(117, "TCON3", UNIPHIER_PIN_IECTRL_NONE, + 105, UNIPHIER_PIN_DRV_4_8, + 105, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(118, "TCON4", UNIPHIER_PIN_IECTRL_NONE, + 106, UNIPHIER_PIN_DRV_4_8, + 106, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(119, "TCON5", UNIPHIER_PIN_IECTRL_NONE, + 107, UNIPHIER_PIN_DRV_4_8, + 107, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(120, "TCON6", 0, + 108, UNIPHIER_PIN_DRV_4_8, + 108, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(121, "TCON7", 0, + 109, UNIPHIER_PIN_DRV_4_8, + 109, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(122, "PWMA", 0, + 110, UNIPHIER_PIN_DRV_4_8, + 110, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(123, "XIRQ1", 0, + 111, UNIPHIER_PIN_DRV_4_8, + 111, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(124, "XIRQ2", 0, + 112, UNIPHIER_PIN_DRV_4_8, + 112, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(125, "XIRQ3", 0, + 113, UNIPHIER_PIN_DRV_4_8, + 113, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(126, "XIRQ4", 0, + 114, UNIPHIER_PIN_DRV_4_8, + 114, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(127, "XIRQ5", 0, + 115, UNIPHIER_PIN_DRV_4_8, + 115, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(128, "XIRQ6", 0, + 116, UNIPHIER_PIN_DRV_4_8, + 116, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(129, "XIRQ7", 0, + 117, UNIPHIER_PIN_DRV_4_8, + 117, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(130, "XIRQ8", 0, + 118, UNIPHIER_PIN_DRV_4_8, + 118, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(131, "XIRQ9", 0, + 119, UNIPHIER_PIN_DRV_4_8, + 119, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(132, "XIRQ10", 0, + 120, UNIPHIER_PIN_DRV_4_8, + 120, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(133, "XIRQ11", 0, + 121, UNIPHIER_PIN_DRV_4_8, + 121, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(134, "XIRQ14", 0, + 122, UNIPHIER_PIN_DRV_4_8, + 122, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(135, "PORT00", 0, + 123, UNIPHIER_PIN_DRV_4_8, + 123, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(136, "PORT01", 0, + 124, UNIPHIER_PIN_DRV_4_8, + 124, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(137, "PORT02", 0, + 125, UNIPHIER_PIN_DRV_4_8, + 125, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(138, "PORT03", 0, + 126, UNIPHIER_PIN_DRV_4_8, + 126, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(139, "PORT04", 0, + 127, UNIPHIER_PIN_DRV_4_8, + 127, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(140, "PORT05", 0, + 128, UNIPHIER_PIN_DRV_4_8, + 128, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(141, "PORT06", 0, + 129, UNIPHIER_PIN_DRV_4_8, + 129, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(142, "PORT07", 0, + 130, UNIPHIER_PIN_DRV_4_8, + 130, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(143, "PORT10", 0, + 131, UNIPHIER_PIN_DRV_4_8, + 131, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(144, "PORT11", 0, + 132, UNIPHIER_PIN_DRV_4_8, + 132, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(145, "PORT12", 0, + 133, UNIPHIER_PIN_DRV_4_8, + 133, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(146, "PORT13", 0, + 134, UNIPHIER_PIN_DRV_4_8, + 134, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(147, "PORT14", 0, + 135, UNIPHIER_PIN_DRV_4_8, + 135, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(148, "PORT15", 0, + 136, UNIPHIER_PIN_DRV_4_8, + 136, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(149, "PORT16", 0, + 137, UNIPHIER_PIN_DRV_4_8, + 137, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(150, "PORT17", UNIPHIER_PIN_IECTRL_NONE, + 138, UNIPHIER_PIN_DRV_4_8, + 138, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(151, "PORT20", 0, + 139, UNIPHIER_PIN_DRV_4_8, + 139, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(152, "PORT21", 0, + 140, UNIPHIER_PIN_DRV_4_8, + 140, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(153, "PORT22", 0, + 141, UNIPHIER_PIN_DRV_4_8, + 141, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(154, "PORT23", 0, + 142, UNIPHIER_PIN_DRV_4_8, + 142, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(155, "PORT24", UNIPHIER_PIN_IECTRL_NONE, + 143, UNIPHIER_PIN_DRV_4_8, + 143, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(156, "PORT25", 0, + 144, UNIPHIER_PIN_DRV_4_8, + 144, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(157, "PORT26", 0, + 145, UNIPHIER_PIN_DRV_4_8, + 145, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(158, "XNFRE", UNIPHIER_PIN_IECTRL_NONE, + 31, UNIPHIER_PIN_DRV_4_8, + 31, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(159, "XNFWE", UNIPHIER_PIN_IECTRL_NONE, + 32, UNIPHIER_PIN_DRV_4_8, + 32, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(160, "NFALE", UNIPHIER_PIN_IECTRL_NONE, + 33, UNIPHIER_PIN_DRV_4_8, + 33, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(161, "NFCLE", UNIPHIER_PIN_IECTRL_NONE, + 34, UNIPHIER_PIN_DRV_4_8, + 34, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(162, "XNFWP", UNIPHIER_PIN_IECTRL_NONE, + 35, UNIPHIER_PIN_DRV_4_8, + 35, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(163, "XNFCE0", UNIPHIER_PIN_IECTRL_NONE, + 36, UNIPHIER_PIN_DRV_4_8, + 36, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(164, "NANDRYBY0", UNIPHIER_PIN_IECTRL_NONE, + 37, UNIPHIER_PIN_DRV_4_8, + 37, UNIPHIER_PIN_PULL_UP), +}; + +static const unsigned emmc_pins[] = {21, 22, 23, 24, 25, 26, 27}; +static const unsigned emmc_muxvals[] = {0, 1, 1, 1, 1, 1, 1}; +static const unsigned emmc_dat8_pins[] = {28, 29, 30, 31}; +static const unsigned emmc_dat8_muxvals[] = {1, 1, 1, 1}; +static const unsigned i2c0_pins[] = {102, 103}; +static const unsigned i2c0_muxvals[] = {0, 0}; +static const unsigned i2c1_pins[] = {104, 105}; +static const unsigned i2c1_muxvals[] = {0, 0}; +static const unsigned i2c2_pins[] = {108, 109}; +static const unsigned i2c2_muxvals[] = {2, 2}; +static const unsigned i2c3_pins[] = {108, 109}; +static const unsigned i2c3_muxvals[] = {3, 3}; +static const unsigned nand_pins[] = {24, 25, 26, 27, 28, 29, 30, 31, 158, 159, + 160, 161, 162, 163, 164}; +static const unsigned nand_muxvals[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0}; +static const unsigned nand_cs1_pins[] = {22, 23}; +static const unsigned nand_cs1_muxvals[] = {0, 0}; +static const unsigned uart0_pins[] = {85, 88}; +static const unsigned uart0_muxvals[] = {1, 1}; +static const unsigned uart1_pins[] = {155, 156}; +static const unsigned uart1_muxvals[] = {13, 13}; +static const unsigned uart1b_pins[] = {69, 70}; +static const unsigned uart1b_muxvals[] = {23, 23}; +static const unsigned uart2_pins[] = {128, 129}; +static const unsigned uart2_muxvals[] = {13, 13}; +static const unsigned uart3_pins[] = {110, 111}; +static const unsigned uart3_muxvals[] = {1, 1}; +static const unsigned usb0_pins[] = {53, 54}; +static const unsigned usb0_muxvals[] = {0, 0}; +static const unsigned usb1_pins[] = {55, 56}; +static const unsigned usb1_muxvals[] = {0, 0}; +static const unsigned usb2_pins[] = {155, 156}; +static const unsigned usb2_muxvals[] = {4, 4}; +static const unsigned usb2b_pins[] = {67, 68}; +static const unsigned usb2b_muxvals[] = {23, 23}; +static const unsigned port_range0_pins[] = { + 135, 136, 137, 138, 139, 140, 141, 142, /* PORT0x */ + 143, 144, 145, 146, 147, 148, 149, 150, /* PORT1x */ + 151, 152, 153, 154, 155, 156, 157, 0, /* PORT2x */ + 1, 2, 3, 4, 5, 120, 121, 122, /* PORT3x */ + 24, 25, 26, 27, 28, 29, 30, 31, /* PORT4x */ + 40, 41, 42, 43, 44, 45, 46, 47, /* PORT5x */ + 48, 49, 50, 51, 52, 53, 54, 55, /* PORT6x */ + 56, 85, 84, 59, 82, 61, 64, 65, /* PORT7x */ + 8, 9, 10, 11, 12, 13, 14, 15, /* PORT8x */ + 66, 67, 68, 69, 70, 71, 72, 73, /* PORT9x */ + 74, 75, 89, 86, 78, 79, 80, 81, /* PORT10x */ + 60, 83, 58, 57, 88, 87, 77, 76, /* PORT11x */ + 90, 91, 92, 93, 94, 95, 96, 97, /* PORT12x */ + 98, 99, 100, 6, 101, 114, 115, 116, /* PORT13x */ + 103, 108, 21, 22, 23, 117, 118, 119, /* PORT14x */ +}; +static const unsigned port_range0_muxvals[] = { + 0, 0, 0, 0, 0, 0, 0, 0, /* PORT0x */ + 0, 0, 0, 0, 0, 0, 0, 0, /* PORT1x */ + 0, 0, 0, 0, 0, 0, 0, 15, /* PORT2x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT3x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT4x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT5x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT6x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT7x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT8x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT9x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT10x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT11x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT12x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT13x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT14x */ +}; +static const unsigned port_range1_pins[] = { + 7, /* PORT166 */ +}; +static const unsigned port_range1_muxvals[] = { + 15, /* PORT166 */ +}; +static const unsigned xirq_range0_pins[] = { + 151, 123, 124, 125, 126, 127, 128, 129, /* XIRQ0-7 */ + 130, 131, 132, 133, 62, /* XIRQ8-12 */ +}; +static const unsigned xirq_range0_muxvals[] = { + 14, 0, 0, 0, 0, 0, 0, 0, /* XIRQ0-7 */ + 0, 0, 0, 0, 14, /* XIRQ8-12 */ +}; +static const unsigned xirq_range1_pins[] = { + 134, 63, /* XIRQ14-15 */ +}; +static const unsigned xirq_range1_muxvals[] = { + 0, 14, /* XIRQ14-15 */ +}; + +static const struct uniphier_pinctrl_group ph1_ld4_groups[] = { + UNIPHIER_PINCTRL_GROUP(emmc), + UNIPHIER_PINCTRL_GROUP(emmc_dat8), + UNIPHIER_PINCTRL_GROUP(i2c0), + UNIPHIER_PINCTRL_GROUP(i2c1), + UNIPHIER_PINCTRL_GROUP(i2c2), + UNIPHIER_PINCTRL_GROUP(i2c3), + UNIPHIER_PINCTRL_GROUP(nand), + UNIPHIER_PINCTRL_GROUP(nand_cs1), + UNIPHIER_PINCTRL_GROUP(uart0), + UNIPHIER_PINCTRL_GROUP(uart1), + UNIPHIER_PINCTRL_GROUP(uart1b), + UNIPHIER_PINCTRL_GROUP(uart2), + UNIPHIER_PINCTRL_GROUP(uart3), + UNIPHIER_PINCTRL_GROUP(usb0), + UNIPHIER_PINCTRL_GROUP(usb1), + UNIPHIER_PINCTRL_GROUP(usb2), + UNIPHIER_PINCTRL_GROUP(usb2b), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range0), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range1), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq_range0), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq_range1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port00, port_range0, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port01, port_range0, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port02, port_range0, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port03, port_range0, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port04, port_range0, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port05, port_range0, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port06, port_range0, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port07, port_range0, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port10, port_range0, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port11, port_range0, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port12, port_range0, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port13, port_range0, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port14, port_range0, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port15, port_range0, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port16, port_range0, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port17, port_range0, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port20, port_range0, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port21, port_range0, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port22, port_range0, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port23, port_range0, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port24, port_range0, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port25, port_range0, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port26, port_range0, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port27, port_range0, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port30, port_range0, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port31, port_range0, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port32, port_range0, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port33, port_range0, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port34, port_range0, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port35, port_range0, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port36, port_range0, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port37, port_range0, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port40, port_range0, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port41, port_range0, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port42, port_range0, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port43, port_range0, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port44, port_range0, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port45, port_range0, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port46, port_range0, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port47, port_range0, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port50, port_range0, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port51, port_range0, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port52, port_range0, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port53, port_range0, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port54, port_range0, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port55, port_range0, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port56, port_range0, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port57, port_range0, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port60, port_range0, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port61, port_range0, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port62, port_range0, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port63, port_range0, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port64, port_range0, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port65, port_range0, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port66, port_range0, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port67, port_range0, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port70, port_range0, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port71, port_range0, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port72, port_range0, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port73, port_range0, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port74, port_range0, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port75, port_range0, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port76, port_range0, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port77, port_range0, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port80, port_range0, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port81, port_range0, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port82, port_range0, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port83, port_range0, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port84, port_range0, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port85, port_range0, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port86, port_range0, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port87, port_range0, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port90, port_range0, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port91, port_range0, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port92, port_range0, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port93, port_range0, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port94, port_range0, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port95, port_range0, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port96, port_range0, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port97, port_range0, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port100, port_range0, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port101, port_range0, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port102, port_range0, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port103, port_range0, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port104, port_range0, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port105, port_range0, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port106, port_range0, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port107, port_range0, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port110, port_range0, 88), + UNIPHIER_PINCTRL_GROUP_SINGLE(port111, port_range0, 89), + UNIPHIER_PINCTRL_GROUP_SINGLE(port112, port_range0, 90), + UNIPHIER_PINCTRL_GROUP_SINGLE(port113, port_range0, 91), + UNIPHIER_PINCTRL_GROUP_SINGLE(port114, port_range0, 92), + UNIPHIER_PINCTRL_GROUP_SINGLE(port115, port_range0, 93), + UNIPHIER_PINCTRL_GROUP_SINGLE(port116, port_range0, 94), + UNIPHIER_PINCTRL_GROUP_SINGLE(port117, port_range0, 95), + UNIPHIER_PINCTRL_GROUP_SINGLE(port120, port_range0, 96), + UNIPHIER_PINCTRL_GROUP_SINGLE(port121, port_range0, 97), + UNIPHIER_PINCTRL_GROUP_SINGLE(port122, port_range0, 98), + UNIPHIER_PINCTRL_GROUP_SINGLE(port123, port_range0, 99), + UNIPHIER_PINCTRL_GROUP_SINGLE(port124, port_range0, 100), + UNIPHIER_PINCTRL_GROUP_SINGLE(port125, port_range0, 101), + UNIPHIER_PINCTRL_GROUP_SINGLE(port126, port_range0, 102), + UNIPHIER_PINCTRL_GROUP_SINGLE(port127, port_range0, 103), + UNIPHIER_PINCTRL_GROUP_SINGLE(port130, port_range0, 104), + UNIPHIER_PINCTRL_GROUP_SINGLE(port131, port_range0, 105), + UNIPHIER_PINCTRL_GROUP_SINGLE(port132, port_range0, 106), + UNIPHIER_PINCTRL_GROUP_SINGLE(port133, port_range0, 107), + UNIPHIER_PINCTRL_GROUP_SINGLE(port134, port_range0, 108), + UNIPHIER_PINCTRL_GROUP_SINGLE(port135, port_range0, 109), + UNIPHIER_PINCTRL_GROUP_SINGLE(port136, port_range0, 110), + UNIPHIER_PINCTRL_GROUP_SINGLE(port137, port_range0, 111), + UNIPHIER_PINCTRL_GROUP_SINGLE(port140, port_range0, 112), + UNIPHIER_PINCTRL_GROUP_SINGLE(port141, port_range0, 113), + UNIPHIER_PINCTRL_GROUP_SINGLE(port142, port_range0, 114), + UNIPHIER_PINCTRL_GROUP_SINGLE(port143, port_range0, 115), + UNIPHIER_PINCTRL_GROUP_SINGLE(port144, port_range0, 116), + UNIPHIER_PINCTRL_GROUP_SINGLE(port145, port_range0, 117), + UNIPHIER_PINCTRL_GROUP_SINGLE(port146, port_range0, 118), + UNIPHIER_PINCTRL_GROUP_SINGLE(port147, port_range0, 119), + UNIPHIER_PINCTRL_GROUP_SINGLE(port165, port_range1, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq0, xirq_range0, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq1, xirq_range0, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq2, xirq_range0, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq3, xirq_range0, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq4, xirq_range0, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq5, xirq_range0, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq6, xirq_range0, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq7, xirq_range0, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq8, xirq_range0, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq9, xirq_range0, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq10, xirq_range0, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq11, xirq_range0, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq12, xirq_range0, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq14, xirq_range1, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq15, xirq_range1, 1), +}; + +static const char * const emmc_groups[] = {"emmc", "emmc_dat8"}; +static const char * const i2c0_groups[] = {"i2c0"}; +static const char * const i2c1_groups[] = {"i2c1"}; +static const char * const i2c2_groups[] = {"i2c2"}; +static const char * const i2c3_groups[] = {"i2c3"}; +static const char * const nand_groups[] = {"nand", "nand_cs1"}; +static const char * const uart0_groups[] = {"uart0"}; +static const char * const uart1_groups[] = {"uart1", "uart1b"}; +static const char * const uart2_groups[] = {"uart2"}; +static const char * const uart3_groups[] = {"uart3"}; +static const char * const usb0_groups[] = {"usb0"}; +static const char * const usb1_groups[] = {"usb1"}; +static const char * const usb2_groups[] = {"usb2", "usb2b"}; +static const char * const port_groups[] = { + "port00", "port01", "port02", "port03", + "port04", "port05", "port06", "port07", + "port10", "port11", "port12", "port13", + "port14", "port15", "port16", "port17", + "port20", "port21", "port22", "port23", + "port24", "port25", "port26", "port27", + "port30", "port31", "port32", "port33", + "port34", "port35", "port36", "port37", + "port40", "port41", "port42", "port43", + "port44", "port45", "port46", "port47", + "port50", "port51", "port52", "port53", + "port54", "port55", "port56", "port57", + "port60", "port61", "port62", "port63", + "port64", "port65", "port66", "port67", + "port70", "port71", "port72", "port73", + "port74", "port75", "port76", "port77", + "port80", "port81", "port82", "port83", + "port84", "port85", "port86", "port87", + "port90", "port91", "port92", "port93", + "port94", "port95", "port96", "port97", + "port100", "port101", "port102", "port103", + "port104", "port105", "port106", "port107", + "port110", "port111", "port112", "port113", + "port114", "port115", "port116", "port117", + "port120", "port121", "port122", "port123", + "port124", "port125", "port126", "port127", + "port130", "port131", "port132", "port133", + "port134", "port135", "port136", "port137", + "port140", "port141", "port142", "port143", + "port144", "port145", "port146", "port147", + /* port150-164 missing */ + /* none */ "port165", +}; +static const char * const xirq_groups[] = { + "xirq0", "xirq1", "xirq2", "xirq3", + "xirq4", "xirq5", "xirq6", "xirq7", + "xirq8", "xirq9", "xirq10", "xirq11", + "xirq12", /* none*/ "xirq14", "xirq15", +}; + +static const struct uniphier_pinmux_function ph1_ld4_functions[] = { + UNIPHIER_PINMUX_FUNCTION(emmc), + UNIPHIER_PINMUX_FUNCTION(i2c0), + UNIPHIER_PINMUX_FUNCTION(i2c1), + UNIPHIER_PINMUX_FUNCTION(i2c2), + UNIPHIER_PINMUX_FUNCTION(i2c3), + UNIPHIER_PINMUX_FUNCTION(nand), + UNIPHIER_PINMUX_FUNCTION(uart0), + UNIPHIER_PINMUX_FUNCTION(uart1), + UNIPHIER_PINMUX_FUNCTION(uart2), + UNIPHIER_PINMUX_FUNCTION(uart3), + UNIPHIER_PINMUX_FUNCTION(usb0), + UNIPHIER_PINMUX_FUNCTION(usb1), + UNIPHIER_PINMUX_FUNCTION(usb2), + UNIPHIER_PINMUX_FUNCTION(port), + UNIPHIER_PINMUX_FUNCTION(xirq), +}; + +static struct uniphier_pinctrl_socdata ph1_ld4_pindata = { + .groups = ph1_ld4_groups, + .groups_count = ARRAY_SIZE(ph1_ld4_groups), + .functions = ph1_ld4_functions, + .functions_count = ARRAY_SIZE(ph1_ld4_functions), + .mux_bits = 8, + .reg_stride = 4, + .load_pinctrl = false, +}; + +static struct pinctrl_desc ph1_ld4_pinctrl_desc = { + .name = DRIVER_NAME, + .pins = ph1_ld4_pins, + .npins = ARRAY_SIZE(ph1_ld4_pins), + .owner = THIS_MODULE, +}; + +static int ph1_ld4_pinctrl_probe(struct platform_device *pdev) +{ + return uniphier_pinctrl_probe(pdev, &ph1_ld4_pinctrl_desc, + &ph1_ld4_pindata); +} + +static const struct of_device_id ph1_ld4_pinctrl_match[] = { + { .compatible = "socionext,ph1-ld4-pinctrl" }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, ph1_ld4_pinctrl_match); + +static struct platform_driver ph1_ld4_pinctrl_driver = { + .probe = ph1_ld4_pinctrl_probe, + .remove = uniphier_pinctrl_remove, + .driver = { + .name = DRIVER_NAME, + .of_match_table = ph1_ld4_pinctrl_match, + }, +}; +module_platform_driver(ph1_ld4_pinctrl_driver); + +MODULE_AUTHOR("Masahiro Yamada "); +MODULE_DESCRIPTION("UniPhier PH1-LD4 pinctrl driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pinctrl/uniphier/pinctrl-ph1-ld6b.c b/drivers/pinctrl/uniphier/pinctrl-ph1-ld6b.c new file mode 100644 index 00000000000000..9720e697fbc1f9 --- /dev/null +++ b/drivers/pinctrl/uniphier/pinctrl-ph1-ld6b.c @@ -0,0 +1,1274 @@ +/* + * Copyright (C) 2015 Masahiro Yamada + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include +#include +#include + +#include "pinctrl-uniphier.h" + +#define DRIVER_NAME "ph1-ld6b-pinctrl" + +static const struct pinctrl_pin_desc ph1_ld6b_pins[] = { + UNIPHIER_PINCTRL_PIN(0, "ED0", UNIPHIER_PIN_IECTRL_NONE, + 0, UNIPHIER_PIN_DRV_4_8, + 0, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(1, "ED1", UNIPHIER_PIN_IECTRL_NONE, + 1, UNIPHIER_PIN_DRV_4_8, + 1, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(2, "ED2", UNIPHIER_PIN_IECTRL_NONE, + 2, UNIPHIER_PIN_DRV_4_8, + 2, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(3, "ED3", UNIPHIER_PIN_IECTRL_NONE, + 3, UNIPHIER_PIN_DRV_4_8, + 3, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(4, "ED4", UNIPHIER_PIN_IECTRL_NONE, + 4, UNIPHIER_PIN_DRV_4_8, + 4, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(5, "ED5", UNIPHIER_PIN_IECTRL_NONE, + 5, UNIPHIER_PIN_DRV_4_8, + 5, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(6, "ED6", UNIPHIER_PIN_IECTRL_NONE, + 6, UNIPHIER_PIN_DRV_4_8, + 6, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(7, "ED7", UNIPHIER_PIN_IECTRL_NONE, + 7, UNIPHIER_PIN_DRV_4_8, + 7, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(8, "XERWE0", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_4_8, + 8, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(9, "XERWE1", UNIPHIER_PIN_IECTRL_NONE, + 9, UNIPHIER_PIN_DRV_4_8, + 9, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(10, "ERXW", UNIPHIER_PIN_IECTRL_NONE, + 10, UNIPHIER_PIN_DRV_4_8, + 10, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(11, "ES0", UNIPHIER_PIN_IECTRL_NONE, + 11, UNIPHIER_PIN_DRV_4_8, + 11, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(12, "ES1", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_4_8, + 12, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(13, "ES2", UNIPHIER_PIN_IECTRL_NONE, + 13, UNIPHIER_PIN_DRV_4_8, + 13, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(14, "XECS1", UNIPHIER_PIN_IECTRL_NONE, + 14, UNIPHIER_PIN_DRV_4_8, + 14, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(15, "PCA00", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 15, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(16, "PCA01", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 16, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(17, "PCA02", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 17, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(18, "PCA03", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 18, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(19, "PCA04", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 19, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(20, "PCA05", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 20, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(21, "PCA06", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 21, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(22, "PCA07", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 22, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(23, "PCA08", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 23, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(24, "PCA09", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 24, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(25, "PCA10", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 25, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(26, "PCA11", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 26, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(27, "PCA12", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 27, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(28, "PCA13", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 28, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(29, "PCA14", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 29, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(30, "XNFRE", UNIPHIER_PIN_IECTRL_NONE, + 30, UNIPHIER_PIN_DRV_4_8, + 30, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(31, "XNFWE", UNIPHIER_PIN_IECTRL_NONE, + 31, UNIPHIER_PIN_DRV_4_8, + 31, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(32, "NFALE", UNIPHIER_PIN_IECTRL_NONE, + 32, UNIPHIER_PIN_DRV_4_8, + 32, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(33, "NFCLE", UNIPHIER_PIN_IECTRL_NONE, + 33, UNIPHIER_PIN_DRV_4_8, + 33, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(34, "XNFWP", UNIPHIER_PIN_IECTRL_NONE, + 34, UNIPHIER_PIN_DRV_4_8, + 34, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(35, "XNFCE0", UNIPHIER_PIN_IECTRL_NONE, + 35, UNIPHIER_PIN_DRV_4_8, + 35, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(36, "NFRYBY0", UNIPHIER_PIN_IECTRL_NONE, + 36, UNIPHIER_PIN_DRV_4_8, + 36, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(37, "XNFCE1", UNIPHIER_PIN_IECTRL_NONE, + 37, UNIPHIER_PIN_DRV_4_8, + 37, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(38, "NFRYBY1", UNIPHIER_PIN_IECTRL_NONE, + 38, UNIPHIER_PIN_DRV_4_8, + 38, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(39, "NFD0", UNIPHIER_PIN_IECTRL_NONE, + 39, UNIPHIER_PIN_DRV_4_8, + 39, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(40, "NFD1", UNIPHIER_PIN_IECTRL_NONE, + 40, UNIPHIER_PIN_DRV_4_8, + 40, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(41, "NFD2", UNIPHIER_PIN_IECTRL_NONE, + 41, UNIPHIER_PIN_DRV_4_8, + 41, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(42, "NFD3", UNIPHIER_PIN_IECTRL_NONE, + 42, UNIPHIER_PIN_DRV_4_8, + 42, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(43, "NFD4", UNIPHIER_PIN_IECTRL_NONE, + 43, UNIPHIER_PIN_DRV_4_8, + 43, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(44, "NFD5", UNIPHIER_PIN_IECTRL_NONE, + 44, UNIPHIER_PIN_DRV_4_8, + 44, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(45, "NFD6", UNIPHIER_PIN_IECTRL_NONE, + 45, UNIPHIER_PIN_DRV_4_8, + 45, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(46, "NFD7", UNIPHIER_PIN_IECTRL_NONE, + 46, UNIPHIER_PIN_DRV_4_8, + 46, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(47, "SDCLK", UNIPHIER_PIN_IECTRL_NONE, + 0, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(48, "SDCMD", UNIPHIER_PIN_IECTRL_NONE, + 4, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(49, "SDDAT0", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(50, "SDDAT1", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(51, "SDDAT2", UNIPHIER_PIN_IECTRL_NONE, + 16, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(52, "SDDAT3", UNIPHIER_PIN_IECTRL_NONE, + 20, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(53, "SDCD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 53, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(54, "SDWP", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 54, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(55, "SDVOLC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 55, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(56, "USB0VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 56, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(57, "USB0OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 57, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(58, "USB1VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 58, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(59, "USB1OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 59, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(60, "USB2VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 60, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(61, "USB2OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 61, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(62, "USB3VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 62, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(63, "USB3OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 63, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(64, "HS0BCLKOUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 64, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(65, "HS0SYNCOUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 65, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(66, "HS0VALOUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 66, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(67, "HS0DOUT0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 67, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(68, "HS0DOUT1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 68, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(69, "HS0DOUT2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 69, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(70, "HS0DOUT3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 70, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(71, "HS0DOUT4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 71, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(72, "HS0DOUT5", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 72, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(73, "HS0DOUT6", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 73, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(74, "HS0DOUT7", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 74, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(75, "HS1BCLKIN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 75, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(76, "HS1SYNCIN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 76, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(77, "HS1VALIN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 77, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(78, "HS1DIN0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 78, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(79, "HS1DIN1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 79, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(80, "HS1DIN2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 80, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(81, "HS1DIN3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 81, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(82, "HS1DIN4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 82, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(83, "HS1DIN5", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 83, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(84, "HS1DIN6", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 84, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(85, "HS1DIN7", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 85, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(86, "HS2BCLKIN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 86, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(87, "HS2SYNCIN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 87, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(88, "HS2VALIN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 88, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(89, "HS2DIN0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 89, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(90, "HS2DIN1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 90, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(91, "HS2DIN2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 91, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(92, "HS2DIN3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 92, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(93, "HS2DIN4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 93, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(94, "HS2DIN5", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 94, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(95, "HS2DIN6", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 95, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(96, "HS2DIN7", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 96, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(97, "AO1IEC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 97, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(98, "AO1DACCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 98, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(99, "AO1BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 99, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(100, "AO1LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 100, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(101, "AO1D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 101, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(102, "AO1D1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 102, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(103, "AO1D2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 103, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(104, "AO1D3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 104, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(105, "AO2DACCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 105, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(106, "AO2BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 106, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(107, "AO2LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 107, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(108, "AO2D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 108, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(109, "SDA0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 109, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(110, "SCL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 110, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(111, "SDA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 111, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(112, "SCL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 112, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(113, "SBO0", 0, + 113, UNIPHIER_PIN_DRV_4_8, + 113, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(114, "SBI0", 0, + 114, UNIPHIER_PIN_DRV_4_8, + 114, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(115, "TXD1", 0, + 115, UNIPHIER_PIN_DRV_4_8, + 115, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(116, "RXD1", 0, + 116, UNIPHIER_PIN_DRV_4_8, + 116, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(117, "PWSRA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 117, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(118, "XIRQ0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 118, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(119, "XIRQ1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 119, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(120, "XIRQ2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 120, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(121, "XIRQ3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 121, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(122, "XIRQ4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 122, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(123, "XIRQ5", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 123, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(124, "XIRQ6", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 124, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(125, "XIRQ7", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 125, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(126, "XIRQ8", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 126, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(127, "PORT00", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 127, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(128, "PORT01", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 128, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(129, "PORT02", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 129, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(130, "PORT03", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 130, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(131, "PORT04", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 131, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(132, "PORT05", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 132, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(133, "PORT06", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 133, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(134, "PORT07", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 134, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(135, "PORT10", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 135, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(136, "PORT11", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 136, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(137, "PORT12", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 137, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(138, "PORT13", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 138, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(139, "PORT14", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 139, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(140, "PORT15", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 140, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(141, "PORT16", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 141, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(142, "LPST", UNIPHIER_PIN_IECTRL_NONE, + 142, UNIPHIER_PIN_DRV_4_8, + 142, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(143, "MDC", 0, + 143, UNIPHIER_PIN_DRV_4_8, + 143, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(144, "MDIO", 0, + 144, UNIPHIER_PIN_DRV_4_8, + 144, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(145, "MDIO_INTL", 0, + 145, UNIPHIER_PIN_DRV_4_8, + 145, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(146, "PHYRSTL", 0, + 146, UNIPHIER_PIN_DRV_4_8, + 146, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(147, "RGMII_RXCLK", 0, + 147, UNIPHIER_PIN_DRV_4_8, + 147, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(148, "RGMII_RXD0", 0, + 148, UNIPHIER_PIN_DRV_4_8, + 148, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(149, "RGMII_RXD1", 0, + 149, UNIPHIER_PIN_DRV_4_8, + 149, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(150, "RGMII_RXD2", 0, + 150, UNIPHIER_PIN_DRV_4_8, + 150, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(151, "RGMII_RXD3", 0, + 151, UNIPHIER_PIN_DRV_4_8, + 151, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(152, "RGMII_RXCTL", 0, + 152, UNIPHIER_PIN_DRV_4_8, + 152, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(153, "RGMII_TXCLK", 0, + 153, UNIPHIER_PIN_DRV_4_8, + 153, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(154, "RGMII_TXD0", 0, + 154, UNIPHIER_PIN_DRV_4_8, + 154, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(155, "RGMII_TXD1", 0, + 155, UNIPHIER_PIN_DRV_4_8, + 155, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(156, "RGMII_TXD2", 0, + 156, UNIPHIER_PIN_DRV_4_8, + 156, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(157, "RGMII_TXD3", 0, + 157, UNIPHIER_PIN_DRV_4_8, + 157, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(158, "RGMII_TXCTL", 0, + 158, UNIPHIER_PIN_DRV_4_8, + 158, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(159, "A_D_PCD00OUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 159, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(160, "A_D_PCD01OUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 160, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(161, "A_D_PCD02OUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 161, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(162, "A_D_PCD03OUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 162, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(163, "A_D_PCD04OUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 163, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(164, "A_D_PCD05OUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 164, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(165, "A_D_PCD06OUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 165, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(166, "A_D_PCD07OUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 166, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(167, "A_D_PCD00IN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 167, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(168, "A_D_PCD01IN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 168, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(169, "A_D_PCD02IN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 169, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(170, "A_D_PCD03IN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 170, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(171, "A_D_PCD04IN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 171, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(172, "A_D_PCD05IN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 172, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(173, "A_D_PCD06IN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 173, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(174, "A_D_PCD07IN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 174, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(175, "A_D_PCDNOE", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 175, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(176, "A_D_PC0READY", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 176, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(177, "A_D_PC0CD1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 177, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(178, "A_D_PC0CD2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 178, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(179, "A_D_PC0WAIT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 179, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(180, "A_D_PC0RESET", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 180, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(181, "A_D_PC0CE1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 181, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(182, "A_D_PC0WE", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 182, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(183, "A_D_PC0OE", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 183, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(184, "A_D_PC0IOWR", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 184, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(185, "A_D_PC0IORD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 185, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(186, "A_D_PC0NOE", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 186, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(187, "A_D_HS0BCLKIN", 0, + 187, UNIPHIER_PIN_DRV_4_8, + 187, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(188, "A_D_HS0SYNCIN", 0, + 188, UNIPHIER_PIN_DRV_4_8, + 188, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(189, "A_D_HS0VALIN", 0, + 189, UNIPHIER_PIN_DRV_4_8, + 189, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(190, "A_D_HS0DIN0", 0, + 190, UNIPHIER_PIN_DRV_4_8, + 190, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(191, "A_D_HS0DIN1", 0, + 191, UNIPHIER_PIN_DRV_4_8, + 191, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(192, "A_D_HS0DIN2", 0, + 192, UNIPHIER_PIN_DRV_4_8, + 192, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(193, "A_D_HS0DIN3", 0, + 193, UNIPHIER_PIN_DRV_4_8, + 193, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(194, "A_D_HS0DIN4", 0, + 194, UNIPHIER_PIN_DRV_4_8, + 194, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(195, "A_D_HS0DIN5", 0, + 195, UNIPHIER_PIN_DRV_4_8, + 195, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(196, "A_D_HS0DIN6", 0, + 196, UNIPHIER_PIN_DRV_4_8, + 196, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(197, "A_D_HS0DIN7", 0, + 197, UNIPHIER_PIN_DRV_4_8, + 197, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(198, "A_D_AO1ARC", 0, + 198, UNIPHIER_PIN_DRV_4_8, + 198, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(199, "A_D_SPIXRST", UNIPHIER_PIN_IECTRL_NONE, + 199, UNIPHIER_PIN_DRV_4_8, + 199, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(200, "A_D_SPISCLK0", UNIPHIER_PIN_IECTRL_NONE, + 200, UNIPHIER_PIN_DRV_4_8, + 200, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(201, "A_D_SPITXD0", UNIPHIER_PIN_IECTRL_NONE, + 201, UNIPHIER_PIN_DRV_4_8, + 201, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(202, "A_D_SPIRXD0", UNIPHIER_PIN_IECTRL_NONE, + 202, UNIPHIER_PIN_DRV_4_8, + 202, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(203, "A_D_DMDCLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 203, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(204, "A_D_DMDPSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 204, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(205, "A_D_DMDVAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 205, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(206, "A_D_DMDDATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 206, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(207, "A_D_HDMIRXXIRQ", 0, + 207, UNIPHIER_PIN_DRV_4_8, + 207, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(208, "A_D_VBIXIRQ", 0, + 208, UNIPHIER_PIN_DRV_4_8, + 208, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(209, "A_D_HDMITXXIRQ", 0, + 209, UNIPHIER_PIN_DRV_4_8, + 209, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(210, "A_D_DMDIRQ", UNIPHIER_PIN_IECTRL_NONE, + 210, UNIPHIER_PIN_DRV_4_8, + 210, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(211, "A_D_SPICIRQ", UNIPHIER_PIN_IECTRL_NONE, + 211, UNIPHIER_PIN_DRV_4_8, + 211, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(212, "A_D_SPIBIRQ", UNIPHIER_PIN_IECTRL_NONE, + 212, UNIPHIER_PIN_DRV_4_8, + 212, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(213, "A_D_BESDAOUT", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 213, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(214, "A_D_BESDAIN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 214, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(215, "A_D_BESCLOUT", UNIPHIER_PIN_IECTRL_NONE, + 215, UNIPHIER_PIN_DRV_4_8, + 215, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(216, "A_D_VDACCLKOUT", 0, + 216, UNIPHIER_PIN_DRV_4_8, + 216, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(217, "A_D_VDACDOUT5", 0, + 217, UNIPHIER_PIN_DRV_4_8, + 217, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(218, "A_D_VDACDOUT6", 0, + 218, UNIPHIER_PIN_DRV_4_8, + 218, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(219, "A_D_VDACDOUT7", 0, + 219, UNIPHIER_PIN_DRV_4_8, + 219, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(220, "A_D_VDACDOUT8", 0, + 220, UNIPHIER_PIN_DRV_4_8, + 220, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(221, "A_D_VDACDOUT9", 0, + 221, UNIPHIER_PIN_DRV_4_8, + 221, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(222, "A_D_SIFBCKIN", 0, + 222, UNIPHIER_PIN_DRV_4_8, + 222, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(223, "A_D_SIFLRCKIN", 0, + 223, UNIPHIER_PIN_DRV_4_8, + 223, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(224, "A_D_SIFDIN", 0, + 224, UNIPHIER_PIN_DRV_4_8, + 224, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(225, "A_D_LIBCKOUT", 0, + 225, UNIPHIER_PIN_DRV_4_8, + 225, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(226, "A_D_LILRCKOUT", 0, + 226, UNIPHIER_PIN_DRV_4_8, + 226, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(227, "A_D_LIDIN", 0, + 227, UNIPHIER_PIN_DRV_4_8, + 227, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(228, "A_D_LODOUT", 0, + 228, UNIPHIER_PIN_DRV_4_8, + 228, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(229, "A_D_HPDOUT", 0, + 229, UNIPHIER_PIN_DRV_4_8, + 229, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(230, "A_D_MCLK", 0, + 230, UNIPHIER_PIN_DRV_4_8, + 230, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(231, "A_D_A2PLLREFOUT", 0, + 231, UNIPHIER_PIN_DRV_4_8, + 231, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(232, "A_D_HDMI3DSDAOUT", 0, + 232, UNIPHIER_PIN_DRV_4_8, + 232, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(233, "A_D_HDMI3DSDAIN", 0, + 233, UNIPHIER_PIN_DRV_4_8, + 233, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(234, "A_D_HDMI3DSCLIN", 0, + 234, UNIPHIER_PIN_DRV_4_8, + 234, UNIPHIER_PIN_PULL_DOWN), +}; + +static const unsigned adinter_pins[] = { + 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, + 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, + 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, + 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, + 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, + 229, 230, 231, 232, 233, 234, +}; +static const unsigned adinter_muxvals[] = { + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, +}; +static const unsigned emmc_pins[] = {36, 37, 38, 39, 40, 41, 42}; +static const unsigned emmc_muxvals[] = {1, 1, 1, 1, 1, 1, 1}; +static const unsigned emmc_dat8_pins[] = {43, 44, 45, 46}; +static const unsigned emmc_dat8_muxvals[] = {1, 1, 1, 1}; +static const unsigned i2c0_pins[] = {109, 110}; +static const unsigned i2c0_muxvals[] = {0, 0}; +static const unsigned i2c1_pins[] = {111, 112}; +static const unsigned i2c1_muxvals[] = {0, 0}; +static const unsigned i2c2_pins[] = {115, 116}; +static const unsigned i2c2_muxvals[] = {1, 1}; +static const unsigned i2c3_pins[] = {118, 119}; +static const unsigned i2c3_muxvals[] = {1, 1}; +static const unsigned nand_pins[] = {30, 31, 32, 33, 34, 35, 36, 39, 40, 41, + 42, 43, 44, 45, 46}; +static const unsigned nand_muxvals[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0}; +static const unsigned nand_cs1_pins[] = {37, 38}; +static const unsigned nand_cs1_muxvals[] = {0, 0}; +static const unsigned uart0_pins[] = {135, 136}; +static const unsigned uart0_muxvals[] = {3, 3}; +static const unsigned uart0b_pins[] = {11, 12}; +static const unsigned uart0b_muxvals[] = {2, 2}; +static const unsigned uart1_pins[] = {115, 116}; +static const unsigned uart1_muxvals[] = {0, 0}; +static const unsigned uart1b_pins[] = {113, 114}; +static const unsigned uart1b_muxvals[] = {1, 1}; +static const unsigned uart2_pins[] = {113, 114}; +static const unsigned uart2_muxvals[] = {2, 2}; +static const unsigned uart2b_pins[] = {86, 87}; +static const unsigned uart2b_muxvals[] = {1, 1}; +static const unsigned usb0_pins[] = {56, 57}; +static const unsigned usb0_muxvals[] = {0, 0}; +static const unsigned usb1_pins[] = {58, 59}; +static const unsigned usb1_muxvals[] = {0, 0}; +static const unsigned usb2_pins[] = {60, 61}; +static const unsigned usb2_muxvals[] = {0, 0}; +static const unsigned usb3_pins[] = {62, 63}; +static const unsigned usb3_muxvals[] = {0, 0}; +static const unsigned port_range0_pins[] = { + 127, 128, 129, 130, 131, 132, 133, 134, /* PORT0x */ + 135, 136, 137, 138, 139, 140, 141, 142, /* PORT1x */ + 0, 1, 2, 3, 4, 5, 6, 7, /* PORT2x */ + 8, 9, 10, 11, 12, 13, 14, 15, /* PORT3x */ + 16, 17, 18, 19, 21, 22, 23, 24, /* PORT4x */ + 25, 30, 31, 32, 33, 34, 35, 36, /* PORT5x */ + 37, 38, 39, 40, 41, 42, 43, 44, /* PORT6x */ + 45, 46, 47, 48, 49, 50, 51, 52, /* PORT7x */ + 53, 54, 55, 56, 57, 58, 59, 60, /* PORT8x */ + 61, 62, 63, 64, 65, 66, 67, 68, /* PORT9x */ + 69, 70, 71, 76, 77, 78, 79, 80, /* PORT10x */ +}; +static const unsigned port_range0_muxvals[] = { + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT0x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT1x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT2x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT3x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT4x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT5x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT6x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT7x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT8x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT9x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT10x */ +}; +static const unsigned port_range1_pins[] = { + 81, 82, 83, 84, 85, 86, 87, 88, /* PORT12x */ + 89, 90, 95, 96, 97, 98, 99, 100, /* PORT13x */ + 101, 102, 103, 104, 105, 106, 107, 108, /* PORT14x */ + 118, 119, 120, 121, 122, 123, 124, 125, /* PORT15x */ + 126, 72, 73, 92, 177, 93, 94, 176, /* PORT16x */ + 74, 91, 27, 28, 29, 75, 20, 26, /* PORT17x */ + 109, 110, 111, 112, 113, 114, 115, 116, /* PORT18x */ + 117, 143, 144, 145, 146, 147, 148, 149, /* PORT19x */ + 150, 151, 152, 153, 154, 155, 156, 157, /* PORT20x */ + 158, 159, 160, 161, 162, 163, 164, 165, /* PORT21x */ + 166, 178, 179, 180, 181, 182, 183, 184, /* PORT22x */ + 185, 187, 188, 189, 190, 191, 192, 193, /* PORT23x */ + 194, 195, 196, 197, 198, 199, 200, 201, /* PORT24x */ + 202, 203, 204, 205, 206, 207, 208, 209, /* PORT25x */ + 210, 211, 212, 213, 214, 215, 216, 217, /* PORT26x */ + 218, 219, 220, 221, 223, 224, 225, 226, /* PORT27x */ + 227, 228, 229, 230, 231, 232, 233, 234, /* PORT28x */ +}; +static const unsigned port_range1_muxvals[] = { + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT12x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT13x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT14x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT15x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT16x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT17x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT18x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT19x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT20x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT21x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT22x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT23x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT24x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT25x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT26x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT27x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT28x */ +}; +static const unsigned xirq_pins[] = { + 118, 119, 120, 121, 122, 123, 124, 125, /* XIRQ0-7 */ + 126, 72, 73, 92, 177, 93, 94, 176, /* XIRQ8-15 */ + 74, 91, 27, 28, 29, 75, 20, 26, /* XIRQ16-23 */ +}; +static const unsigned xirq_muxvals[] = { + 14, 14, 14, 14, 14, 14, 14, 14, /* XIRQ0-7 */ + 14, 14, 14, 14, 14, 14, 14, 14, /* XIRQ8-15 */ + 14, 14, 14, 14, 14, 14, 14, 14, /* XIRQ16-23 */ +}; + +static const struct uniphier_pinctrl_group ph1_ld6b_groups[] = { + UNIPHIER_PINCTRL_GROUP(adinter), + UNIPHIER_PINCTRL_GROUP(emmc), + UNIPHIER_PINCTRL_GROUP(emmc_dat8), + UNIPHIER_PINCTRL_GROUP(i2c0), + UNIPHIER_PINCTRL_GROUP(i2c1), + UNIPHIER_PINCTRL_GROUP(i2c2), + UNIPHIER_PINCTRL_GROUP(i2c3), + UNIPHIER_PINCTRL_GROUP(nand), + UNIPHIER_PINCTRL_GROUP(nand_cs1), + UNIPHIER_PINCTRL_GROUP(uart0), + UNIPHIER_PINCTRL_GROUP(uart0b), + UNIPHIER_PINCTRL_GROUP(uart1), + UNIPHIER_PINCTRL_GROUP(uart1b), + UNIPHIER_PINCTRL_GROUP(uart2), + UNIPHIER_PINCTRL_GROUP(uart2b), + UNIPHIER_PINCTRL_GROUP(usb0), + UNIPHIER_PINCTRL_GROUP(usb1), + UNIPHIER_PINCTRL_GROUP(usb2), + UNIPHIER_PINCTRL_GROUP(usb3), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range0), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range1), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq), + UNIPHIER_PINCTRL_GROUP_SINGLE(port00, port_range0, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port01, port_range0, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port02, port_range0, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port03, port_range0, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port04, port_range0, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port05, port_range0, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port06, port_range0, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port07, port_range0, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port10, port_range0, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port11, port_range0, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port12, port_range0, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port13, port_range0, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port14, port_range0, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port15, port_range0, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port16, port_range0, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port17, port_range0, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port20, port_range0, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port21, port_range0, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port22, port_range0, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port23, port_range0, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port24, port_range0, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port25, port_range0, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port26, port_range0, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port27, port_range0, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port30, port_range0, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port31, port_range0, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port32, port_range0, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port33, port_range0, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port34, port_range0, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port35, port_range0, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port36, port_range0, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port37, port_range0, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port40, port_range0, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port41, port_range0, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port42, port_range0, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port43, port_range0, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port44, port_range0, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port45, port_range0, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port46, port_range0, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port47, port_range0, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port50, port_range0, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port51, port_range0, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port52, port_range0, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port53, port_range0, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port54, port_range0, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port55, port_range0, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port56, port_range0, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port57, port_range0, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port60, port_range0, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port61, port_range0, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port62, port_range0, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port63, port_range0, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port64, port_range0, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port65, port_range0, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port66, port_range0, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port67, port_range0, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port70, port_range0, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port71, port_range0, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port72, port_range0, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port73, port_range0, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port74, port_range0, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port75, port_range0, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port76, port_range0, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port77, port_range0, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port80, port_range0, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port81, port_range0, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port82, port_range0, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port83, port_range0, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port84, port_range0, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port85, port_range0, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port86, port_range0, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port87, port_range0, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port90, port_range0, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port91, port_range0, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port92, port_range0, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port93, port_range0, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port94, port_range0, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port95, port_range0, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port96, port_range0, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port97, port_range0, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port100, port_range0, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port101, port_range0, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port102, port_range0, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port103, port_range0, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port104, port_range0, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port105, port_range0, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port106, port_range0, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port107, port_range0, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port120, port_range1, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port121, port_range1, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port122, port_range1, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port123, port_range1, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port124, port_range1, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port125, port_range1, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port126, port_range1, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port127, port_range1, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port130, port_range1, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port131, port_range1, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port132, port_range1, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port133, port_range1, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port134, port_range1, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port135, port_range1, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port136, port_range1, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port137, port_range1, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port140, port_range1, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port141, port_range1, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port142, port_range1, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port143, port_range1, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port144, port_range1, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port145, port_range1, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port146, port_range1, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port147, port_range1, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port150, port_range1, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port151, port_range1, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port152, port_range1, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port153, port_range1, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port154, port_range1, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port155, port_range1, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port156, port_range1, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port157, port_range1, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port160, port_range1, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port161, port_range1, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port162, port_range1, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port163, port_range1, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port164, port_range1, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port165, port_range1, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port166, port_range1, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port167, port_range1, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port170, port_range1, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port171, port_range1, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port172, port_range1, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port173, port_range1, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port174, port_range1, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port175, port_range1, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port176, port_range1, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port177, port_range1, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port180, port_range1, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port181, port_range1, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port182, port_range1, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port183, port_range1, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port184, port_range1, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port185, port_range1, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port186, port_range1, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port187, port_range1, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port190, port_range1, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port191, port_range1, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port192, port_range1, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port193, port_range1, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port194, port_range1, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port195, port_range1, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port196, port_range1, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port197, port_range1, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port200, port_range1, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port201, port_range1, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port202, port_range1, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port203, port_range1, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port204, port_range1, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port205, port_range1, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port206, port_range1, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port207, port_range1, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port210, port_range1, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port211, port_range1, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port212, port_range1, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port213, port_range1, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port214, port_range1, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port215, port_range1, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port216, port_range1, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port217, port_range1, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port220, port_range1, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port221, port_range1, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port222, port_range1, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port223, port_range1, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port224, port_range1, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port225, port_range1, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port226, port_range1, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port227, port_range1, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port230, port_range1, 88), + UNIPHIER_PINCTRL_GROUP_SINGLE(port231, port_range1, 89), + UNIPHIER_PINCTRL_GROUP_SINGLE(port232, port_range1, 90), + UNIPHIER_PINCTRL_GROUP_SINGLE(port233, port_range1, 91), + UNIPHIER_PINCTRL_GROUP_SINGLE(port234, port_range1, 92), + UNIPHIER_PINCTRL_GROUP_SINGLE(port235, port_range1, 93), + UNIPHIER_PINCTRL_GROUP_SINGLE(port236, port_range1, 94), + UNIPHIER_PINCTRL_GROUP_SINGLE(port237, port_range1, 95), + UNIPHIER_PINCTRL_GROUP_SINGLE(port240, port_range1, 96), + UNIPHIER_PINCTRL_GROUP_SINGLE(port241, port_range1, 97), + UNIPHIER_PINCTRL_GROUP_SINGLE(port242, port_range1, 98), + UNIPHIER_PINCTRL_GROUP_SINGLE(port243, port_range1, 99), + UNIPHIER_PINCTRL_GROUP_SINGLE(port244, port_range1, 100), + UNIPHIER_PINCTRL_GROUP_SINGLE(port245, port_range1, 101), + UNIPHIER_PINCTRL_GROUP_SINGLE(port246, port_range1, 102), + UNIPHIER_PINCTRL_GROUP_SINGLE(port247, port_range1, 103), + UNIPHIER_PINCTRL_GROUP_SINGLE(port250, port_range1, 104), + UNIPHIER_PINCTRL_GROUP_SINGLE(port251, port_range1, 105), + UNIPHIER_PINCTRL_GROUP_SINGLE(port252, port_range1, 106), + UNIPHIER_PINCTRL_GROUP_SINGLE(port253, port_range1, 107), + UNIPHIER_PINCTRL_GROUP_SINGLE(port254, port_range1, 108), + UNIPHIER_PINCTRL_GROUP_SINGLE(port255, port_range1, 109), + UNIPHIER_PINCTRL_GROUP_SINGLE(port256, port_range1, 110), + UNIPHIER_PINCTRL_GROUP_SINGLE(port257, port_range1, 111), + UNIPHIER_PINCTRL_GROUP_SINGLE(port260, port_range1, 112), + UNIPHIER_PINCTRL_GROUP_SINGLE(port261, port_range1, 113), + UNIPHIER_PINCTRL_GROUP_SINGLE(port262, port_range1, 114), + UNIPHIER_PINCTRL_GROUP_SINGLE(port263, port_range1, 115), + UNIPHIER_PINCTRL_GROUP_SINGLE(port264, port_range1, 116), + UNIPHIER_PINCTRL_GROUP_SINGLE(port265, port_range1, 117), + UNIPHIER_PINCTRL_GROUP_SINGLE(port266, port_range1, 118), + UNIPHIER_PINCTRL_GROUP_SINGLE(port267, port_range1, 119), + UNIPHIER_PINCTRL_GROUP_SINGLE(port270, port_range1, 120), + UNIPHIER_PINCTRL_GROUP_SINGLE(port271, port_range1, 121), + UNIPHIER_PINCTRL_GROUP_SINGLE(port272, port_range1, 122), + UNIPHIER_PINCTRL_GROUP_SINGLE(port273, port_range1, 123), + UNIPHIER_PINCTRL_GROUP_SINGLE(port274, port_range1, 124), + UNIPHIER_PINCTRL_GROUP_SINGLE(port275, port_range1, 125), + UNIPHIER_PINCTRL_GROUP_SINGLE(port276, port_range1, 126), + UNIPHIER_PINCTRL_GROUP_SINGLE(port277, port_range1, 127), + UNIPHIER_PINCTRL_GROUP_SINGLE(port280, port_range1, 128), + UNIPHIER_PINCTRL_GROUP_SINGLE(port281, port_range1, 129), + UNIPHIER_PINCTRL_GROUP_SINGLE(port282, port_range1, 130), + UNIPHIER_PINCTRL_GROUP_SINGLE(port283, port_range1, 131), + UNIPHIER_PINCTRL_GROUP_SINGLE(port284, port_range1, 132), + UNIPHIER_PINCTRL_GROUP_SINGLE(port285, port_range1, 133), + UNIPHIER_PINCTRL_GROUP_SINGLE(port286, port_range1, 134), + UNIPHIER_PINCTRL_GROUP_SINGLE(port287, port_range1, 135), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq0, xirq, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq1, xirq, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq2, xirq, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq3, xirq, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq4, xirq, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq5, xirq, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq6, xirq, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq7, xirq, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq8, xirq, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq9, xirq, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq10, xirq, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq11, xirq, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq12, xirq, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq13, xirq, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq14, xirq, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq15, xirq, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq16, xirq, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq17, xirq, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq18, xirq, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq19, xirq, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq20, xirq, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq21, xirq, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq22, xirq, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq23, xirq, 23), +}; + +static const char * const adinter_groups[] = {"adinter"}; +static const char * const emmc_groups[] = {"emmc", "emmc_dat8"}; +static const char * const i2c0_groups[] = {"i2c0"}; +static const char * const i2c1_groups[] = {"i2c1"}; +static const char * const i2c2_groups[] = {"i2c2"}; +static const char * const i2c3_groups[] = {"i2c3"}; +static const char * const nand_groups[] = {"nand", "nand_cs1"}; +static const char * const uart0_groups[] = {"uart0", "uart0b"}; +static const char * const uart1_groups[] = {"uart1", "uart1b"}; +static const char * const uart2_groups[] = {"uart2", "uart2b"}; +static const char * const usb0_groups[] = {"usb0"}; +static const char * const usb1_groups[] = {"usb1"}; +static const char * const usb2_groups[] = {"usb2"}; +static const char * const usb3_groups[] = {"usb3"}; +static const char * const port_groups[] = { + "port00", "port01", "port02", "port03", + "port04", "port05", "port06", "port07", + "port10", "port11", "port12", "port13", + "port14", "port15", "port16", "port17", + "port20", "port21", "port22", "port23", + "port24", "port25", "port26", "port27", + "port30", "port31", "port32", "port33", + "port34", "port35", "port36", "port37", + "port40", "port41", "port42", "port43", + "port44", "port45", "port46", "port47", + "port50", "port51", "port52", "port53", + "port54", "port55", "port56", "port57", + "port60", "port61", "port62", "port63", + "port64", "port65", "port66", "port67", + "port70", "port71", "port72", "port73", + "port74", "port75", "port76", "port77", + "port80", "port81", "port82", "port83", + "port84", "port85", "port86", "port87", + "port90", "port91", "port92", "port93", + "port94", "port95", "port96", "port97", + "port100", "port101", "port102", "port103", + "port104", "port105", "port106", "port107", + /* port110-117 missing */ + "port120", "port121", "port122", "port123", + "port124", "port125", "port126", "port127", + "port130", "port131", "port132", "port133", + "port134", "port135", "port136", "port137", + "port140", "port141", "port142", "port143", + "port144", "port145", "port146", "port147", + "port150", "port151", "port152", "port153", + "port154", "port155", "port156", "port157", + "port160", "port161", "port162", "port163", + "port164", "port165", "port166", "port167", + "port170", "port171", "port172", "port173", + "port174", "port175", "port176", "port177", + "port180", "port181", "port182", "port183", + "port184", "port185", "port186", "port187", + "port190", "port191", "port192", "port193", + "port194", "port195", "port196", "port197", + "port200", "port201", "port202", "port203", + "port204", "port205", "port206", "port207", + "port210", "port211", "port212", "port213", + "port214", "port215", "port216", "port217", + "port220", "port221", "port222", "port223", + "port224", "port225", "port226", "port227", + "port230", "port231", "port232", "port233", + "port234", "port235", "port236", "port237", + "port240", "port241", "port242", "port243", + "port244", "port245", "port246", "port247", + "port250", "port251", "port252", "port253", + "port254", "port255", "port256", "port257", + "port260", "port261", "port262", "port263", + "port264", "port265", "port266", "port267", + "port270", "port271", "port272", "port273", + "port274", "port275", "port276", "port277", + "port280", "port281", "port282", "port283", + "port284", "port285", "port286", "port287", +}; +static const char * const xirq_groups[] = { + "xirq0", "xirq1", "xirq2", "xirq3", + "xirq4", "xirq5", "xirq6", "xirq7", + "xirq8", "xirq9", "xirq10", "xirq11", + "xirq12", "xirq13", "xirq14", "xirq15", + "xirq16", "xirq17", "xirq18", "xirq19", + "xirq20", "xirq21", "xirq22", "xirq23", +}; + +static const struct uniphier_pinmux_function ph1_ld6b_functions[] = { + UNIPHIER_PINMUX_FUNCTION(adinter), /* Achip-Dchip interconnect */ + UNIPHIER_PINMUX_FUNCTION(emmc), + UNIPHIER_PINMUX_FUNCTION(i2c0), + UNIPHIER_PINMUX_FUNCTION(i2c1), + UNIPHIER_PINMUX_FUNCTION(i2c2), + UNIPHIER_PINMUX_FUNCTION(i2c3), + UNIPHIER_PINMUX_FUNCTION(nand), + UNIPHIER_PINMUX_FUNCTION(uart0), + UNIPHIER_PINMUX_FUNCTION(uart1), + UNIPHIER_PINMUX_FUNCTION(uart2), + UNIPHIER_PINMUX_FUNCTION(usb0), + UNIPHIER_PINMUX_FUNCTION(usb1), + UNIPHIER_PINMUX_FUNCTION(usb2), + UNIPHIER_PINMUX_FUNCTION(usb3), + UNIPHIER_PINMUX_FUNCTION(port), + UNIPHIER_PINMUX_FUNCTION(xirq), +}; + +static struct uniphier_pinctrl_socdata ph1_ld6b_pindata = { + .groups = ph1_ld6b_groups, + .groups_count = ARRAY_SIZE(ph1_ld6b_groups), + .functions = ph1_ld6b_functions, + .functions_count = ARRAY_SIZE(ph1_ld6b_functions), + .mux_bits = 8, + .reg_stride = 4, + .load_pinctrl = false, +}; + +static struct pinctrl_desc ph1_ld6b_pinctrl_desc = { + .name = DRIVER_NAME, + .pins = ph1_ld6b_pins, + .npins = ARRAY_SIZE(ph1_ld6b_pins), + .owner = THIS_MODULE, +}; + +static int ph1_ld6b_pinctrl_probe(struct platform_device *pdev) +{ + return uniphier_pinctrl_probe(pdev, &ph1_ld6b_pinctrl_desc, + &ph1_ld6b_pindata); +} + +static const struct of_device_id ph1_ld6b_pinctrl_match[] = { + { .compatible = "socionext,ph1-ld6b-pinctrl" }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, ph1_ld6b_pinctrl_match); + +static struct platform_driver ph1_ld6b_pinctrl_driver = { + .probe = ph1_ld6b_pinctrl_probe, + .remove = uniphier_pinctrl_remove, + .driver = { + .name = DRIVER_NAME, + .of_match_table = ph1_ld6b_pinctrl_match, + }, +}; +module_platform_driver(ph1_ld6b_pinctrl_driver); + +MODULE_AUTHOR("Masahiro Yamada "); +MODULE_DESCRIPTION("UniPhier PH1-LD6b pinctrl driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pinctrl/uniphier/pinctrl-ph1-pro4.c b/drivers/pinctrl/uniphier/pinctrl-ph1-pro4.c new file mode 100644 index 00000000000000..96921e40da5f9b --- /dev/null +++ b/drivers/pinctrl/uniphier/pinctrl-ph1-pro4.c @@ -0,0 +1,1554 @@ +/* + * Copyright (C) 2015 Masahiro Yamada + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program5 is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include +#include +#include + +#include "pinctrl-uniphier.h" + +#define DRIVER_NAME "ph1-pro4-pinctrl" + +static const struct pinctrl_pin_desc ph1_pro4_pins[] = { + UNIPHIER_PINCTRL_PIN(0, "CK24O", UNIPHIER_PIN_IECTRL_NONE, + 0, UNIPHIER_PIN_DRV_4_8, + 0, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(1, "VC27A", UNIPHIER_PIN_IECTRL_NONE, + 1, UNIPHIER_PIN_DRV_4_8, + 1, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(2, "CK27AI", UNIPHIER_PIN_IECTRL_NONE, + 2, UNIPHIER_PIN_DRV_4_8, + 2, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(3, "CK27AO", UNIPHIER_PIN_IECTRL_NONE, + 3, UNIPHIER_PIN_DRV_4_8, + 3, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(4, "CKSEL", UNIPHIER_PIN_IECTRL_NONE, + 4, UNIPHIER_PIN_DRV_4_8, + 4, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(5, "CK27AV", UNIPHIER_PIN_IECTRL_NONE, + 5, UNIPHIER_PIN_DRV_4_8, + 5, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(6, "AEXCKA", UNIPHIER_PIN_IECTRL_NONE, + 6, UNIPHIER_PIN_DRV_4_8, + 6, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(7, "ASEL", UNIPHIER_PIN_IECTRL_NONE, + 7, UNIPHIER_PIN_DRV_4_8, + 7, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(8, "ARCRESET", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_4_8, + 8, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(9, "ARCUNLOCK", UNIPHIER_PIN_IECTRL_NONE, + 9, UNIPHIER_PIN_DRV_4_8, + 9, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(10, "XSRST", UNIPHIER_PIN_IECTRL_NONE, + 10, UNIPHIER_PIN_DRV_4_8, + 10, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(11, "XNMIRQ", UNIPHIER_PIN_IECTRL_NONE, + 11, UNIPHIER_PIN_DRV_4_8, + 11, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(12, "XSCIRQ", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_4_8, + 12, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(13, "EXTRG", UNIPHIER_PIN_IECTRL_NONE, + 13, UNIPHIER_PIN_DRV_4_8, + 13, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(14, "TRCCLK", UNIPHIER_PIN_IECTRL_NONE, + 14, UNIPHIER_PIN_DRV_4_8, + 14, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(15, "TRCCTL", UNIPHIER_PIN_IECTRL_NONE, + 15, UNIPHIER_PIN_DRV_4_8, + 15, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(16, "TRCD0", UNIPHIER_PIN_IECTRL_NONE, + 16, UNIPHIER_PIN_DRV_4_8, + 16, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(17, "TRCD1", UNIPHIER_PIN_IECTRL_NONE, + 17, UNIPHIER_PIN_DRV_4_8, + 17, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(18, "TRCD2", UNIPHIER_PIN_IECTRL_NONE, + 18, UNIPHIER_PIN_DRV_4_8, + 18, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(19, "TRCD3", UNIPHIER_PIN_IECTRL_NONE, + 19, UNIPHIER_PIN_DRV_4_8, + 19, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(20, "TRCD4", UNIPHIER_PIN_IECTRL_NONE, + 20, UNIPHIER_PIN_DRV_4_8, + 20, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(21, "TRCD5", UNIPHIER_PIN_IECTRL_NONE, + 21, UNIPHIER_PIN_DRV_4_8, + 21, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(22, "TRCD6", UNIPHIER_PIN_IECTRL_NONE, + 22, UNIPHIER_PIN_DRV_4_8, + 22, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(23, "TRCD7", UNIPHIER_PIN_IECTRL_NONE, + 23, UNIPHIER_PIN_DRV_4_8, + 23, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(24, "XECS1", UNIPHIER_PIN_IECTRL_NONE, + 24, UNIPHIER_PIN_DRV_4_8, + 24, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(25, "ERXW", UNIPHIER_PIN_IECTRL_NONE, + 25, UNIPHIER_PIN_DRV_4_8, + 25, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(26, "XERWE0", UNIPHIER_PIN_IECTRL_NONE, + 26, UNIPHIER_PIN_DRV_4_8, + 26, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(27, "XERWE1", UNIPHIER_PIN_IECTRL_NONE, + 27, UNIPHIER_PIN_DRV_4_8, + 27, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(28, "ES0", UNIPHIER_PIN_IECTRL_NONE, + 28, UNIPHIER_PIN_DRV_4_8, + 28, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(29, "ES1", UNIPHIER_PIN_IECTRL_NONE, + 29, UNIPHIER_PIN_DRV_4_8, + 29, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(30, "ES2", UNIPHIER_PIN_IECTRL_NONE, + 30, UNIPHIER_PIN_DRV_4_8, + 30, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(31, "ED0", UNIPHIER_PIN_IECTRL_NONE, + 31, UNIPHIER_PIN_DRV_4_8, + 31, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(32, "ED1", UNIPHIER_PIN_IECTRL_NONE, + 32, UNIPHIER_PIN_DRV_4_8, + 32, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(33, "ED2", UNIPHIER_PIN_IECTRL_NONE, + 33, UNIPHIER_PIN_DRV_4_8, + 33, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(34, "ED3", UNIPHIER_PIN_IECTRL_NONE, + 34, UNIPHIER_PIN_DRV_4_8, + 34, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(35, "ED4", UNIPHIER_PIN_IECTRL_NONE, + 35, UNIPHIER_PIN_DRV_4_8, + 35, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(36, "ED5", UNIPHIER_PIN_IECTRL_NONE, + 36, UNIPHIER_PIN_DRV_4_8, + 36, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(37, "ED6", UNIPHIER_PIN_IECTRL_NONE, + 37, UNIPHIER_PIN_DRV_4_8, + 37, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(38, "ED7", UNIPHIER_PIN_IECTRL_NONE, + 38, UNIPHIER_PIN_DRV_4_8, + 38, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(39, "BOOTSWAP", UNIPHIER_PIN_IECTRL_NONE, + 39, UNIPHIER_PIN_DRV_NONE, + 39, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(40, "NFD0", UNIPHIER_PIN_IECTRL_NONE, + 2, UNIPHIER_PIN_DRV_8_12_16_20, + 40, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(41, "NFD1", UNIPHIER_PIN_IECTRL_NONE, + 3, UNIPHIER_PIN_DRV_8_12_16_20, + 41, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(42, "NFD2", UNIPHIER_PIN_IECTRL_NONE, + 4, UNIPHIER_PIN_DRV_8_12_16_20, + 42, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(43, "NFD3", UNIPHIER_PIN_IECTRL_NONE, + 5, UNIPHIER_PIN_DRV_8_12_16_20, + 43, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(44, "NFD4", UNIPHIER_PIN_IECTRL_NONE, + 6, UNIPHIER_PIN_DRV_8_12_16_20, + 44, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(45, "NFD5", UNIPHIER_PIN_IECTRL_NONE, + 7, UNIPHIER_PIN_DRV_8_12_16_20, + 45, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(46, "NFD6", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_8_12_16_20, + 46, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(47, "NFD7", UNIPHIER_PIN_IECTRL_NONE, + 9, UNIPHIER_PIN_DRV_8_12_16_20, + 47, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(48, "NFALE", UNIPHIER_PIN_IECTRL_NONE, + 48, UNIPHIER_PIN_DRV_4_8, + 48, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(49, "NFCLE", UNIPHIER_PIN_IECTRL_NONE, + 49, UNIPHIER_PIN_DRV_4_8, + 49, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(50, "XNFRE", UNIPHIER_PIN_IECTRL_NONE, + 50, UNIPHIER_PIN_DRV_4_8, + 50, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(51, "XNFWE", UNIPHIER_PIN_IECTRL_NONE, + 0, UNIPHIER_PIN_DRV_8_12_16_20, + 51, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(52, "XNFWP", UNIPHIER_PIN_IECTRL_NONE, + 52, UNIPHIER_PIN_DRV_4_8, + 52, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(53, "XNFCE0", UNIPHIER_PIN_IECTRL_NONE, + 1, UNIPHIER_PIN_DRV_8_12_16_20, + 53, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(54, "NRYBY0", UNIPHIER_PIN_IECTRL_NONE, + 54, UNIPHIER_PIN_DRV_4_8, + 54, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(55, "DMDSCLTST", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_NONE, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(56, "DMDSDATST", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(57, "AGCI0", 3, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 55, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(58, "DMDSCL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(59, "DMDSDA0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(60, "AGCBS0", 5, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 56, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(61, "DMDSCL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(62, "DMDSDA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(63, "ANTSHORT", UNIPHIER_PIN_IECTRL_NONE, + 57, UNIPHIER_PIN_DRV_4_8, + 57, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(64, "CH0CLK", UNIPHIER_PIN_IECTRL_NONE, + 58, UNIPHIER_PIN_DRV_4_8, + 58, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(65, "CH0VAL", UNIPHIER_PIN_IECTRL_NONE, + 59, UNIPHIER_PIN_DRV_4_8, + 59, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(66, "CH0PSYNC", UNIPHIER_PIN_IECTRL_NONE, + 60, UNIPHIER_PIN_DRV_4_8, + 60, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(67, "CH0DATA", UNIPHIER_PIN_IECTRL_NONE, + 61, UNIPHIER_PIN_DRV_4_8, + 61, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(68, "CH1CLK", UNIPHIER_PIN_IECTRL_NONE, + 62, UNIPHIER_PIN_DRV_4_8, + 62, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(69, "CH1VAL", UNIPHIER_PIN_IECTRL_NONE, + 63, UNIPHIER_PIN_DRV_4_8, + 63, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(70, "CH1PSYNC", UNIPHIER_PIN_IECTRL_NONE, + 64, UNIPHIER_PIN_DRV_4_8, + 64, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(71, "CH1DATA", UNIPHIER_PIN_IECTRL_NONE, + 65, UNIPHIER_PIN_DRV_4_8, + 65, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(72, "CH2CLK", UNIPHIER_PIN_IECTRL_NONE, + 66, UNIPHIER_PIN_DRV_4_8, + 66, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(73, "CH2VAL", UNIPHIER_PIN_IECTRL_NONE, + 67, UNIPHIER_PIN_DRV_4_8, + 67, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(74, "CH2PSYNC", UNIPHIER_PIN_IECTRL_NONE, + 68, UNIPHIER_PIN_DRV_4_8, + 68, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(75, "CH2DATA", UNIPHIER_PIN_IECTRL_NONE, + 69, UNIPHIER_PIN_DRV_4_8, + 69, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(76, "CH3CLK", UNIPHIER_PIN_IECTRL_NONE, + 70, UNIPHIER_PIN_DRV_4_8, + 70, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(77, "CH3VAL", UNIPHIER_PIN_IECTRL_NONE, + 71, UNIPHIER_PIN_DRV_4_8, + 71, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(78, "CH3PSYNC", UNIPHIER_PIN_IECTRL_NONE, + 72, UNIPHIER_PIN_DRV_4_8, + 72, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(79, "CH3DATA", UNIPHIER_PIN_IECTRL_NONE, + 73, UNIPHIER_PIN_DRV_4_8, + 73, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(80, "CH4CLK", UNIPHIER_PIN_IECTRL_NONE, + 74, UNIPHIER_PIN_DRV_4_8, + 74, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(81, "CH4VAL", UNIPHIER_PIN_IECTRL_NONE, + 75, UNIPHIER_PIN_DRV_4_8, + 75, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(82, "CH4PSYNC", UNIPHIER_PIN_IECTRL_NONE, + 76, UNIPHIER_PIN_DRV_4_8, + 76, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(83, "CH4DATA", UNIPHIER_PIN_IECTRL_NONE, + 77, UNIPHIER_PIN_DRV_4_8, + 77, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(84, "CH5CLK", UNIPHIER_PIN_IECTRL_NONE, + 78, UNIPHIER_PIN_DRV_4_8, + 78, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(85, "CH5VAL", UNIPHIER_PIN_IECTRL_NONE, + 79, UNIPHIER_PIN_DRV_4_8, + 79, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(86, "CH5PSYNC", UNIPHIER_PIN_IECTRL_NONE, + 80, UNIPHIER_PIN_DRV_4_8, + 80, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(87, "CH5DATA", UNIPHIER_PIN_IECTRL_NONE, + 81, UNIPHIER_PIN_DRV_4_8, + 81, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(88, "CH6CLK", UNIPHIER_PIN_IECTRL_NONE, + 82, UNIPHIER_PIN_DRV_4_8, + 82, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(89, "CH6VAL", UNIPHIER_PIN_IECTRL_NONE, + 83, UNIPHIER_PIN_DRV_4_8, + 83, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(90, "CH6PSYNC", UNIPHIER_PIN_IECTRL_NONE, + 84, UNIPHIER_PIN_DRV_4_8, + 84, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(91, "CH6DATA", UNIPHIER_PIN_IECTRL_NONE, + 85, UNIPHIER_PIN_DRV_4_8, + 85, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(92, "CKFEO", UNIPHIER_PIN_IECTRL_NONE, + 86, UNIPHIER_PIN_DRV_4_8, + 86, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(93, "XFERST", UNIPHIER_PIN_IECTRL_NONE, + 87, UNIPHIER_PIN_DRV_4_8, + 87, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(94, "P_FE_ON", UNIPHIER_PIN_IECTRL_NONE, + 88, UNIPHIER_PIN_DRV_4_8, + 88, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(95, "P_TU0_ON", UNIPHIER_PIN_IECTRL_NONE, + 89, UNIPHIER_PIN_DRV_4_8, + 89, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(96, "XFEIRQ0", UNIPHIER_PIN_IECTRL_NONE, + 90, UNIPHIER_PIN_DRV_4_8, + 90, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(97, "XFEIRQ1", UNIPHIER_PIN_IECTRL_NONE, + 91, UNIPHIER_PIN_DRV_4_8, + 91, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(98, "XFEIRQ2", UNIPHIER_PIN_IECTRL_NONE, + 92, UNIPHIER_PIN_DRV_4_8, + 92, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(99, "XFEIRQ3", UNIPHIER_PIN_IECTRL_NONE, + 93, UNIPHIER_PIN_DRV_4_8, + 93, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(100, "XFEIRQ4", UNIPHIER_PIN_IECTRL_NONE, + 94, UNIPHIER_PIN_DRV_4_8, + 94, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(101, "XFEIRQ5", UNIPHIER_PIN_IECTRL_NONE, + 95, UNIPHIER_PIN_DRV_4_8, + 95, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(102, "XFEIRQ6", UNIPHIER_PIN_IECTRL_NONE, + 96, UNIPHIER_PIN_DRV_4_8, + 96, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(103, "SMTCLK0", UNIPHIER_PIN_IECTRL_NONE, + 97, UNIPHIER_PIN_DRV_4_8, + 97, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(104, "SMTRST0", UNIPHIER_PIN_IECTRL_NONE, + 98, UNIPHIER_PIN_DRV_4_8, + 98, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(105, "SMTCMD0", UNIPHIER_PIN_IECTRL_NONE, + 99, UNIPHIER_PIN_DRV_4_8, + 99, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(106, "SMTD0", UNIPHIER_PIN_IECTRL_NONE, + 100, UNIPHIER_PIN_DRV_4_8, + 100, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(107, "SMTSEL0", UNIPHIER_PIN_IECTRL_NONE, + 101, UNIPHIER_PIN_DRV_4_8, + 101, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(108, "SMTDET0", UNIPHIER_PIN_IECTRL_NONE, + 102, UNIPHIER_PIN_DRV_4_8, + 102, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(109, "SMTCLK1", UNIPHIER_PIN_IECTRL_NONE, + 103, UNIPHIER_PIN_DRV_4_8, + 103, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(110, "SMTRST1", UNIPHIER_PIN_IECTRL_NONE, + 104, UNIPHIER_PIN_DRV_4_8, + 104, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(111, "SMTCMD1", UNIPHIER_PIN_IECTRL_NONE, + 105, UNIPHIER_PIN_DRV_4_8, + 105, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(112, "SMTD1", UNIPHIER_PIN_IECTRL_NONE, + 106, UNIPHIER_PIN_DRV_4_8, + 106, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(113, "SMTSEL1", UNIPHIER_PIN_IECTRL_NONE, + 107, UNIPHIER_PIN_DRV_4_8, + 107, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(114, "SMTDET1", UNIPHIER_PIN_IECTRL_NONE, + 108, UNIPHIER_PIN_DRV_4_8, + 108, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(115, "XINTM", UNIPHIER_PIN_IECTRL_NONE, + 109, UNIPHIER_PIN_DRV_4_8, + 109, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(116, "SCLKM", UNIPHIER_PIN_IECTRL_NONE, + 110, UNIPHIER_PIN_DRV_4_8, + 110, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(117, "SBMTP", UNIPHIER_PIN_IECTRL_NONE, + 111, UNIPHIER_PIN_DRV_4_8, + 111, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(118, "SBPTM", UNIPHIER_PIN_IECTRL_NONE, + 112, UNIPHIER_PIN_DRV_4_8, + 112, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(119, "XMPREQ", UNIPHIER_PIN_IECTRL_NONE, + 113, UNIPHIER_PIN_DRV_4_8, + 113, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(120, "XINTP", UNIPHIER_PIN_IECTRL_NONE, + 114, UNIPHIER_PIN_DRV_4_8, + 114, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(121, "LPST", UNIPHIER_PIN_IECTRL_NONE, + 115, UNIPHIER_PIN_DRV_4_8, + 115, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(122, "SDBOOT", UNIPHIER_PIN_IECTRL_NONE, + 116, UNIPHIER_PIN_DRV_4_8, + 116, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(123, "BFAIL", UNIPHIER_PIN_IECTRL_NONE, + 117, UNIPHIER_PIN_DRV_4_8, + 117, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(124, "XFWE", UNIPHIER_PIN_IECTRL_NONE, + 118, UNIPHIER_PIN_DRV_4_8, + 118, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(125, "RF_COM_RDY", UNIPHIER_PIN_IECTRL_NONE, + 119, UNIPHIER_PIN_DRV_4_8, + 119, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(126, "XDIAG0", UNIPHIER_PIN_IECTRL_NONE, + 120, UNIPHIER_PIN_DRV_4_8, + 120, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(127, "RXD0", UNIPHIER_PIN_IECTRL_NONE, + 121, UNIPHIER_PIN_DRV_4_8, + 121, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(128, "TXD0", UNIPHIER_PIN_IECTRL_NONE, + 122, UNIPHIER_PIN_DRV_4_8, + 122, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(129, "RXD1", UNIPHIER_PIN_IECTRL_NONE, + 123, UNIPHIER_PIN_DRV_4_8, + 123, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(130, "TXD1", UNIPHIER_PIN_IECTRL_NONE, + 124, UNIPHIER_PIN_DRV_4_8, + 124, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(131, "RXD2", UNIPHIER_PIN_IECTRL_NONE, + 125, UNIPHIER_PIN_DRV_4_8, + 125, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(132, "TXD2", UNIPHIER_PIN_IECTRL_NONE, + 126, UNIPHIER_PIN_DRV_4_8, + 126, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(133, "SS0CS", UNIPHIER_PIN_IECTRL_NONE, + 127, UNIPHIER_PIN_DRV_4_8, + 127, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(134, "SS0CLK", UNIPHIER_PIN_IECTRL_NONE, + 128, UNIPHIER_PIN_DRV_4_8, + 128, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(135, "SS0DO", UNIPHIER_PIN_IECTRL_NONE, + 129, UNIPHIER_PIN_DRV_4_8, + 129, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(136, "SS0DI", UNIPHIER_PIN_IECTRL_NONE, + 130, UNIPHIER_PIN_DRV_4_8, + 130, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(137, "MS0CS0", UNIPHIER_PIN_IECTRL_NONE, + 131, UNIPHIER_PIN_DRV_4_8, + 131, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(138, "MS0CLK", UNIPHIER_PIN_IECTRL_NONE, + 132, UNIPHIER_PIN_DRV_4_8, + 132, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(139, "MS0DI", UNIPHIER_PIN_IECTRL_NONE, + 133, UNIPHIER_PIN_DRV_4_8, + 133, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(140, "MS0DO", UNIPHIER_PIN_IECTRL_NONE, + 134, UNIPHIER_PIN_DRV_4_8, + 134, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(141, "XMDMRST", UNIPHIER_PIN_IECTRL_NONE, + 135, UNIPHIER_PIN_DRV_4_8, + 135, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(142, "SCL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(143, "SDA0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(144, "SCL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(145, "SDA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(146, "SCL2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(147, "SDA2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(148, "SCL3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(149, "SDA3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(150, "SD0DAT0", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_8_12_16_20, + 136, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(151, "SD0DAT1", UNIPHIER_PIN_IECTRL_NONE, + 13, UNIPHIER_PIN_DRV_8_12_16_20, + 137, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(152, "SD0DAT2", UNIPHIER_PIN_IECTRL_NONE, + 14, UNIPHIER_PIN_DRV_8_12_16_20, + 138, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(153, "SD0DAT3", UNIPHIER_PIN_IECTRL_NONE, + 15, UNIPHIER_PIN_DRV_8_12_16_20, + 139, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(154, "SD0CMD", UNIPHIER_PIN_IECTRL_NONE, + 11, UNIPHIER_PIN_DRV_8_12_16_20, + 141, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(155, "SD0CLK", UNIPHIER_PIN_IECTRL_NONE, + 10, UNIPHIER_PIN_DRV_8_12_16_20, + 140, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(156, "SD0CD", UNIPHIER_PIN_IECTRL_NONE, + 142, UNIPHIER_PIN_DRV_4_8, + 142, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(157, "SD0WP", UNIPHIER_PIN_IECTRL_NONE, + 143, UNIPHIER_PIN_DRV_4_8, + 143, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(158, "SD0VTCG", UNIPHIER_PIN_IECTRL_NONE, + 144, UNIPHIER_PIN_DRV_4_8, + 144, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(159, "CK25O", UNIPHIER_PIN_IECTRL_NONE, + 145, UNIPHIER_PIN_DRV_4_8, + 145, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(160, "RGMII_TXCLK", 6, + 146, UNIPHIER_PIN_DRV_4_8, + 146, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(161, "RGMII_TXD0", 6, + 147, UNIPHIER_PIN_DRV_4_8, + 147, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(162, "RGMII_TXD1", 6, + 148, UNIPHIER_PIN_DRV_4_8, + 148, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(163, "RGMII_TXD2", 6, + 149, UNIPHIER_PIN_DRV_4_8, + 149, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(164, "RGMII_TXD3", 6, + 150, UNIPHIER_PIN_DRV_4_8, + 150, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(165, "RGMII_TXCTL", 6, + 151, UNIPHIER_PIN_DRV_4_8, + 151, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(166, "MII_TXER", UNIPHIER_PIN_IECTRL_NONE, + 152, UNIPHIER_PIN_DRV_4_8, + 152, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(167, "RGMII_RXCLK", 6, + 153, UNIPHIER_PIN_DRV_4_8, + 153, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(168, "RGMII_RXD0", 6, + 154, UNIPHIER_PIN_DRV_4_8, + 154, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(169, "RGMII_RXD1", 6, + 155, UNIPHIER_PIN_DRV_4_8, + 155, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(170, "RGMII_RXD2", 6, + 156, UNIPHIER_PIN_DRV_4_8, + 156, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(171, "RGMII_RXD3", 6, + 157, UNIPHIER_PIN_DRV_4_8, + 157, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(172, "RGMII_RXCTL", 6, + 158, UNIPHIER_PIN_DRV_4_8, + 158, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(173, "MII_RXER", 6, + 159, UNIPHIER_PIN_DRV_4_8, + 159, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(174, "MII_CRS", 6, + 160, UNIPHIER_PIN_DRV_4_8, + 160, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(175, "MII_COL", 6, + 161, UNIPHIER_PIN_DRV_4_8, + 161, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(176, "MDC", 6, + 162, UNIPHIER_PIN_DRV_4_8, + 162, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(177, "MDIO", 6, + 163, UNIPHIER_PIN_DRV_4_8, + 163, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(178, "MDIO_INTL", 6, + 164, UNIPHIER_PIN_DRV_4_8, + 164, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(179, "XETH_RST", 6, + 165, UNIPHIER_PIN_DRV_4_8, + 165, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(180, "USB0VBUS", UNIPHIER_PIN_IECTRL_NONE, + 166, UNIPHIER_PIN_DRV_4_8, + 166, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(181, "USB0OD", UNIPHIER_PIN_IECTRL_NONE, + 167, UNIPHIER_PIN_DRV_4_8, + 167, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(182, "USB1VBUS", UNIPHIER_PIN_IECTRL_NONE, + 168, UNIPHIER_PIN_DRV_4_8, + 168, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(183, "USB1OD", UNIPHIER_PIN_IECTRL_NONE, + 169, UNIPHIER_PIN_DRV_4_8, + 169, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(184, "USB2VBUS", UNIPHIER_PIN_IECTRL_NONE, + 170, UNIPHIER_PIN_DRV_4_8, + 170, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(185, "USB2OD", UNIPHIER_PIN_IECTRL_NONE, + 171, UNIPHIER_PIN_DRV_4_8, + 171, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(186, "USB2ID", UNIPHIER_PIN_IECTRL_NONE, + 172, UNIPHIER_PIN_DRV_4_8, + 172, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(187, "USB3VBUS", UNIPHIER_PIN_IECTRL_NONE, + 173, UNIPHIER_PIN_DRV_4_8, + 173, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(188, "USB3OD", UNIPHIER_PIN_IECTRL_NONE, + 174, UNIPHIER_PIN_DRV_4_8, + 174, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(189, "LINKCLK", UNIPHIER_PIN_IECTRL_NONE, + 175, UNIPHIER_PIN_DRV_4_8, + 175, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(190, "LINKREQ", UNIPHIER_PIN_IECTRL_NONE, + 176, UNIPHIER_PIN_DRV_4_8, + 176, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(191, "LINKCTL0", UNIPHIER_PIN_IECTRL_NONE, + 177, UNIPHIER_PIN_DRV_4_8, + 177, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(192, "LINKCTL1", UNIPHIER_PIN_IECTRL_NONE, + 178, UNIPHIER_PIN_DRV_4_8, + 178, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(193, "LINKDT0", UNIPHIER_PIN_IECTRL_NONE, + 179, UNIPHIER_PIN_DRV_4_8, + 179, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(194, "LINKDT1", UNIPHIER_PIN_IECTRL_NONE, + 180, UNIPHIER_PIN_DRV_4_8, + 180, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(195, "LINKDT2", UNIPHIER_PIN_IECTRL_NONE, + 181, UNIPHIER_PIN_DRV_4_8, + 181, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(196, "LINKDT3", UNIPHIER_PIN_IECTRL_NONE, + 182, UNIPHIER_PIN_DRV_4_8, + 182, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(197, "LINKDT4", UNIPHIER_PIN_IECTRL_NONE, + 183, UNIPHIER_PIN_DRV_4_8, + 183, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(198, "LINKDT5", UNIPHIER_PIN_IECTRL_NONE, + 184, UNIPHIER_PIN_DRV_4_8, + 184, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(199, "LINKDT6", UNIPHIER_PIN_IECTRL_NONE, + 185, UNIPHIER_PIN_DRV_4_8, + 185, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(200, "LINKDT7", UNIPHIER_PIN_IECTRL_NONE, + 186, UNIPHIER_PIN_DRV_4_8, + 186, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(201, "CKDVO", UNIPHIER_PIN_IECTRL_NONE, + 187, UNIPHIER_PIN_DRV_4_8, + 187, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(202, "PHY_PD", UNIPHIER_PIN_IECTRL_NONE, + 188, UNIPHIER_PIN_DRV_4_8, + 188, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(203, "X1394_RST", UNIPHIER_PIN_IECTRL_NONE, + 189, UNIPHIER_PIN_DRV_4_8, + 189, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(204, "VOUT_MUTE_L", UNIPHIER_PIN_IECTRL_NONE, + 190, UNIPHIER_PIN_DRV_4_8, + 190, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(205, "CLK54O", UNIPHIER_PIN_IECTRL_NONE, + 191, UNIPHIER_PIN_DRV_4_8, + 191, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(206, "CLK54I", UNIPHIER_PIN_IECTRL_NONE, + 192, UNIPHIER_PIN_DRV_NONE, + 192, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(207, "YIN0", UNIPHIER_PIN_IECTRL_NONE, + 193, UNIPHIER_PIN_DRV_4_8, + 193, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(208, "YIN1", UNIPHIER_PIN_IECTRL_NONE, + 194, UNIPHIER_PIN_DRV_4_8, + 194, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(209, "YIN2", UNIPHIER_PIN_IECTRL_NONE, + 195, UNIPHIER_PIN_DRV_4_8, + 195, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(210, "YIN3", UNIPHIER_PIN_IECTRL_NONE, + 196, UNIPHIER_PIN_DRV_4_8, + 196, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(211, "YIN4", UNIPHIER_PIN_IECTRL_NONE, + 197, UNIPHIER_PIN_DRV_4_8, + 197, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(212, "YIN5", UNIPHIER_PIN_IECTRL_NONE, + 198, UNIPHIER_PIN_DRV_4_8, + 198, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(213, "CIN0", UNIPHIER_PIN_IECTRL_NONE, + 199, UNIPHIER_PIN_DRV_4_8, + 199, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(214, "CIN1", UNIPHIER_PIN_IECTRL_NONE, + 200, UNIPHIER_PIN_DRV_4_8, + 200, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(215, "CIN2", UNIPHIER_PIN_IECTRL_NONE, + 201, UNIPHIER_PIN_DRV_4_8, + 201, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(216, "CIN3", UNIPHIER_PIN_IECTRL_NONE, + 202, UNIPHIER_PIN_DRV_4_8, + 202, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(217, "CIN4", UNIPHIER_PIN_IECTRL_NONE, + 203, UNIPHIER_PIN_DRV_4_8, + 203, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(218, "CIN5", UNIPHIER_PIN_IECTRL_NONE, + 204, UNIPHIER_PIN_DRV_4_8, + 204, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(219, "GCP", UNIPHIER_PIN_IECTRL_NONE, + 205, UNIPHIER_PIN_DRV_4_8, + 205, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(220, "ADFLG", UNIPHIER_PIN_IECTRL_NONE, + 206, UNIPHIER_PIN_DRV_4_8, + 206, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(221, "CK27AIOF", UNIPHIER_PIN_IECTRL_NONE, + 207, UNIPHIER_PIN_DRV_4_8, + 207, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(222, "DACOUT", UNIPHIER_PIN_IECTRL_NONE, + 208, UNIPHIER_PIN_DRV_4_8, + 208, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(223, "DAFLG", UNIPHIER_PIN_IECTRL_NONE, + 209, UNIPHIER_PIN_DRV_4_8, + 209, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(224, "VBIH", UNIPHIER_PIN_IECTRL_NONE, + 210, UNIPHIER_PIN_DRV_4_8, + 210, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(225, "VBIL", UNIPHIER_PIN_IECTRL_NONE, + 211, UNIPHIER_PIN_DRV_4_8, + 211, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(226, "XSUB_RST", UNIPHIER_PIN_IECTRL_NONE, + 212, UNIPHIER_PIN_DRV_4_8, + 212, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(227, "XADC_PD", UNIPHIER_PIN_IECTRL_NONE, + 213, UNIPHIER_PIN_DRV_4_8, + 213, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(228, "AI1ADCCK", UNIPHIER_PIN_IECTRL_NONE, + 214, UNIPHIER_PIN_DRV_4_8, + 214, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(229, "AI1BCK", UNIPHIER_PIN_IECTRL_NONE, + 215, UNIPHIER_PIN_DRV_4_8, + 215, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(230, "AI1LRCK", UNIPHIER_PIN_IECTRL_NONE, + 216, UNIPHIER_PIN_DRV_4_8, + 216, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(231, "AI1DMIX", UNIPHIER_PIN_IECTRL_NONE, + 217, UNIPHIER_PIN_DRV_4_8, + 217, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(232, "CK27HD", UNIPHIER_PIN_IECTRL_NONE, + 218, UNIPHIER_PIN_DRV_4_8, + 218, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(233, "XHD_RST", UNIPHIER_PIN_IECTRL_NONE, + 219, UNIPHIER_PIN_DRV_4_8, + 219, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(234, "INTHD", UNIPHIER_PIN_IECTRL_NONE, + 220, UNIPHIER_PIN_DRV_4_8, + 220, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(235, "VO1HDCK", UNIPHIER_PIN_IECTRL_NONE, + 221, UNIPHIER_PIN_DRV_4_8, + 221, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(236, "VO1HSYNC", UNIPHIER_PIN_IECTRL_NONE, + 222, UNIPHIER_PIN_DRV_4_8, + 222, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(237, "VO1VSYNC", UNIPHIER_PIN_IECTRL_NONE, + 223, UNIPHIER_PIN_DRV_4_8, + 223, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(238, "VO1DE", UNIPHIER_PIN_IECTRL_NONE, + 224, UNIPHIER_PIN_DRV_4_8, + 224, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(239, "VO1Y0", UNIPHIER_PIN_IECTRL_NONE, + 225, UNIPHIER_PIN_DRV_4_8, + 225, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(240, "VO1Y1", UNIPHIER_PIN_IECTRL_NONE, + 226, UNIPHIER_PIN_DRV_4_8, + 226, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(241, "VO1Y2", UNIPHIER_PIN_IECTRL_NONE, + 227, UNIPHIER_PIN_DRV_4_8, + 227, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(242, "VO1Y3", UNIPHIER_PIN_IECTRL_NONE, + 228, UNIPHIER_PIN_DRV_4_8, + 228, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(243, "VO1Y4", UNIPHIER_PIN_IECTRL_NONE, + 229, UNIPHIER_PIN_DRV_4_8, + 229, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(244, "VO1Y5", UNIPHIER_PIN_IECTRL_NONE, + 230, UNIPHIER_PIN_DRV_4_8, + 230, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(245, "VO1Y6", UNIPHIER_PIN_IECTRL_NONE, + 231, UNIPHIER_PIN_DRV_4_8, + 231, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(246, "VO1Y7", UNIPHIER_PIN_IECTRL_NONE, + 232, UNIPHIER_PIN_DRV_4_8, + 232, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(247, "VO1Y8", UNIPHIER_PIN_IECTRL_NONE, + 233, UNIPHIER_PIN_DRV_4_8, + 233, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(248, "VO1Y9", UNIPHIER_PIN_IECTRL_NONE, + 234, UNIPHIER_PIN_DRV_4_8, + 234, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(249, "VO1Y10", UNIPHIER_PIN_IECTRL_NONE, + 235, UNIPHIER_PIN_DRV_4_8, + 235, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(250, "VO1Y11", UNIPHIER_PIN_IECTRL_NONE, + 236, UNIPHIER_PIN_DRV_4_8, + 236, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(251, "VO1CB0", UNIPHIER_PIN_IECTRL_NONE, + 237, UNIPHIER_PIN_DRV_4_8, + 237, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(252, "VO1CB1", UNIPHIER_PIN_IECTRL_NONE, + 238, UNIPHIER_PIN_DRV_4_8, + 238, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(253, "VO1CB2", UNIPHIER_PIN_IECTRL_NONE, + 239, UNIPHIER_PIN_DRV_4_8, + 239, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(254, "VO1CB3", UNIPHIER_PIN_IECTRL_NONE, + 240, UNIPHIER_PIN_DRV_4_8, + 240, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(255, "VO1CB4", UNIPHIER_PIN_IECTRL_NONE, + 241, UNIPHIER_PIN_DRV_4_8, + 241, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(256, "VO1CB5", UNIPHIER_PIN_IECTRL_NONE, + 242, UNIPHIER_PIN_DRV_4_8, + 242, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(257, "VO1CB6", UNIPHIER_PIN_IECTRL_NONE, + 243, UNIPHIER_PIN_DRV_4_8, + 243, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(258, "VO1CB7", UNIPHIER_PIN_IECTRL_NONE, + 244, UNIPHIER_PIN_DRV_4_8, + 244, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(259, "VO1CB8", UNIPHIER_PIN_IECTRL_NONE, + 245, UNIPHIER_PIN_DRV_4_8, + 245, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(260, "VO1CB9", UNIPHIER_PIN_IECTRL_NONE, + 246, UNIPHIER_PIN_DRV_4_8, + 246, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(261, "VO1CB10", UNIPHIER_PIN_IECTRL_NONE, + 247, UNIPHIER_PIN_DRV_4_8, + 247, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(262, "VO1CB11", UNIPHIER_PIN_IECTRL_NONE, + 248, UNIPHIER_PIN_DRV_4_8, + 248, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(263, "VO1CR0", UNIPHIER_PIN_IECTRL_NONE, + 249, UNIPHIER_PIN_DRV_4_8, + 249, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(264, "VO1CR1", UNIPHIER_PIN_IECTRL_NONE, + 250, UNIPHIER_PIN_DRV_4_8, + 250, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(265, "VO1CR2", UNIPHIER_PIN_IECTRL_NONE, + 251, UNIPHIER_PIN_DRV_4_8, + 251, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(266, "VO1CR3", UNIPHIER_PIN_IECTRL_NONE, + 252, UNIPHIER_PIN_DRV_4_8, + 252, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(267, "VO1CR4", UNIPHIER_PIN_IECTRL_NONE, + 253, UNIPHIER_PIN_DRV_4_8, + 253, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(268, "VO1CR5", UNIPHIER_PIN_IECTRL_NONE, + 254, UNIPHIER_PIN_DRV_4_8, + 254, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(269, "VO1CR6", UNIPHIER_PIN_IECTRL_NONE, + 255, UNIPHIER_PIN_DRV_4_8, + 255, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(270, "VO1CR7", UNIPHIER_PIN_IECTRL_NONE, + 256, UNIPHIER_PIN_DRV_4_8, + 256, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(271, "VO1CR8", UNIPHIER_PIN_IECTRL_NONE, + 257, UNIPHIER_PIN_DRV_4_8, + 257, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(272, "VO1CR9", UNIPHIER_PIN_IECTRL_NONE, + 258, UNIPHIER_PIN_DRV_4_8, + 258, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(273, "VO1CR10", UNIPHIER_PIN_IECTRL_NONE, + 259, UNIPHIER_PIN_DRV_4_8, + 259, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(274, "VO1CR11", UNIPHIER_PIN_IECTRL_NONE, + 260, UNIPHIER_PIN_DRV_4_8, + 260, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(275, "VO1EX0", UNIPHIER_PIN_IECTRL_NONE, + 261, UNIPHIER_PIN_DRV_4_8, + 261, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(276, "VO1EX1", UNIPHIER_PIN_IECTRL_NONE, + 262, UNIPHIER_PIN_DRV_4_8, + 262, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(277, "VO1EX2", UNIPHIER_PIN_IECTRL_NONE, + 263, UNIPHIER_PIN_DRV_4_8, + 263, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(278, "VO1EX3", UNIPHIER_PIN_IECTRL_NONE, + 264, UNIPHIER_PIN_DRV_4_8, + 264, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(279, "VEXCKA", UNIPHIER_PIN_IECTRL_NONE, + 265, UNIPHIER_PIN_DRV_4_8, + 265, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(280, "VSEL0", UNIPHIER_PIN_IECTRL_NONE, + 266, UNIPHIER_PIN_DRV_4_8, + 266, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(281, "VSEL1", UNIPHIER_PIN_IECTRL_NONE, + 267, UNIPHIER_PIN_DRV_4_8, + 267, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(282, "AO1DACCK", UNIPHIER_PIN_IECTRL_NONE, + 268, UNIPHIER_PIN_DRV_4_8, + 268, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(283, "AO1BCK", UNIPHIER_PIN_IECTRL_NONE, + 269, UNIPHIER_PIN_DRV_4_8, + 269, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(284, "AO1LRCK", UNIPHIER_PIN_IECTRL_NONE, + 270, UNIPHIER_PIN_DRV_4_8, + 270, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(285, "AO1D0", UNIPHIER_PIN_IECTRL_NONE, + 271, UNIPHIER_PIN_DRV_4_8, + 271, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(286, "AO1D1", UNIPHIER_PIN_IECTRL_NONE, + 272, UNIPHIER_PIN_DRV_4_8, + 272, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(287, "AO1D2", UNIPHIER_PIN_IECTRL_NONE, + 273, UNIPHIER_PIN_DRV_4_8, + 273, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(288, "AO1D3", UNIPHIER_PIN_IECTRL_NONE, + 274, UNIPHIER_PIN_DRV_4_8, + 274, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(289, "AO1IEC", UNIPHIER_PIN_IECTRL_NONE, + 275, UNIPHIER_PIN_DRV_4_8, + 275, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(290, "XDAC_PD", UNIPHIER_PIN_IECTRL_NONE, + 276, UNIPHIER_PIN_DRV_4_8, + 276, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(291, "EX_A_MUTE", UNIPHIER_PIN_IECTRL_NONE, + 277, UNIPHIER_PIN_DRV_4_8, + 277, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(292, "AO2DACCK", UNIPHIER_PIN_IECTRL_NONE, + 278, UNIPHIER_PIN_DRV_4_8, + 278, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(293, "AO2BCK", UNIPHIER_PIN_IECTRL_NONE, + 279, UNIPHIER_PIN_DRV_4_8, + 279, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(294, "AO2LRCK", UNIPHIER_PIN_IECTRL_NONE, + 280, UNIPHIER_PIN_DRV_4_8, + 280, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(295, "AO2DMIX", UNIPHIER_PIN_IECTRL_NONE, + 281, UNIPHIER_PIN_DRV_4_8, + 281, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(296, "AO2IEC", UNIPHIER_PIN_IECTRL_NONE, + 282, UNIPHIER_PIN_DRV_4_8, + 282, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(297, "HTHPD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_5, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(298, "HTSCL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_5, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(299, "HTSDA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_5, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(300, "PORT00", UNIPHIER_PIN_IECTRL_NONE, + 284, UNIPHIER_PIN_DRV_4_8, + 284, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(301, "PORT01", UNIPHIER_PIN_IECTRL_NONE, + 285, UNIPHIER_PIN_DRV_4_8, + 285, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(302, "PORT02", UNIPHIER_PIN_IECTRL_NONE, + 286, UNIPHIER_PIN_DRV_4_8, + 286, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(303, "PORT03", UNIPHIER_PIN_IECTRL_NONE, + 287, UNIPHIER_PIN_DRV_4_8, + 287, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(304, "PORT04", UNIPHIER_PIN_IECTRL_NONE, + 288, UNIPHIER_PIN_DRV_4_8, + 288, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(305, "PORT05", UNIPHIER_PIN_IECTRL_NONE, + 289, UNIPHIER_PIN_DRV_4_8, + 289, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(306, "PORT06", UNIPHIER_PIN_IECTRL_NONE, + 290, UNIPHIER_PIN_DRV_4_8, + 290, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(307, "PORT07", UNIPHIER_PIN_IECTRL_NONE, + 291, UNIPHIER_PIN_DRV_4_8, + 291, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(308, "PORT10", UNIPHIER_PIN_IECTRL_NONE, + 292, UNIPHIER_PIN_DRV_4_8, + 292, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(309, "PORT11", UNIPHIER_PIN_IECTRL_NONE, + 293, UNIPHIER_PIN_DRV_4_8, + 293, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(310, "PORT12", UNIPHIER_PIN_IECTRL_NONE, + 294, UNIPHIER_PIN_DRV_4_8, + 294, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(311, "PORT13", UNIPHIER_PIN_IECTRL_NONE, + 295, UNIPHIER_PIN_DRV_4_8, + 295, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(312, "PORT14", UNIPHIER_PIN_IECTRL_NONE, + 296, UNIPHIER_PIN_DRV_4_8, + 296, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(313, "PORT15", UNIPHIER_PIN_IECTRL_NONE, + 297, UNIPHIER_PIN_DRV_4_8, + 297, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(314, "PORT16", UNIPHIER_PIN_IECTRL_NONE, + 298, UNIPHIER_PIN_DRV_4_8, + 298, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(315, "PORT17", UNIPHIER_PIN_IECTRL_NONE, + 299, UNIPHIER_PIN_DRV_4_8, + 299, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(316, "PORT20", UNIPHIER_PIN_IECTRL_NONE, + 300, UNIPHIER_PIN_DRV_4_8, + 300, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(317, "PORT21", UNIPHIER_PIN_IECTRL_NONE, + 301, UNIPHIER_PIN_DRV_4_8, + 301, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(318, "PORT22", UNIPHIER_PIN_IECTRL_NONE, + 302, UNIPHIER_PIN_DRV_4_8, + 302, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(319, "SD1DAT0", UNIPHIER_PIN_IECTRL_NONE, + 303, UNIPHIER_PIN_DRV_4_8, + 303, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(320, "SD1DAT1", UNIPHIER_PIN_IECTRL_NONE, + 304, UNIPHIER_PIN_DRV_4_8, + 304, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(321, "SD1DAT2", UNIPHIER_PIN_IECTRL_NONE, + 305, UNIPHIER_PIN_DRV_4_8, + 305, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(322, "SD1DAT3", UNIPHIER_PIN_IECTRL_NONE, + 306, UNIPHIER_PIN_DRV_4_8, + 306, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(323, "SD1CMD", UNIPHIER_PIN_IECTRL_NONE, + 307, UNIPHIER_PIN_DRV_4_8, + 307, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(324, "SD1CLK", UNIPHIER_PIN_IECTRL_NONE, + 308, UNIPHIER_PIN_DRV_4_8, + 308, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(325, "SD1CD", UNIPHIER_PIN_IECTRL_NONE, + 309, UNIPHIER_PIN_DRV_4_8, + 309, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(326, "SD1WP", UNIPHIER_PIN_IECTRL_NONE, + 310, UNIPHIER_PIN_DRV_4_8, + 310, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(327, "SD1VTCG", UNIPHIER_PIN_IECTRL_NONE, + 311, UNIPHIER_PIN_DRV_4_8, + 311, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(328, "DMDISO", UNIPHIER_PIN_IECTRL_NONE, + 312, UNIPHIER_PIN_DRV_NONE, + 312, UNIPHIER_PIN_PULL_DOWN), +}; + +static const unsigned emmc_pins[] = {40, 41, 42, 43, 51, 52, 53}; +static const unsigned emmc_muxvals[] = {1, 1, 1, 1, 1, 1, 1}; +static const unsigned emmc_dat8_pins[] = {44, 45, 46, 47}; +static const unsigned emmc_dat8_muxvals[] = {1, 1, 1, 1}; +static const unsigned i2c0_pins[] = {142, 143}; +static const unsigned i2c0_muxvals[] = {0, 0}; +static const unsigned i2c1_pins[] = {144, 145}; +static const unsigned i2c1_muxvals[] = {0, 0}; +static const unsigned i2c2_pins[] = {146, 147}; +static const unsigned i2c2_muxvals[] = {0, 0}; +static const unsigned i2c3_pins[] = {148, 149}; +static const unsigned i2c3_muxvals[] = {0, 0}; +static const unsigned i2c6_pins[] = {308, 309}; +static const unsigned i2c6_muxvals[] = {6, 6}; +static const unsigned nand_pins[] = {40, 41, 42, 43, 44, 45, 46, 47, 48, 49, + 50, 51, 52, 53, 54}; +static const unsigned nand_muxvals[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0}; +static const unsigned nand_cs1_pins[] = {131, 132}; +static const unsigned nand_cs1_muxvals[] = {1, 1}; +static const unsigned uart0_pins[] = {127, 128}; +static const unsigned uart0_muxvals[] = {0, 0}; +static const unsigned uart1_pins[] = {129, 130}; +static const unsigned uart1_muxvals[] = {0, 0}; +static const unsigned uart2_pins[] = {131, 132}; +static const unsigned uart2_muxvals[] = {0, 0}; +static const unsigned uart3_pins[] = {88, 89}; +static const unsigned uart3_muxvals[] = {2, 2}; +static const unsigned usb0_pins[] = {180, 181}; +static const unsigned usb0_muxvals[] = {0, 0}; +static const unsigned usb1_pins[] = {182, 183}; +static const unsigned usb1_muxvals[] = {0, 0}; +static const unsigned usb2_pins[] = {184, 185}; +static const unsigned usb2_muxvals[] = {0, 0}; +static const unsigned usb3_pins[] = {186, 187}; +static const unsigned usb3_muxvals[] = {0, 0}; +static const unsigned port_range0_pins[] = { + 300, 301, 302, 303, 304, 305, 306, 307, /* PORT0x */ + 308, 309, 310, 311, 312, 313, 314, 315, /* PORT1x */ + 316, 317, 318, 16, 17, 18, 19, 20, /* PORT2x */ + 21, 22, 23, 4, 93, 94, 95, 63, /* PORT3x */ + 123, 122, 124, 125, 126, 141, 202, 203, /* PORT4x */ + 204, 226, 227, 290, 291, 233, 280, 281, /* PORT5x */ + 8, 7, 10, 29, 30, 48, 49, 50, /* PORT6x */ + 40, 41, 42, 43, 44, 45, 46, 47, /* PORT7x */ + 54, 51, 52, 53, 127, 128, 129, 130, /* PORT8x */ + 131, 132, 57, 60, 134, 133, 135, 136, /* PORT9x */ + 138, 137, 140, 139, 64, 65, 66, 67, /* PORT10x */ + 107, 106, 105, 104, 113, 112, 111, 110, /* PORT11x */ + 68, 69, 70, 71, 72, 73, 74, 75, /* PORT12x */ + 76, 77, 78, 79, 80, 81, 82, 83, /* PORT13x */ + 84, 85, 86, 87, 88, 89, 90, 91, /* PORT14x */ +}; +static const unsigned port_range0_muxvals[] = { + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT0x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT1x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT2x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT3x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT4x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT5x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT6x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT7x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT8x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT9x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT10x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT11x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT12x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT13x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT14x */ +}; +static const unsigned port_range1_pins[] = { + 13, 14, 15, /* PORT175-177 */ + 157, 158, 156, 154, 150, 151, 152, 153, /* PORT18x */ + 326, 327, 325, 323, 319, 320, 321, 322, /* PORT19x */ + 160, 161, 162, 163, 164, 165, 166, 167, /* PORT20x */ + 168, 169, 170, 171, 172, 173, 174, 175, /* PORT21x */ + 180, 181, 182, 183, 184, 185, 187, 188, /* PORT22x */ + 193, 194, 195, 196, 197, 198, 199, 200, /* PORT23x */ + 191, 192, 215, 216, 217, 218, 219, 220, /* PORT24x */ + 222, 223, 224, 225, 228, 229, 230, 231, /* PORT25x */ + 282, 283, 284, 285, 286, 287, 288, 289, /* PORT26x */ + 292, 293, 294, 295, 296, 236, 237, 238, /* PORT27x */ + 275, 276, 277, 278, 239, 240, 249, 250, /* PORT28x */ + 251, 252, 261, 262, 263, 264, 273, 274, /* PORT29x */ + 31, 32, 33, 34, 35, 36, 37, 38, /* PORT30x */ +}; +static const unsigned port_range1_muxvals[] = { + 7, 7, 7, /* PORT175-177 */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT18x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT19x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT20x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT21x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT22x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT23x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT24x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT25x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT26x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT27x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT28x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT29x */ + 7, 7, 7, 7, 7, 7, 7, 7, /* PORT30x */ +}; +static const unsigned xirq_pins[] = { + 11, 9, 12, 96, 97, 98, 108, 114, /* XIRQ0-7 */ + 234, 186, 99, 100, 101, 102, 184, 301, /* XIRQ8-15 */ + 302, 303, 304, 305, 306, /* XIRQ16-20 */ +}; +static const unsigned xirq_muxvals[] = { + 7, 7, 7, 7, 7, 7, 7, 7, /* XIRQ0-7 */ + 7, 7, 7, 7, 7, 7, 2, 2, /* XIRQ8-15 */ + 2, 2, 2, 2, 2, /* XIRQ16-20 */ +}; +static const unsigned xirq_alternatives_pins[] = { + 184, 310, 316, +}; +static const unsigned xirq_alternatives_muxvals[] = { + 2, 2, 2, +}; + +static const struct uniphier_pinctrl_group ph1_pro4_groups[] = { + UNIPHIER_PINCTRL_GROUP(emmc), + UNIPHIER_PINCTRL_GROUP(emmc_dat8), + UNIPHIER_PINCTRL_GROUP(i2c0), + UNIPHIER_PINCTRL_GROUP(i2c1), + UNIPHIER_PINCTRL_GROUP(i2c2), + UNIPHIER_PINCTRL_GROUP(i2c3), + UNIPHIER_PINCTRL_GROUP(i2c6), + UNIPHIER_PINCTRL_GROUP(nand), + UNIPHIER_PINCTRL_GROUP(nand_cs1), + UNIPHIER_PINCTRL_GROUP(uart0), + UNIPHIER_PINCTRL_GROUP(uart1), + UNIPHIER_PINCTRL_GROUP(uart2), + UNIPHIER_PINCTRL_GROUP(uart3), + UNIPHIER_PINCTRL_GROUP(usb0), + UNIPHIER_PINCTRL_GROUP(usb1), + UNIPHIER_PINCTRL_GROUP(usb2), + UNIPHIER_PINCTRL_GROUP(usb3), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range0), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range1), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq_alternatives), + UNIPHIER_PINCTRL_GROUP_SINGLE(port00, port_range0, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port01, port_range0, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port02, port_range0, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port03, port_range0, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port04, port_range0, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port05, port_range0, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port06, port_range0, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port07, port_range0, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port10, port_range0, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port11, port_range0, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port12, port_range0, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port13, port_range0, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port14, port_range0, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port15, port_range0, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port16, port_range0, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port17, port_range0, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port20, port_range0, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port21, port_range0, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port22, port_range0, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port23, port_range0, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port24, port_range0, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port25, port_range0, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port26, port_range0, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port27, port_range0, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port30, port_range0, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port31, port_range0, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port32, port_range0, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port33, port_range0, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port34, port_range0, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port35, port_range0, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port36, port_range0, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port37, port_range0, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port40, port_range0, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port41, port_range0, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port42, port_range0, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port43, port_range0, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port44, port_range0, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port45, port_range0, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port46, port_range0, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port47, port_range0, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port50, port_range0, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port51, port_range0, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port52, port_range0, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port53, port_range0, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port54, port_range0, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port55, port_range0, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port56, port_range0, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port57, port_range0, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port60, port_range0, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port61, port_range0, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port62, port_range0, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port63, port_range0, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port64, port_range0, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port65, port_range0, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port66, port_range0, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port67, port_range0, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port70, port_range0, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port71, port_range0, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port72, port_range0, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port73, port_range0, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port74, port_range0, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port75, port_range0, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port76, port_range0, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port77, port_range0, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port80, port_range0, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port81, port_range0, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port82, port_range0, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port83, port_range0, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port84, port_range0, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port85, port_range0, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port86, port_range0, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port87, port_range0, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port90, port_range0, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port91, port_range0, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port92, port_range0, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port93, port_range0, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port94, port_range0, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port95, port_range0, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port96, port_range0, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port97, port_range0, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port100, port_range0, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port101, port_range0, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port102, port_range0, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port103, port_range0, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port104, port_range0, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port105, port_range0, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port106, port_range0, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port107, port_range0, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port110, port_range0, 88), + UNIPHIER_PINCTRL_GROUP_SINGLE(port111, port_range0, 89), + UNIPHIER_PINCTRL_GROUP_SINGLE(port112, port_range0, 90), + UNIPHIER_PINCTRL_GROUP_SINGLE(port113, port_range0, 91), + UNIPHIER_PINCTRL_GROUP_SINGLE(port114, port_range0, 92), + UNIPHIER_PINCTRL_GROUP_SINGLE(port115, port_range0, 93), + UNIPHIER_PINCTRL_GROUP_SINGLE(port116, port_range0, 94), + UNIPHIER_PINCTRL_GROUP_SINGLE(port117, port_range0, 95), + UNIPHIER_PINCTRL_GROUP_SINGLE(port120, port_range0, 96), + UNIPHIER_PINCTRL_GROUP_SINGLE(port121, port_range0, 97), + UNIPHIER_PINCTRL_GROUP_SINGLE(port122, port_range0, 98), + UNIPHIER_PINCTRL_GROUP_SINGLE(port123, port_range0, 99), + UNIPHIER_PINCTRL_GROUP_SINGLE(port124, port_range0, 100), + UNIPHIER_PINCTRL_GROUP_SINGLE(port125, port_range0, 101), + UNIPHIER_PINCTRL_GROUP_SINGLE(port126, port_range0, 102), + UNIPHIER_PINCTRL_GROUP_SINGLE(port127, port_range0, 103), + UNIPHIER_PINCTRL_GROUP_SINGLE(port130, port_range0, 104), + UNIPHIER_PINCTRL_GROUP_SINGLE(port131, port_range0, 105), + UNIPHIER_PINCTRL_GROUP_SINGLE(port132, port_range0, 106), + UNIPHIER_PINCTRL_GROUP_SINGLE(port133, port_range0, 107), + UNIPHIER_PINCTRL_GROUP_SINGLE(port134, port_range0, 108), + UNIPHIER_PINCTRL_GROUP_SINGLE(port135, port_range0, 109), + UNIPHIER_PINCTRL_GROUP_SINGLE(port136, port_range0, 110), + UNIPHIER_PINCTRL_GROUP_SINGLE(port137, port_range0, 111), + UNIPHIER_PINCTRL_GROUP_SINGLE(port140, port_range0, 112), + UNIPHIER_PINCTRL_GROUP_SINGLE(port141, port_range0, 113), + UNIPHIER_PINCTRL_GROUP_SINGLE(port142, port_range0, 114), + UNIPHIER_PINCTRL_GROUP_SINGLE(port143, port_range0, 115), + UNIPHIER_PINCTRL_GROUP_SINGLE(port144, port_range0, 116), + UNIPHIER_PINCTRL_GROUP_SINGLE(port145, port_range0, 117), + UNIPHIER_PINCTRL_GROUP_SINGLE(port146, port_range0, 118), + UNIPHIER_PINCTRL_GROUP_SINGLE(port147, port_range0, 119), + UNIPHIER_PINCTRL_GROUP_SINGLE(port175, port_range1, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port176, port_range1, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port177, port_range1, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port180, port_range1, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port181, port_range1, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port182, port_range1, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port183, port_range1, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port184, port_range1, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port185, port_range1, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port186, port_range1, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port187, port_range1, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port190, port_range1, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port191, port_range1, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port192, port_range1, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port193, port_range1, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port194, port_range1, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port195, port_range1, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port196, port_range1, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port197, port_range1, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port200, port_range1, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port201, port_range1, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port202, port_range1, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port203, port_range1, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port204, port_range1, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port205, port_range1, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port206, port_range1, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port207, port_range1, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port210, port_range1, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port211, port_range1, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port212, port_range1, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port213, port_range1, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port214, port_range1, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port215, port_range1, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port216, port_range1, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port217, port_range1, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port220, port_range1, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port221, port_range1, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port222, port_range1, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port223, port_range1, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port224, port_range1, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port225, port_range1, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port226, port_range1, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port227, port_range1, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port230, port_range1, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port231, port_range1, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port232, port_range1, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port233, port_range1, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port234, port_range1, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port235, port_range1, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port236, port_range1, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port237, port_range1, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port240, port_range1, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port241, port_range1, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port242, port_range1, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port243, port_range1, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port244, port_range1, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port245, port_range1, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port246, port_range1, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port247, port_range1, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port250, port_range1, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port251, port_range1, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port252, port_range1, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port253, port_range1, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port254, port_range1, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port255, port_range1, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port256, port_range1, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port257, port_range1, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port260, port_range1, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port261, port_range1, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port262, port_range1, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port263, port_range1, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port264, port_range1, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port265, port_range1, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port266, port_range1, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port267, port_range1, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port270, port_range1, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port271, port_range1, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port272, port_range1, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port273, port_range1, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port274, port_range1, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port275, port_range1, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port276, port_range1, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port277, port_range1, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port280, port_range1, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port281, port_range1, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port282, port_range1, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port283, port_range1, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port284, port_range1, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port285, port_range1, 88), + UNIPHIER_PINCTRL_GROUP_SINGLE(port286, port_range1, 89), + UNIPHIER_PINCTRL_GROUP_SINGLE(port287, port_range1, 90), + UNIPHIER_PINCTRL_GROUP_SINGLE(port290, port_range1, 91), + UNIPHIER_PINCTRL_GROUP_SINGLE(port291, port_range1, 92), + UNIPHIER_PINCTRL_GROUP_SINGLE(port292, port_range1, 93), + UNIPHIER_PINCTRL_GROUP_SINGLE(port293, port_range1, 94), + UNIPHIER_PINCTRL_GROUP_SINGLE(port294, port_range1, 95), + UNIPHIER_PINCTRL_GROUP_SINGLE(port295, port_range1, 96), + UNIPHIER_PINCTRL_GROUP_SINGLE(port296, port_range1, 97), + UNIPHIER_PINCTRL_GROUP_SINGLE(port297, port_range1, 98), + UNIPHIER_PINCTRL_GROUP_SINGLE(port300, port_range1, 99), + UNIPHIER_PINCTRL_GROUP_SINGLE(port301, port_range1, 100), + UNIPHIER_PINCTRL_GROUP_SINGLE(port302, port_range1, 101), + UNIPHIER_PINCTRL_GROUP_SINGLE(port303, port_range1, 102), + UNIPHIER_PINCTRL_GROUP_SINGLE(port304, port_range1, 103), + UNIPHIER_PINCTRL_GROUP_SINGLE(port305, port_range1, 104), + UNIPHIER_PINCTRL_GROUP_SINGLE(port306, port_range1, 105), + UNIPHIER_PINCTRL_GROUP_SINGLE(port307, port_range1, 106), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq0, xirq, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq1, xirq, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq2, xirq, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq3, xirq, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq4, xirq, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq5, xirq, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq6, xirq, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq7, xirq, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq8, xirq, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq9, xirq, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq10, xirq, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq11, xirq, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq12, xirq, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq13, xirq, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq14, xirq, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq15, xirq, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq16, xirq, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq17, xirq, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq18, xirq, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq19, xirq, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq20, xirq, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq14b, xirq_alternatives, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq17b, xirq_alternatives, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq18b, xirq_alternatives, 2), +}; + +static const char * const emmc_groups[] = {"emmc", "emmc_dat8"}; +static const char * const i2c0_groups[] = {"i2c0"}; +static const char * const i2c1_groups[] = {"i2c1"}; +static const char * const i2c2_groups[] = {"i2c2"}; +static const char * const i2c3_groups[] = {"i2c3"}; +static const char * const i2c6_groups[] = {"i2c6"}; +static const char * const nand_groups[] = {"nand", "nand_cs1"}; +static const char * const uart0_groups[] = {"uart0"}; +static const char * const uart1_groups[] = {"uart1"}; +static const char * const uart2_groups[] = {"uart2"}; +static const char * const uart3_groups[] = {"uart3"}; +static const char * const usb0_groups[] = {"usb0"}; +static const char * const usb1_groups[] = {"usb1"}; +static const char * const usb2_groups[] = {"usb2"}; +static const char * const usb3_groups[] = {"usb3"}; +static const char * const port_groups[] = { + "port00", "port01", "port02", "port03", + "port04", "port05", "port06", "port07", + "port10", "port11", "port12", "port13", + "port14", "port15", "port16", "port17", + "port20", "port21", "port22", "port23", + "port24", "port25", "port26", "port27", + "port30", "port31", "port32", "port33", + "port34", "port35", "port36", "port37", + "port40", "port41", "port42", "port43", + "port44", "port45", "port46", "port47", + "port50", "port51", "port52", "port53", + "port54", "port55", "port56", "port57", + "port60", "port61", "port62", "port63", + "port64", "port65", "port66", "port67", + "port70", "port71", "port72", "port73", + "port74", "port75", "port76", "port77", + "port80", "port81", "port82", "port83", + "port84", "port85", "port86", "port87", + "port90", "port91", "port92", "port93", + "port94", "port95", "port96", "port97", + "port100", "port101", "port102", "port103", + "port104", "port105", "port106", "port107", + "port110", "port111", "port112", "port113", + "port114", "port115", "port116", "port117", + "port120", "port121", "port122", "port123", + "port124", "port125", "port126", "port127", + "port130", "port131", "port132", "port133", + "port134", "port135", "port136", "port137", + "port140", "port141", "port142", "port143", + "port144", "port145", "port146", "port147", + /* port150-174 missing */ + /* none */ "port175", "port176", "port177", + "port180", "port181", "port182", "port183", + "port184", "port185", "port186", "port187", + "port190", "port191", "port192", "port193", + "port194", "port195", "port196", "port197", + "port200", "port201", "port202", "port203", + "port204", "port205", "port206", "port207", + "port210", "port211", "port212", "port213", + "port214", "port215", "port216", "port217", + "port220", "port221", "port222", "port223", + "port224", "port225", "port226", "port227", + "port230", "port231", "port232", "port233", + "port234", "port235", "port236", "port237", + "port240", "port241", "port242", "port243", + "port244", "port245", "port246", "port247", + "port250", "port251", "port252", "port253", + "port254", "port255", "port256", "port257", + "port260", "port261", "port262", "port263", + "port264", "port265", "port266", "port267", + "port270", "port271", "port272", "port273", + "port274", "port275", "port276", "port277", + "port280", "port281", "port282", "port283", + "port284", "port285", "port286", "port287", + "port290", "port291", "port292", "port293", + "port294", "port295", "port296", "port297", + "port300", "port301", "port302", "port303", + "port304", "port305", "port306", "port307", +}; +static const char * const xirq_groups[] = { + "xirq0", "xirq1", "xirq2", "xirq3", + "xirq4", "xirq5", "xirq6", "xirq7", + "xirq8", "xirq9", "xirq10", "xirq11", + "xirq12", "xirq13", "xirq14", "xirq15", + "xirq16", "xirq17", "xirq18", "xirq19", + "xirq20", + "xirq14b", "xirq17b", "xirq18b", +}; + +static const struct uniphier_pinmux_function ph1_pro4_functions[] = { + UNIPHIER_PINMUX_FUNCTION(emmc), + UNIPHIER_PINMUX_FUNCTION(i2c0), + UNIPHIER_PINMUX_FUNCTION(i2c1), + UNIPHIER_PINMUX_FUNCTION(i2c2), + UNIPHIER_PINMUX_FUNCTION(i2c3), + UNIPHIER_PINMUX_FUNCTION(i2c6), + UNIPHIER_PINMUX_FUNCTION(nand), + UNIPHIER_PINMUX_FUNCTION(uart0), + UNIPHIER_PINMUX_FUNCTION(uart1), + UNIPHIER_PINMUX_FUNCTION(uart2), + UNIPHIER_PINMUX_FUNCTION(uart3), + UNIPHIER_PINMUX_FUNCTION(usb0), + UNIPHIER_PINMUX_FUNCTION(usb1), + UNIPHIER_PINMUX_FUNCTION(usb2), + UNIPHIER_PINMUX_FUNCTION(usb3), + UNIPHIER_PINMUX_FUNCTION(port), + UNIPHIER_PINMUX_FUNCTION(xirq), +}; + +static struct uniphier_pinctrl_socdata ph1_pro4_pindata = { + .groups = ph1_pro4_groups, + .groups_count = ARRAY_SIZE(ph1_pro4_groups), + .functions = ph1_pro4_functions, + .functions_count = ARRAY_SIZE(ph1_pro4_functions), + .mux_bits = 4, + .reg_stride = 8, + .load_pinctrl = true, +}; + +static struct pinctrl_desc ph1_pro4_pinctrl_desc = { + .name = DRIVER_NAME, + .pins = ph1_pro4_pins, + .npins = ARRAY_SIZE(ph1_pro4_pins), + .owner = THIS_MODULE, +}; + +static int ph1_pro4_pinctrl_probe(struct platform_device *pdev) +{ + return uniphier_pinctrl_probe(pdev, &ph1_pro4_pinctrl_desc, + &ph1_pro4_pindata); +} + +static const struct of_device_id ph1_pro4_pinctrl_match[] = { + { .compatible = "socionext,ph1-pro4-pinctrl" }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, ph1_pro4_pinctrl_match); + +static struct platform_driver ph1_pro4_pinctrl_driver = { + .probe = ph1_pro4_pinctrl_probe, + .remove = uniphier_pinctrl_remove, + .driver = { + .name = DRIVER_NAME, + .of_match_table = ph1_pro4_pinctrl_match, + }, +}; +module_platform_driver(ph1_pro4_pinctrl_driver); + +MODULE_AUTHOR("Masahiro Yamada "); +MODULE_DESCRIPTION("UniPhier PH1-Pro4 pinctrl driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pinctrl/uniphier/pinctrl-ph1-pro5.c b/drivers/pinctrl/uniphier/pinctrl-ph1-pro5.c new file mode 100644 index 00000000000000..9af455978058cd --- /dev/null +++ b/drivers/pinctrl/uniphier/pinctrl-ph1-pro5.c @@ -0,0 +1,1351 @@ +/* + * Copyright (C) 2015 Masahiro Yamada + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program5 is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include +#include +#include + +#include "pinctrl-uniphier.h" + +#define DRIVER_NAME "ph1-pro5-pinctrl" + +static const struct pinctrl_pin_desc ph1_pro5_pins[] = { + UNIPHIER_PINCTRL_PIN(0, "AEXCKA1", 0, + 0, UNIPHIER_PIN_DRV_4_8, + 0, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(1, "AEXCKA2", 0, + 1, UNIPHIER_PIN_DRV_4_8, + 1, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(2, "CK27EXI", 0, + 2, UNIPHIER_PIN_DRV_4_8, + 2, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(3, "CK54EXI", 0, + 3, UNIPHIER_PIN_DRV_4_8, + 3, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(4, "ED0", UNIPHIER_PIN_IECTRL_NONE, + 4, UNIPHIER_PIN_DRV_4_8, + 4, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(5, "ED1", UNIPHIER_PIN_IECTRL_NONE, + 5, UNIPHIER_PIN_DRV_4_8, + 5, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(6, "ED2", UNIPHIER_PIN_IECTRL_NONE, + 6, UNIPHIER_PIN_DRV_4_8, + 6, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(7, "ED3", UNIPHIER_PIN_IECTRL_NONE, + 7, UNIPHIER_PIN_DRV_4_8, + 7, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(8, "ED4", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_4_8, + 8, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(9, "ED5", UNIPHIER_PIN_IECTRL_NONE, + 9, UNIPHIER_PIN_DRV_4_8, + 9, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(10, "ED6", UNIPHIER_PIN_IECTRL_NONE, + 10, UNIPHIER_PIN_DRV_4_8, + 10, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(11, "ED7", UNIPHIER_PIN_IECTRL_NONE, + 11, UNIPHIER_PIN_DRV_4_8, + 11, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(12, "XERWE0", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_4_8, + 12, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(13, "XERWE1", UNIPHIER_PIN_IECTRL_NONE, + 13, UNIPHIER_PIN_DRV_4_8, + 13, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(14, "ERXW", UNIPHIER_PIN_IECTRL_NONE, + 14, UNIPHIER_PIN_DRV_4_8, + 14, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(15, "ES0", UNIPHIER_PIN_IECTRL_NONE, + 15, UNIPHIER_PIN_DRV_4_8, + 15, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(16, "ES1", UNIPHIER_PIN_IECTRL_NONE, + 16, UNIPHIER_PIN_DRV_4_8, + 16, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(17, "ES2", UNIPHIER_PIN_IECTRL_NONE, + 17, UNIPHIER_PIN_DRV_4_8, + 17, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(18, "XECS1", UNIPHIER_PIN_IECTRL_NONE, + 18, UNIPHIER_PIN_DRV_4_8, + 18, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(19, "XNFRE", UNIPHIER_PIN_IECTRL_NONE, + 19, UNIPHIER_PIN_DRV_4_8, + 19, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(20, "XNFWE", UNIPHIER_PIN_IECTRL_NONE, + 20, UNIPHIER_PIN_DRV_4_8, + 20, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(21, "NFALE", UNIPHIER_PIN_IECTRL_NONE, + 21, UNIPHIER_PIN_DRV_4_8, + 21, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(22, "NFCLE", UNIPHIER_PIN_IECTRL_NONE, + 22, UNIPHIER_PIN_DRV_4_8, + 22, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(23, "XNFWP", UNIPHIER_PIN_IECTRL_NONE, + 23, UNIPHIER_PIN_DRV_4_8, + 23, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(24, "XNFCE0", UNIPHIER_PIN_IECTRL_NONE, + 24, UNIPHIER_PIN_DRV_4_8, + 24, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(25, "NFRYBY0", UNIPHIER_PIN_IECTRL_NONE, + 25, UNIPHIER_PIN_DRV_4_8, + 25, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(26, "XNFCE1", UNIPHIER_PIN_IECTRL_NONE, + 26, UNIPHIER_PIN_DRV_4_8, + 26, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(27, "NFRYBY1", UNIPHIER_PIN_IECTRL_NONE, + 27, UNIPHIER_PIN_DRV_4_8, + 27, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(28, "NFD0", UNIPHIER_PIN_IECTRL_NONE, + 28, UNIPHIER_PIN_DRV_4_8, + 28, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(29, "NFD1", UNIPHIER_PIN_IECTRL_NONE, + 29, UNIPHIER_PIN_DRV_4_8, + 29, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(30, "NFD2", UNIPHIER_PIN_IECTRL_NONE, + 30, UNIPHIER_PIN_DRV_4_8, + 30, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(31, "NFD3", UNIPHIER_PIN_IECTRL_NONE, + 31, UNIPHIER_PIN_DRV_4_8, + 31, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(32, "NFD4", UNIPHIER_PIN_IECTRL_NONE, + 32, UNIPHIER_PIN_DRV_4_8, + 32, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(33, "NFD5", UNIPHIER_PIN_IECTRL_NONE, + 33, UNIPHIER_PIN_DRV_4_8, + 33, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(34, "NFD6", UNIPHIER_PIN_IECTRL_NONE, + 34, UNIPHIER_PIN_DRV_4_8, + 34, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(35, "NFD7", UNIPHIER_PIN_IECTRL_NONE, + 35, UNIPHIER_PIN_DRV_4_8, + 35, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(36, "XERST", UNIPHIER_PIN_IECTRL_NONE, + 36, UNIPHIER_PIN_DRV_4_8, + 36, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(37, "MMCCLK", UNIPHIER_PIN_IECTRL_NONE, + 37, UNIPHIER_PIN_DRV_4_8, + 37, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(38, "MMCCMD", UNIPHIER_PIN_IECTRL_NONE, + 38, UNIPHIER_PIN_DRV_4_8, + 38, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(39, "MMCDAT0", UNIPHIER_PIN_IECTRL_NONE, + 39, UNIPHIER_PIN_DRV_4_8, + 39, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(40, "MMCDAT1", UNIPHIER_PIN_IECTRL_NONE, + 40, UNIPHIER_PIN_DRV_4_8, + 40, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(41, "MMCDAT2", UNIPHIER_PIN_IECTRL_NONE, + 41, UNIPHIER_PIN_DRV_4_8, + 41, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(42, "MMCDAT3", UNIPHIER_PIN_IECTRL_NONE, + 42, UNIPHIER_PIN_DRV_4_8, + 42, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(43, "MMCDAT4", UNIPHIER_PIN_IECTRL_NONE, + 43, UNIPHIER_PIN_DRV_4_8, + 43, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(44, "MMCDAT5", UNIPHIER_PIN_IECTRL_NONE, + 44, UNIPHIER_PIN_DRV_4_8, + 44, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(45, "MMCDAT6", UNIPHIER_PIN_IECTRL_NONE, + 45, UNIPHIER_PIN_DRV_4_8, + 45, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(46, "MMCDAT7", UNIPHIER_PIN_IECTRL_NONE, + 46, UNIPHIER_PIN_DRV_4_8, + 46, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(47, "TXD0", 0, + 47, UNIPHIER_PIN_DRV_4_8, + 47, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(48, "RXD0", 0, + 48, UNIPHIER_PIN_DRV_4_8, + 48, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(49, "TXD1", 0, + 49, UNIPHIER_PIN_DRV_4_8, + 49, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(50, "RXD1", 0, + 50, UNIPHIER_PIN_DRV_4_8, + 50, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(51, "TXD2", UNIPHIER_PIN_IECTRL_NONE, + 51, UNIPHIER_PIN_DRV_4_8, + 51, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(52, "RXD2", UNIPHIER_PIN_IECTRL_NONE, + 52, UNIPHIER_PIN_DRV_4_8, + 52, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(53, "TXD3", 0, + 53, UNIPHIER_PIN_DRV_4_8, + 53, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(54, "RXD3", 0, + 54, UNIPHIER_PIN_DRV_4_8, + 54, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(55, "MS0CS0", 0, + 55, UNIPHIER_PIN_DRV_4_8, + 55, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(56, "MS0DO", 0, + 56, UNIPHIER_PIN_DRV_4_8, + 56, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(57, "MS0DI", 0, + 57, UNIPHIER_PIN_DRV_4_8, + 57, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(58, "MS0CLK", 0, + 58, UNIPHIER_PIN_DRV_4_8, + 58, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(59, "CSCLK", 0, + 59, UNIPHIER_PIN_DRV_4_8, + 59, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(60, "CSBPTM", 0, + 60, UNIPHIER_PIN_DRV_4_8, + 60, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(61, "CSBMTP", 0, + 61, UNIPHIER_PIN_DRV_4_8, + 61, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(62, "XCINTP", 0, + 62, UNIPHIER_PIN_DRV_4_8, + 62, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(63, "XCINTM", 0, + 63, UNIPHIER_PIN_DRV_4_8, + 63, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(64, "XCMPREQ", 0, + 64, UNIPHIER_PIN_DRV_4_8, + 64, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(65, "XSRST", 0, + 65, UNIPHIER_PIN_DRV_4_8, + 65, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(66, "LPST", UNIPHIER_PIN_IECTRL_NONE, + 66, UNIPHIER_PIN_DRV_4_8, + 66, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(67, "PWMA", 0, + 67, UNIPHIER_PIN_DRV_4_8, + 67, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(68, "XIRQ0", 0, + 68, UNIPHIER_PIN_DRV_4_8, + 68, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(69, "XIRQ1", 0, + 69, UNIPHIER_PIN_DRV_4_8, + 69, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(70, "XIRQ2", 0, + 70, UNIPHIER_PIN_DRV_4_8, + 70, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(71, "XIRQ3", 0, + 71, UNIPHIER_PIN_DRV_4_8, + 71, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(72, "XIRQ4", 0, + 72, UNIPHIER_PIN_DRV_4_8, + 72, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(73, "XIRQ5", 0, + 73, UNIPHIER_PIN_DRV_4_8, + 73, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(74, "XIRQ6", 0, + 74, UNIPHIER_PIN_DRV_4_8, + 74, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(75, "XIRQ7", 0, + 75, UNIPHIER_PIN_DRV_4_8, + 75, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(76, "XIRQ8", 0, + 76, UNIPHIER_PIN_DRV_4_8, + 76, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(77, "XIRQ9", 0, + 77, UNIPHIER_PIN_DRV_4_8, + 77, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(78, "XIRQ10", 0, + 78, UNIPHIER_PIN_DRV_4_8, + 78, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(79, "XIRQ11", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 79, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(80, "XIRQ12", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 80, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(81, "XIRQ13", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 81, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(82, "XIRQ14", 0, + 82, UNIPHIER_PIN_DRV_4_8, + 82, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(83, "XIRQ15", 0, + 83, UNIPHIER_PIN_DRV_4_8, + 83, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(84, "XIRQ16", 0, + 84, UNIPHIER_PIN_DRV_4_8, + 84, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(85, "XIRQ17", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 85, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(86, "XIRQ18", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 86, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(87, "XIRQ19", 0, + 87, UNIPHIER_PIN_DRV_4_8, + 87, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(88, "XIRQ20", 0, + 88, UNIPHIER_PIN_DRV_4_8, + 88, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(89, "PORT00", 0, + 89, UNIPHIER_PIN_DRV_4_8, + 89, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(90, "PORT01", 0, + 90, UNIPHIER_PIN_DRV_4_8, + 90, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(91, "PORT02", 0, + 91, UNIPHIER_PIN_DRV_4_8, + 91, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(92, "PORT03", 0, + 92, UNIPHIER_PIN_DRV_4_8, + 92, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(93, "PORT04", 0, + 93, UNIPHIER_PIN_DRV_4_8, + 93, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(94, "PORT05", 0, + 94, UNIPHIER_PIN_DRV_4_8, + 94, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(95, "PORT06", 0, + 95, UNIPHIER_PIN_DRV_4_8, + 95, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(96, "PORT07", 0, + 96, UNIPHIER_PIN_DRV_4_8, + 96, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(97, "PORT10", 0, + 97, UNIPHIER_PIN_DRV_4_8, + 97, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(98, "PORT11", 0, + 98, UNIPHIER_PIN_DRV_4_8, + 98, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(99, "PORT12", 0, + 99, UNIPHIER_PIN_DRV_4_8, + 99, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(100, "PORT13", 0, + 100, UNIPHIER_PIN_DRV_4_8, + 100, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(101, "PORT14", 0, + 101, UNIPHIER_PIN_DRV_4_8, + 101, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(102, "PORT15", 0, + 102, UNIPHIER_PIN_DRV_4_8, + 102, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(103, "PORT16", 0, + 103, UNIPHIER_PIN_DRV_4_8, + 103, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(104, "PORT17", 0, + 104, UNIPHIER_PIN_DRV_4_8, + 104, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(105, "T0HPD", 0, + 105, UNIPHIER_PIN_DRV_4_8, + 105, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(106, "T1HPD", 0, + 106, UNIPHIER_PIN_DRV_4_8, + 106, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(107, "R0HPD", 0, + 107, UNIPHIER_PIN_DRV_4_8, + 107, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(108, "R1HPD", 0, + 108, UNIPHIER_PIN_DRV_4_8, + 108, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(109, "XPERST", 0, + 109, UNIPHIER_PIN_DRV_4_8, + 109, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(110, "XPEWAKE", 0, + 110, UNIPHIER_PIN_DRV_4_8, + 110, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(111, "XPECLKRQ", 0, + 111, UNIPHIER_PIN_DRV_4_8, + 111, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(112, "SDA0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 112, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(113, "SCL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 113, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(114, "SDA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 114, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(115, "SCL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 115, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(116, "SDA2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 116, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(117, "SCL2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 117, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(118, "SDA3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 118, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(119, "SCL3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 119, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(120, "SPISYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 120, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(121, "SPISCLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 121, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(122, "SPITXD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 122, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(123, "SPIRXD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 123, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(124, "USB0VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 124, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(125, "USB0OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 125, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(126, "USB1VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 126, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(127, "USB1OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 127, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(128, "USB2VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 128, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(129, "USB2OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 129, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(130, "SMTRST0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 130, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(131, "SMTCMD0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 131, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(132, "SMTD0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 132, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(133, "SMTSEL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 133, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(134, "SMTCLK0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 134, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(135, "SMTRST1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 135, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(136, "SMTCMD1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 136, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(137, "SMTD1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 137, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(138, "SMTSEL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 138, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(139, "SMTCLK1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 139, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(140, "CH0CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 140, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(141, "CH0PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 141, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(142, "CH0VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 142, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(143, "CH0DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 143, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(144, "CH1CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 144, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(145, "CH1PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 145, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(146, "CH1VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 146, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(147, "CH1DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 147, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(148, "CH2CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 148, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(149, "CH2PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 149, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(150, "CH2VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 150, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(151, "CH2DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 151, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(152, "CH3CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 152, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(153, "CH3PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 153, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(154, "CH3VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 154, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(155, "CH3DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 155, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(156, "CH4CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 156, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(157, "CH4PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 157, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(158, "CH4VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 158, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(159, "CH4DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 159, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(160, "CH5CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 160, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(161, "CH5PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 161, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(162, "CH5VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 162, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(163, "CH5DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 163, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(164, "CH6CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 164, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(165, "CH6PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 165, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(166, "CH6VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 166, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(167, "CH6DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 167, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(168, "CH7CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 168, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(169, "CH7PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 169, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(170, "CH7VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 170, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(171, "CH7DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 171, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(172, "AI1ADCCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 172, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(173, "AI1BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 173, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(174, "AI1LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 174, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(175, "AI1D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 175, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(176, "AI1D1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 176, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(177, "AI1D2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 177, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(178, "AI1D3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 178, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(179, "AI2ADCCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 179, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(180, "AI2BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 180, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(181, "AI2LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 181, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(182, "AI2D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 182, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(183, "AI2D1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 183, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(184, "AI2D2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 184, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(185, "AI2D3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 185, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(186, "AI3ADCCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 186, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(187, "AI3BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 187, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(188, "AI3LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 188, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(189, "AI3D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 189, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(190, "AO1IEC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 190, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(191, "AO1DACCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 191, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(192, "AO1BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 192, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(193, "AO1LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 193, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(194, "AO1D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 194, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(195, "AO1D1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 195, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(196, "AO1D2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 196, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(197, "AO1D3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 197, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(198, "AO2IEC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 198, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(199, "AO2DACCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 199, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(200, "AO2BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 200, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(201, "AO2LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 201, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(202, "AO2D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 202, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(203, "AO2D1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 203, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(204, "AO2D2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 204, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(205, "AO2D3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 205, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(206, "AO3DACCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 206, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(207, "AO3BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 207, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(208, "AO3LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 208, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(209, "AO3DMIX", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 209, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(210, "AO4DACCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 210, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(211, "AO4BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 211, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(212, "AO4LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 212, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(213, "AO4DMIX", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 213, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(214, "VI1CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 214, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(215, "VI1C0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 215, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(216, "VI1C1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 216, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(217, "VI1C2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 217, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(218, "VI1C3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 218, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(219, "VI1C4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 219, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(220, "VI1C5", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 220, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(221, "VI1C6", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 221, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(222, "VI1C7", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 222, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(223, "VI1C8", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 223, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(224, "VI1C9", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 224, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(225, "VI1Y0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 225, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(226, "VI1Y1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 226, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(227, "VI1Y2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 227, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(228, "VI1Y3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 228, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(229, "VI1Y4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 229, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(230, "VI1Y5", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 230, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(231, "VI1Y6", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 231, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(232, "VI1Y7", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 232, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(233, "VI1Y8", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 233, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(234, "VI1Y9", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 234, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(235, "VI1DE", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 235, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(236, "VI1HSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 236, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(237, "VI1VSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 237, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(238, "VO1CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 238, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(239, "VO1D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 239, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(240, "VO1D1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 240, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(241, "VO1D2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 241, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(242, "VO1D3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 242, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(243, "VO1D4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 243, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(244, "VO1D5", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 244, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(245, "VO1D6", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 245, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(246, "VO1D7", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 246, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(247, "SDCD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 247, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(248, "SDWP", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 248, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(249, "SDVOLC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 249, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(250, "SDCLK", UNIPHIER_PIN_IECTRL_NONE, + 40, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(251, "SDCMD", UNIPHIER_PIN_IECTRL_NONE, + 44, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(252, "SDDAT0", UNIPHIER_PIN_IECTRL_NONE, + 48, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(253, "SDDAT1", UNIPHIER_PIN_IECTRL_NONE, + 52, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(254, "SDDAT2", UNIPHIER_PIN_IECTRL_NONE, + 56, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(255, "SDDAT3", UNIPHIER_PIN_IECTRL_NONE, + 60, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), +}; + +static const unsigned emmc_pins[] = {36, 37, 38, 39, 40, 41, 42}; +static const unsigned emmc_muxvals[] = {0, 0, 0, 0, 0, 0, 0}; +static const unsigned emmc_dat8_pins[] = {43, 44, 45, 46}; +static const unsigned emmc_dat8_muxvals[] = {0, 0, 0, 0}; +static const unsigned i2c0_pins[] = {112, 113}; +static const unsigned i2c0_muxvals[] = {0, 0}; +static const unsigned i2c1_pins[] = {114, 115}; +static const unsigned i2c1_muxvals[] = {0, 0}; +static const unsigned i2c2_pins[] = {116, 117}; +static const unsigned i2c2_muxvals[] = {0, 0}; +static const unsigned i2c3_pins[] = {118, 119}; +static const unsigned i2c3_muxvals[] = {0, 0}; +static const unsigned i2c5_pins[] = {87, 88}; +static const unsigned i2c5_muxvals[] = {2, 2}; +static const unsigned i2c5b_pins[] = {196, 197}; +static const unsigned i2c5b_muxvals[] = {2, 2}; +static const unsigned i2c5c_pins[] = {215, 216}; +static const unsigned i2c5c_muxvals[] = {2, 2}; +static const unsigned i2c6_pins[] = {101, 102}; +static const unsigned i2c6_muxvals[] = {2, 2}; +static const unsigned nand_pins[] = {19, 20, 21, 22, 23, 24, 25, 28, 29, 30, + 31, 32, 33, 34, 35}; +static const unsigned nand_muxvals[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0}; +static const unsigned nand_cs1_pins[] = {26, 27}; +static const unsigned nand_cs1_muxvals[] = {0, 0}; +static const unsigned uart0_pins[] = {47, 48}; +static const unsigned uart0_muxvals[] = {0, 0}; +static const unsigned uart0b_pins[] = {227, 228}; +static const unsigned uart0b_muxvals[] = {3, 3}; +static const unsigned uart1_pins[] = {49, 50}; +static const unsigned uart1_muxvals[] = {0, 0}; +static const unsigned uart2_pins[] = {51, 52}; +static const unsigned uart2_muxvals[] = {0, 0}; +static const unsigned uart3_pins[] = {53, 54}; +static const unsigned uart3_muxvals[] = {0, 0}; +static const unsigned usb0_pins[] = {124, 125}; +static const unsigned usb0_muxvals[] = {0, 0}; +static const unsigned usb1_pins[] = {126, 127}; +static const unsigned usb1_muxvals[] = {0, 0}; +static const unsigned usb2_pins[] = {128, 129}; +static const unsigned usb2_muxvals[] = {0, 0}; +static const unsigned port_range0_pins[] = { + 89, 90, 91, 92, 93, 94, 95, 96, /* PORT0x */ + 97, 98, 99, 100, 101, 102, 103, 104, /* PORT1x */ + 251, 252, 253, 254, 255, 247, 248, 249, /* PORT2x */ + 39, 40, 41, 42, 43, 44, 45, 46, /* PORT3x */ + 156, 157, 158, 159, 160, 161, 162, 163, /* PORT4x */ + 164, 165, 166, 167, 168, 169, 170, 171, /* PORT5x */ + 190, 191, 192, 193, 194, 195, 196, 197, /* PORT6x */ + 198, 199, 200, 201, 202, 203, 204, 205, /* PORT7x */ + 120, 121, 122, 123, 55, 56, 57, 58, /* PORT8x */ + 124, 125, 126, 127, 49, 50, 53, 54, /* PORT9x */ + 148, 149, 150, 151, 152, 153, 154, 155, /* PORT10x */ + 133, 134, 131, 130, 138, 139, 136, 135, /* PORT11x */ + 28, 29, 30, 31, 32, 33, 34, 35, /* PORT12x */ + 179, 180, 181, 182, 186, 187, 188, 189, /* PORT13x */ + 4, 5, 6, 7, 8, 9, 10, 11, /* PORT14x */ +}; +static const unsigned port_range0_muxvals[] = { + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT0x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT1x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT2x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT3x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT4x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT5x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT6x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT7x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT8x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT9x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT10x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT11x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT12x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT13x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT14x */ +}; +static const unsigned port_range1_pins[] = { + 109, 110, 111, /* PORT175-177 */ + 206, 207, 208, 209, 210, 211, 212, 213, /* PORT18x */ + 12, 13, 14, 15, 16, 17, 107, 108, /* PORT19x */ + 140, 141, 142, 143, 144, 145, 146, 147, /* PORT20x */ + 59, 60, 61, 62, 63, 64, 65, 66, /* PORT21x */ + 214, 215, 216, 217, 218, 219, 220, 221, /* PORT22x */ + 222, 223, 224, 225, 226, 227, 228, 229, /* PORT23x */ + 19, 20, 21, 22, 23, 24, 25, 26, /* PORT24x */ + 230, 231, 232, 233, 234, 235, 236, 237, /* PORT25x */ + 239, 240, 241, 242, 243, 244, 245, 246, /* PORT26x */ + 172, 173, 174, 175, 176, 177, 178, 129, /* PORT27x */ + 0, 1, 2, 67, 85, 86, 87, 88, /* PORT28x */ + 105, 106, 18, 27, 36, 128, 132, 137, /* PORT29x */ + 183, 184, 185, 84, 47, 48, 51, 52, /* PORT30x */ +}; +static const unsigned port_range1_muxvals[] = { + 15, 15, 15, /* PORT175-177 */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT18x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT19x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT20x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT21x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT22x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT23x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT24x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT25x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT26x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT27x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT28x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT29x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT30x */ +}; +static const unsigned xirq_pins[] = { + 68, 69, 70, 71, 72, 73, 74, 75, /* XIRQ0-7 */ + 76, 77, 78, 79, 80, 81, 82, 83, /* XIRQ8-15 */ + 84, 85, 86, 87, 88, /* XIRQ16-20 */ +}; +static const unsigned xirq_muxvals[] = { + 14, 14, 14, 14, 14, 14, 14, 14, /* XIRQ0-7 */ + 14, 14, 14, 14, 14, 14, 14, 14, /* XIRQ8-15 */ + 14, 14, 14, 14, 14, /* XIRQ16-20 */ +}; +static const unsigned xirq_alternatives_pins[] = { + 91, 92, 239, 144, 240, 156, 241, 106, 128, +}; +static const unsigned xirq_alternatives_muxvals[] = { + 14, 14, 14, 14, 14, 14, 14, 14, 14, +}; + +static const struct uniphier_pinctrl_group ph1_pro5_groups[] = { + UNIPHIER_PINCTRL_GROUP(nand), + UNIPHIER_PINCTRL_GROUP(nand_cs1), + UNIPHIER_PINCTRL_GROUP(emmc), + UNIPHIER_PINCTRL_GROUP(emmc_dat8), + UNIPHIER_PINCTRL_GROUP(i2c0), + UNIPHIER_PINCTRL_GROUP(i2c1), + UNIPHIER_PINCTRL_GROUP(i2c2), + UNIPHIER_PINCTRL_GROUP(i2c3), + UNIPHIER_PINCTRL_GROUP(i2c5), + UNIPHIER_PINCTRL_GROUP(i2c5b), + UNIPHIER_PINCTRL_GROUP(i2c5c), + UNIPHIER_PINCTRL_GROUP(i2c6), + UNIPHIER_PINCTRL_GROUP(uart0), + UNIPHIER_PINCTRL_GROUP(uart0b), + UNIPHIER_PINCTRL_GROUP(uart1), + UNIPHIER_PINCTRL_GROUP(uart2), + UNIPHIER_PINCTRL_GROUP(uart3), + UNIPHIER_PINCTRL_GROUP(usb0), + UNIPHIER_PINCTRL_GROUP(usb1), + UNIPHIER_PINCTRL_GROUP(usb2), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range0), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range1), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq_alternatives), + UNIPHIER_PINCTRL_GROUP_SINGLE(port00, port_range0, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port01, port_range0, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port02, port_range0, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port03, port_range0, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port04, port_range0, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port05, port_range0, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port06, port_range0, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port07, port_range0, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port10, port_range0, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port11, port_range0, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port12, port_range0, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port13, port_range0, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port14, port_range0, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port15, port_range0, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port16, port_range0, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port17, port_range0, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port20, port_range0, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port21, port_range0, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port22, port_range0, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port23, port_range0, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port24, port_range0, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port25, port_range0, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port26, port_range0, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port27, port_range0, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port30, port_range0, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port31, port_range0, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port32, port_range0, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port33, port_range0, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port34, port_range0, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port35, port_range0, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port36, port_range0, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port37, port_range0, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port40, port_range0, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port41, port_range0, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port42, port_range0, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port43, port_range0, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port44, port_range0, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port45, port_range0, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port46, port_range0, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port47, port_range0, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port50, port_range0, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port51, port_range0, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port52, port_range0, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port53, port_range0, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port54, port_range0, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port55, port_range0, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port56, port_range0, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port57, port_range0, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port60, port_range0, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port61, port_range0, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port62, port_range0, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port63, port_range0, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port64, port_range0, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port65, port_range0, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port66, port_range0, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port67, port_range0, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port70, port_range0, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port71, port_range0, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port72, port_range0, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port73, port_range0, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port74, port_range0, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port75, port_range0, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port76, port_range0, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port77, port_range0, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port80, port_range0, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port81, port_range0, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port82, port_range0, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port83, port_range0, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port84, port_range0, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port85, port_range0, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port86, port_range0, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port87, port_range0, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port90, port_range0, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port91, port_range0, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port92, port_range0, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port93, port_range0, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port94, port_range0, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port95, port_range0, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port96, port_range0, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port97, port_range0, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port100, port_range0, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port101, port_range0, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port102, port_range0, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port103, port_range0, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port104, port_range0, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port105, port_range0, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port106, port_range0, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port107, port_range0, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port110, port_range0, 88), + UNIPHIER_PINCTRL_GROUP_SINGLE(port111, port_range0, 89), + UNIPHIER_PINCTRL_GROUP_SINGLE(port112, port_range0, 90), + UNIPHIER_PINCTRL_GROUP_SINGLE(port113, port_range0, 91), + UNIPHIER_PINCTRL_GROUP_SINGLE(port114, port_range0, 92), + UNIPHIER_PINCTRL_GROUP_SINGLE(port115, port_range0, 93), + UNIPHIER_PINCTRL_GROUP_SINGLE(port116, port_range0, 94), + UNIPHIER_PINCTRL_GROUP_SINGLE(port117, port_range0, 95), + UNIPHIER_PINCTRL_GROUP_SINGLE(port120, port_range0, 96), + UNIPHIER_PINCTRL_GROUP_SINGLE(port121, port_range0, 97), + UNIPHIER_PINCTRL_GROUP_SINGLE(port122, port_range0, 98), + UNIPHIER_PINCTRL_GROUP_SINGLE(port123, port_range0, 99), + UNIPHIER_PINCTRL_GROUP_SINGLE(port124, port_range0, 100), + UNIPHIER_PINCTRL_GROUP_SINGLE(port125, port_range0, 101), + UNIPHIER_PINCTRL_GROUP_SINGLE(port126, port_range0, 102), + UNIPHIER_PINCTRL_GROUP_SINGLE(port127, port_range0, 103), + UNIPHIER_PINCTRL_GROUP_SINGLE(port130, port_range0, 104), + UNIPHIER_PINCTRL_GROUP_SINGLE(port131, port_range0, 105), + UNIPHIER_PINCTRL_GROUP_SINGLE(port132, port_range0, 106), + UNIPHIER_PINCTRL_GROUP_SINGLE(port133, port_range0, 107), + UNIPHIER_PINCTRL_GROUP_SINGLE(port134, port_range0, 108), + UNIPHIER_PINCTRL_GROUP_SINGLE(port135, port_range0, 109), + UNIPHIER_PINCTRL_GROUP_SINGLE(port136, port_range0, 110), + UNIPHIER_PINCTRL_GROUP_SINGLE(port137, port_range0, 111), + UNIPHIER_PINCTRL_GROUP_SINGLE(port140, port_range0, 112), + UNIPHIER_PINCTRL_GROUP_SINGLE(port141, port_range0, 113), + UNIPHIER_PINCTRL_GROUP_SINGLE(port142, port_range0, 114), + UNIPHIER_PINCTRL_GROUP_SINGLE(port143, port_range0, 115), + UNIPHIER_PINCTRL_GROUP_SINGLE(port144, port_range0, 116), + UNIPHIER_PINCTRL_GROUP_SINGLE(port145, port_range0, 117), + UNIPHIER_PINCTRL_GROUP_SINGLE(port146, port_range0, 118), + UNIPHIER_PINCTRL_GROUP_SINGLE(port147, port_range0, 119), + UNIPHIER_PINCTRL_GROUP_SINGLE(port175, port_range1, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port176, port_range1, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port177, port_range1, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port180, port_range1, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port181, port_range1, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port182, port_range1, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port183, port_range1, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port184, port_range1, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port185, port_range1, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port186, port_range1, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port187, port_range1, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port190, port_range1, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port191, port_range1, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port192, port_range1, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port193, port_range1, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port194, port_range1, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port195, port_range1, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port196, port_range1, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port197, port_range1, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port200, port_range1, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port201, port_range1, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port202, port_range1, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port203, port_range1, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port204, port_range1, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port205, port_range1, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port206, port_range1, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port207, port_range1, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port210, port_range1, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port211, port_range1, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port212, port_range1, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port213, port_range1, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port214, port_range1, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port215, port_range1, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port216, port_range1, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port217, port_range1, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port220, port_range1, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port221, port_range1, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port222, port_range1, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port223, port_range1, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port224, port_range1, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port225, port_range1, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port226, port_range1, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port227, port_range1, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port230, port_range1, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port231, port_range1, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port232, port_range1, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port233, port_range1, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port234, port_range1, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port235, port_range1, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port236, port_range1, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port237, port_range1, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port240, port_range1, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port241, port_range1, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port242, port_range1, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port243, port_range1, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port244, port_range1, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port245, port_range1, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port246, port_range1, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port247, port_range1, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port250, port_range1, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port251, port_range1, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port252, port_range1, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port253, port_range1, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port254, port_range1, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port255, port_range1, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port256, port_range1, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port257, port_range1, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port260, port_range1, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port261, port_range1, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port262, port_range1, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port263, port_range1, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port264, port_range1, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port265, port_range1, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port266, port_range1, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port267, port_range1, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port270, port_range1, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port271, port_range1, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port272, port_range1, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port273, port_range1, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port274, port_range1, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port275, port_range1, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port276, port_range1, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port277, port_range1, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port280, port_range1, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port281, port_range1, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port282, port_range1, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port283, port_range1, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port284, port_range1, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port285, port_range1, 88), + UNIPHIER_PINCTRL_GROUP_SINGLE(port286, port_range1, 89), + UNIPHIER_PINCTRL_GROUP_SINGLE(port287, port_range1, 90), + UNIPHIER_PINCTRL_GROUP_SINGLE(port290, port_range1, 91), + UNIPHIER_PINCTRL_GROUP_SINGLE(port291, port_range1, 92), + UNIPHIER_PINCTRL_GROUP_SINGLE(port292, port_range1, 93), + UNIPHIER_PINCTRL_GROUP_SINGLE(port293, port_range1, 94), + UNIPHIER_PINCTRL_GROUP_SINGLE(port294, port_range1, 95), + UNIPHIER_PINCTRL_GROUP_SINGLE(port295, port_range1, 96), + UNIPHIER_PINCTRL_GROUP_SINGLE(port296, port_range1, 97), + UNIPHIER_PINCTRL_GROUP_SINGLE(port297, port_range1, 98), + UNIPHIER_PINCTRL_GROUP_SINGLE(port300, port_range1, 99), + UNIPHIER_PINCTRL_GROUP_SINGLE(port301, port_range1, 100), + UNIPHIER_PINCTRL_GROUP_SINGLE(port302, port_range1, 101), + UNIPHIER_PINCTRL_GROUP_SINGLE(port303, port_range1, 102), + UNIPHIER_PINCTRL_GROUP_SINGLE(port304, port_range1, 103), + UNIPHIER_PINCTRL_GROUP_SINGLE(port305, port_range1, 104), + UNIPHIER_PINCTRL_GROUP_SINGLE(port306, port_range1, 105), + UNIPHIER_PINCTRL_GROUP_SINGLE(port307, port_range1, 106), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq0, xirq, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq1, xirq, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq2, xirq, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq3, xirq, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq4, xirq, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq5, xirq, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq6, xirq, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq7, xirq, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq8, xirq, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq9, xirq, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq10, xirq, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq11, xirq, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq12, xirq, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq13, xirq, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq14, xirq, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq15, xirq, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq16, xirq, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq17, xirq, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq18, xirq, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq19, xirq, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq20, xirq, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq3b, xirq_alternatives, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq4b, xirq_alternatives, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq16b, xirq_alternatives, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq17b, xirq_alternatives, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq17c, xirq_alternatives, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq18b, xirq_alternatives, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq18c, xirq_alternatives, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq19b, xirq_alternatives, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq20b, xirq_alternatives, 8), +}; + +static const char * const emmc_groups[] = {"emmc", "emmc_dat8"}; +static const char * const i2c0_groups[] = {"i2c0"}; +static const char * const i2c1_groups[] = {"i2c1"}; +static const char * const i2c2_groups[] = {"i2c2"}; +static const char * const i2c3_groups[] = {"i2c3"}; +static const char * const i2c5_groups[] = {"i2c5", "i2c5b", "i2c5c"}; +static const char * const i2c6_groups[] = {"i2c6"}; +static const char * const nand_groups[] = {"nand", "nand_cs1"}; +static const char * const uart0_groups[] = {"uart0", "uart0b"}; +static const char * const uart1_groups[] = {"uart1"}; +static const char * const uart2_groups[] = {"uart2"}; +static const char * const uart3_groups[] = {"uart3"}; +static const char * const usb0_groups[] = {"usb0"}; +static const char * const usb1_groups[] = {"usb1"}; +static const char * const usb2_groups[] = {"usb2"}; +static const char * const port_groups[] = { + "port00", "port01", "port02", "port03", + "port04", "port05", "port06", "port07", + "port10", "port11", "port12", "port13", + "port14", "port15", "port16", "port17", + "port20", "port21", "port22", "port23", + "port24", "port25", "port26", "port27", + "port30", "port31", "port32", "port33", + "port34", "port35", "port36", "port37", + "port40", "port41", "port42", "port43", + "port44", "port45", "port46", "port47", + "port50", "port51", "port52", "port53", + "port54", "port55", "port56", "port57", + "port60", "port61", "port62", "port63", + "port64", "port65", "port66", "port67", + "port70", "port71", "port72", "port73", + "port74", "port75", "port76", "port77", + "port80", "port81", "port82", "port83", + "port84", "port85", "port86", "port87", + "port90", "port91", "port92", "port93", + "port94", "port95", "port96", "port97", + "port100", "port101", "port102", "port103", + "port104", "port105", "port106", "port107", + "port110", "port111", "port112", "port113", + "port114", "port115", "port116", "port117", + "port120", "port121", "port122", "port123", + "port124", "port125", "port126", "port127", + "port130", "port131", "port132", "port133", + "port134", "port135", "port136", "port137", + "port140", "port141", "port142", "port143", + "port144", "port145", "port146", "port147", + /* port150-174 missing */ + /* none */ "port175", "port176", "port177", + "port180", "port181", "port182", "port183", + "port184", "port185", "port186", "port187", + "port190", "port191", "port192", "port193", + "port194", "port195", "port196", "port197", + "port200", "port201", "port202", "port203", + "port204", "port205", "port206", "port207", + "port210", "port211", "port212", "port213", + "port214", "port215", "port216", "port217", + "port220", "port221", "port222", "port223", + "port224", "port225", "port226", "port227", + "port230", "port231", "port232", "port233", + "port234", "port235", "port236", "port237", + "port240", "port241", "port242", "port243", + "port244", "port245", "port246", "port247", + "port250", "port251", "port252", "port253", + "port254", "port255", "port256", "port257", + "port260", "port261", "port262", "port263", + "port264", "port265", "port266", "port267", + "port270", "port271", "port272", "port273", + "port274", "port275", "port276", "port277", + "port280", "port281", "port282", "port283", + "port284", "port285", "port286", "port287", + "port290", "port291", "port292", "port293", + "port294", "port295", "port296", "port297", + "port300", "port301", "port302", "port303", + "port304", "port305", "port306", "port307", +}; +static const char * const xirq_groups[] = { + "xirq0", "xirq1", "xirq2", "xirq3", + "xirq4", "xirq5", "xirq6", "xirq7", + "xirq8", "xirq9", "xirq10", "xirq11", + "xirq12", "xirq13", "xirq14", "xirq15", + "xirq16", "xirq17", "xirq18", "xirq19", + "xirq20", + "xirq3b", "xirq4b", "xirq16b", "xirq17b", "xirq17c", + "xirq18b", "xirq18c", "xirq19b", "xirq20b", +}; + +static const struct uniphier_pinmux_function ph1_pro5_functions[] = { + UNIPHIER_PINMUX_FUNCTION(emmc), + UNIPHIER_PINMUX_FUNCTION(i2c0), + UNIPHIER_PINMUX_FUNCTION(i2c1), + UNIPHIER_PINMUX_FUNCTION(i2c2), + UNIPHIER_PINMUX_FUNCTION(i2c3), + UNIPHIER_PINMUX_FUNCTION(i2c5), + UNIPHIER_PINMUX_FUNCTION(i2c6), + UNIPHIER_PINMUX_FUNCTION(nand), + UNIPHIER_PINMUX_FUNCTION(uart0), + UNIPHIER_PINMUX_FUNCTION(uart1), + UNIPHIER_PINMUX_FUNCTION(uart2), + UNIPHIER_PINMUX_FUNCTION(uart3), + UNIPHIER_PINMUX_FUNCTION(usb0), + UNIPHIER_PINMUX_FUNCTION(usb1), + UNIPHIER_PINMUX_FUNCTION(usb2), + UNIPHIER_PINMUX_FUNCTION(port), + UNIPHIER_PINMUX_FUNCTION(xirq), +}; + +static struct uniphier_pinctrl_socdata ph1_pro5_pindata = { + .groups = ph1_pro5_groups, + .groups_count = ARRAY_SIZE(ph1_pro5_groups), + .functions = ph1_pro5_functions, + .functions_count = ARRAY_SIZE(ph1_pro5_functions), + .mux_bits = 4, + .reg_stride = 8, + .load_pinctrl = true, +}; + +static struct pinctrl_desc ph1_pro5_pinctrl_desc = { + .name = DRIVER_NAME, + .pins = ph1_pro5_pins, + .npins = ARRAY_SIZE(ph1_pro5_pins), + .owner = THIS_MODULE, +}; + +static int ph1_pro5_pinctrl_probe(struct platform_device *pdev) +{ + return uniphier_pinctrl_probe(pdev, &ph1_pro5_pinctrl_desc, + &ph1_pro5_pindata); +} + +static const struct of_device_id ph1_pro5_pinctrl_match[] = { + { .compatible = "socionext,ph1-pro5-pinctrl" }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, ph1_pro5_pinctrl_match); + +static struct platform_driver ph1_pro5_pinctrl_driver = { + .probe = ph1_pro5_pinctrl_probe, + .remove = uniphier_pinctrl_remove, + .driver = { + .name = DRIVER_NAME, + .of_match_table = ph1_pro5_pinctrl_match, + }, +}; +module_platform_driver(ph1_pro5_pinctrl_driver); + +MODULE_AUTHOR("Masahiro Yamada "); +MODULE_DESCRIPTION("UniPhier PH1-Pro5 pinctrl driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pinctrl/uniphier/pinctrl-ph1-sld8.c b/drivers/pinctrl/uniphier/pinctrl-ph1-sld8.c new file mode 100644 index 00000000000000..7e9dae54fcb22e --- /dev/null +++ b/drivers/pinctrl/uniphier/pinctrl-ph1-sld8.c @@ -0,0 +1,794 @@ +/* + * Copyright (C) 2015 Masahiro Yamada + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include +#include +#include + +#include "pinctrl-uniphier.h" + +#define DRIVER_NAME "ph1-sld8-pinctrl" + +static const struct pinctrl_pin_desc ph1_sld8_pins[] = { + UNIPHIER_PINCTRL_PIN(0, "PCA00", UNIPHIER_PIN_IECTRL_NONE, + 15, UNIPHIER_PIN_DRV_4_8, + 15, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(1, "PCA01", UNIPHIER_PIN_IECTRL_NONE, + 16, UNIPHIER_PIN_DRV_4_8, + 16, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(2, "PCA02", UNIPHIER_PIN_IECTRL_NONE, + 17, UNIPHIER_PIN_DRV_4_8, + 17, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(3, "PCA03", UNIPHIER_PIN_IECTRL_NONE, + 18, UNIPHIER_PIN_DRV_4_8, + 18, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(4, "PCA04", UNIPHIER_PIN_IECTRL_NONE, + 19, UNIPHIER_PIN_DRV_4_8, + 19, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(5, "PCA05", UNIPHIER_PIN_IECTRL_NONE, + 20, UNIPHIER_PIN_DRV_4_8, + 20, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(6, "PCA06", UNIPHIER_PIN_IECTRL_NONE, + 21, UNIPHIER_PIN_DRV_4_8, + 21, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(7, "PCA07", UNIPHIER_PIN_IECTRL_NONE, + 22, UNIPHIER_PIN_DRV_4_8, + 22, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(8, "PCA08", UNIPHIER_PIN_IECTRL_NONE, + 23, UNIPHIER_PIN_DRV_4_8, + 23, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(9, "PCA09", UNIPHIER_PIN_IECTRL_NONE, + 24, UNIPHIER_PIN_DRV_4_8, + 24, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(10, "PCA10", UNIPHIER_PIN_IECTRL_NONE, + 25, UNIPHIER_PIN_DRV_4_8, + 25, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(11, "PCA11", UNIPHIER_PIN_IECTRL_NONE, + 26, UNIPHIER_PIN_DRV_4_8, + 26, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(12, "PCA12", UNIPHIER_PIN_IECTRL_NONE, + 27, UNIPHIER_PIN_DRV_4_8, + 27, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(13, "PCA13", UNIPHIER_PIN_IECTRL_NONE, + 28, UNIPHIER_PIN_DRV_4_8, + 28, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(14, "PCA14", UNIPHIER_PIN_IECTRL_NONE, + 29, UNIPHIER_PIN_DRV_4_8, + 29, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(15, "XNFRE_GB", UNIPHIER_PIN_IECTRL_NONE, + 30, UNIPHIER_PIN_DRV_4_8, + 30, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(16, "XNFWE_GB", UNIPHIER_PIN_IECTRL_NONE, + 31, UNIPHIER_PIN_DRV_4_8, + 31, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(17, "NFALE_GB", UNIPHIER_PIN_IECTRL_NONE, + 32, UNIPHIER_PIN_DRV_4_8, + 32, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(18, "NFCLE_GB", UNIPHIER_PIN_IECTRL_NONE, + 33, UNIPHIER_PIN_DRV_4_8, + 33, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(19, "XNFWP_GB", UNIPHIER_PIN_IECTRL_NONE, + 34, UNIPHIER_PIN_DRV_4_8, + 34, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(20, "XNFCE0_GB", UNIPHIER_PIN_IECTRL_NONE, + 35, UNIPHIER_PIN_DRV_4_8, + 35, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(21, "NANDRYBY0_GB", UNIPHIER_PIN_IECTRL_NONE, + 36, UNIPHIER_PIN_DRV_4_8, + 36, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(22, "XNFCE1_GB", UNIPHIER_PIN_IECTRL_NONE, + 0, UNIPHIER_PIN_DRV_8_12_16_20, + 119, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(23, "NANDRYBY1_GB", UNIPHIER_PIN_IECTRL_NONE, + 4, UNIPHIER_PIN_DRV_8_12_16_20, + 120, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(24, "NFD0_GB", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_8_12_16_20, + 121, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(25, "NFD1_GB", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_8_12_16_20, + 122, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(26, "NFD2_GB", UNIPHIER_PIN_IECTRL_NONE, + 16, UNIPHIER_PIN_DRV_8_12_16_20, + 123, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(27, "NFD3_GB", UNIPHIER_PIN_IECTRL_NONE, + 20, UNIPHIER_PIN_DRV_8_12_16_20, + 124, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(28, "NFD4_GB", UNIPHIER_PIN_IECTRL_NONE, + 24, UNIPHIER_PIN_DRV_8_12_16_20, + 125, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(29, "NFD5_GB", UNIPHIER_PIN_IECTRL_NONE, + 28, UNIPHIER_PIN_DRV_8_12_16_20, + 126, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(30, "NFD6_GB", UNIPHIER_PIN_IECTRL_NONE, + 32, UNIPHIER_PIN_DRV_8_12_16_20, + 127, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(31, "NFD7_GB", UNIPHIER_PIN_IECTRL_NONE, + 36, UNIPHIER_PIN_DRV_8_12_16_20, + 128, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(32, "SDCLK", UNIPHIER_PIN_IECTRL_NONE, + 40, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(33, "SDCMD", UNIPHIER_PIN_IECTRL_NONE, + 44, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(34, "SDDAT0", UNIPHIER_PIN_IECTRL_NONE, + 48, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(35, "SDDAT1", UNIPHIER_PIN_IECTRL_NONE, + 52, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(36, "SDDAT2", UNIPHIER_PIN_IECTRL_NONE, + 56, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(37, "SDDAT3", UNIPHIER_PIN_IECTRL_NONE, + 60, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(38, "SDCD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 129, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(39, "SDWP", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 130, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(40, "SDVOLC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 131, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(41, "USB0VBUS", UNIPHIER_PIN_IECTRL_NONE, + 37, UNIPHIER_PIN_DRV_4_8, + 37, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(42, "USB0OD", UNIPHIER_PIN_IECTRL_NONE, + 38, UNIPHIER_PIN_DRV_4_8, + 38, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(43, "USB1VBUS", UNIPHIER_PIN_IECTRL_NONE, + 39, UNIPHIER_PIN_DRV_4_8, + 39, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(44, "USB1OD", UNIPHIER_PIN_IECTRL_NONE, + 40, UNIPHIER_PIN_DRV_4_8, + 40, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(45, "PCRESET", UNIPHIER_PIN_IECTRL_NONE, + 41, UNIPHIER_PIN_DRV_4_8, + 41, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(46, "PCREG", UNIPHIER_PIN_IECTRL_NONE, + 42, UNIPHIER_PIN_DRV_4_8, + 42, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(47, "PCCE2", UNIPHIER_PIN_IECTRL_NONE, + 43, UNIPHIER_PIN_DRV_4_8, + 43, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(48, "PCVS1", UNIPHIER_PIN_IECTRL_NONE, + 44, UNIPHIER_PIN_DRV_4_8, + 44, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(49, "PCCD2", UNIPHIER_PIN_IECTRL_NONE, + 45, UNIPHIER_PIN_DRV_4_8, + 45, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(50, "PCCD1", UNIPHIER_PIN_IECTRL_NONE, + 46, UNIPHIER_PIN_DRV_4_8, + 46, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(51, "PCREADY", UNIPHIER_PIN_IECTRL_NONE, + 47, UNIPHIER_PIN_DRV_4_8, + 47, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(52, "PCDOE", UNIPHIER_PIN_IECTRL_NONE, + 48, UNIPHIER_PIN_DRV_4_8, + 48, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(53, "PCCE1", UNIPHIER_PIN_IECTRL_NONE, + 49, UNIPHIER_PIN_DRV_4_8, + 49, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(54, "PCWE", UNIPHIER_PIN_IECTRL_NONE, + 50, UNIPHIER_PIN_DRV_4_8, + 50, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(55, "PCOE", UNIPHIER_PIN_IECTRL_NONE, + 51, UNIPHIER_PIN_DRV_4_8, + 51, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(56, "PCWAIT", UNIPHIER_PIN_IECTRL_NONE, + 52, UNIPHIER_PIN_DRV_4_8, + 52, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(57, "PCIOWR", UNIPHIER_PIN_IECTRL_NONE, + 53, UNIPHIER_PIN_DRV_4_8, + 53, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(58, "PCIORD", UNIPHIER_PIN_IECTRL_NONE, + 54, UNIPHIER_PIN_DRV_4_8, + 54, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(59, "HS0DIN0", UNIPHIER_PIN_IECTRL_NONE, + 55, UNIPHIER_PIN_DRV_4_8, + 55, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(60, "HS0DIN1", UNIPHIER_PIN_IECTRL_NONE, + 56, UNIPHIER_PIN_DRV_4_8, + 56, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(61, "HS0DIN2", UNIPHIER_PIN_IECTRL_NONE, + 57, UNIPHIER_PIN_DRV_4_8, + 57, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(62, "HS0DIN3", UNIPHIER_PIN_IECTRL_NONE, + 58, UNIPHIER_PIN_DRV_4_8, + 58, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(63, "HS0DIN4", UNIPHIER_PIN_IECTRL_NONE, + 59, UNIPHIER_PIN_DRV_4_8, + 59, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(64, "HS0DIN5", UNIPHIER_PIN_IECTRL_NONE, + 60, UNIPHIER_PIN_DRV_4_8, + 60, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(65, "HS0DIN6", UNIPHIER_PIN_IECTRL_NONE, + 61, UNIPHIER_PIN_DRV_4_8, + 61, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(66, "HS0DIN7", UNIPHIER_PIN_IECTRL_NONE, + 62, UNIPHIER_PIN_DRV_4_8, + 62, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(67, "HS0BCLKIN", UNIPHIER_PIN_IECTRL_NONE, + 63, UNIPHIER_PIN_DRV_4_8, + 63, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(68, "HS0VALIN", UNIPHIER_PIN_IECTRL_NONE, + 64, UNIPHIER_PIN_DRV_4_8, + 64, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(69, "HS0SYNCIN", UNIPHIER_PIN_IECTRL_NONE, + 65, UNIPHIER_PIN_DRV_4_8, + 65, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(70, "HSDOUT0", UNIPHIER_PIN_IECTRL_NONE, + 66, UNIPHIER_PIN_DRV_4_8, + 66, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(71, "HSDOUT1", UNIPHIER_PIN_IECTRL_NONE, + 67, UNIPHIER_PIN_DRV_4_8, + 67, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(72, "HSDOUT2", UNIPHIER_PIN_IECTRL_NONE, + 68, UNIPHIER_PIN_DRV_4_8, + 68, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(73, "HSDOUT3", UNIPHIER_PIN_IECTRL_NONE, + 69, UNIPHIER_PIN_DRV_4_8, + 69, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(74, "HSDOUT4", UNIPHIER_PIN_IECTRL_NONE, + 70, UNIPHIER_PIN_DRV_4_8, + 70, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(75, "HSDOUT5", UNIPHIER_PIN_IECTRL_NONE, + 71, UNIPHIER_PIN_DRV_4_8, + 71, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(76, "HSDOUT6", UNIPHIER_PIN_IECTRL_NONE, + 72, UNIPHIER_PIN_DRV_4_8, + 72, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(77, "HSDOUT7", UNIPHIER_PIN_IECTRL_NONE, + 73, UNIPHIER_PIN_DRV_4_8, + 73, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(78, "HSBCLKOUT", UNIPHIER_PIN_IECTRL_NONE, + 74, UNIPHIER_PIN_DRV_4_8, + 74, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(79, "HSVALOUT", UNIPHIER_PIN_IECTRL_NONE, + 75, UNIPHIER_PIN_DRV_4_8, + 75, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(80, "HSSYNCOUT", UNIPHIER_PIN_IECTRL_NONE, + 76, UNIPHIER_PIN_DRV_4_8, + 76, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(81, "HS1DIN0", UNIPHIER_PIN_IECTRL_NONE, + 77, UNIPHIER_PIN_DRV_4_8, + 77, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(82, "HS1DIN1", UNIPHIER_PIN_IECTRL_NONE, + 78, UNIPHIER_PIN_DRV_4_8, + 78, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(83, "HS1DIN2", UNIPHIER_PIN_IECTRL_NONE, + 79, UNIPHIER_PIN_DRV_4_8, + 79, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(84, "HS1DIN3", UNIPHIER_PIN_IECTRL_NONE, + 80, UNIPHIER_PIN_DRV_4_8, + 80, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(85, "HS1DIN4", UNIPHIER_PIN_IECTRL_NONE, + 81, UNIPHIER_PIN_DRV_4_8, + 81, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(86, "HS1DIN5", UNIPHIER_PIN_IECTRL_NONE, + 82, UNIPHIER_PIN_DRV_4_8, + 82, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(87, "HS1DIN6", UNIPHIER_PIN_IECTRL_NONE, + 83, UNIPHIER_PIN_DRV_4_8, + 83, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(88, "HS1DIN7", UNIPHIER_PIN_IECTRL_NONE, + 84, UNIPHIER_PIN_DRV_4_8, + 84, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(89, "HS1BCLKIN", UNIPHIER_PIN_IECTRL_NONE, + 85, UNIPHIER_PIN_DRV_4_8, + 85, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(90, "HS1VALIN", UNIPHIER_PIN_IECTRL_NONE, + 86, UNIPHIER_PIN_DRV_4_8, + 86, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(91, "HS1SYNCIN", UNIPHIER_PIN_IECTRL_NONE, + 87, UNIPHIER_PIN_DRV_4_8, + 87, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(92, "AGCI", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 132, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(93, "AGCR", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 133, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(94, "AGCBS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 134, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(95, "IECOUT", UNIPHIER_PIN_IECTRL_NONE, + 88, UNIPHIER_PIN_DRV_4_8, + 88, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(96, "ASMCK", UNIPHIER_PIN_IECTRL_NONE, + 89, UNIPHIER_PIN_DRV_4_8, + 89, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(97, "ABCKO", UNIPHIER_PIN_IECTRL_NONE, + 90, UNIPHIER_PIN_DRV_4_8, + 90, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(98, "ALRCKO", UNIPHIER_PIN_IECTRL_NONE, + 91, UNIPHIER_PIN_DRV_4_8, + 91, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(99, "ASDOUT0", UNIPHIER_PIN_IECTRL_NONE, + 92, UNIPHIER_PIN_DRV_4_8, + 92, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(100, "ASDOUT1", UNIPHIER_PIN_IECTRL_NONE, + 93, UNIPHIER_PIN_DRV_4_8, + 93, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(101, "ARCOUT", UNIPHIER_PIN_IECTRL_NONE, + 94, UNIPHIER_PIN_DRV_4_8, + 94, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(102, "SDA0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(103, "SCL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(104, "SDA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(105, "SCL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(106, "DMDSDA0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(107, "DMDSCL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(108, "DMDSDA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(109, "DMDSCL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(110, "SBO0", UNIPHIER_PIN_IECTRL_NONE, + 95, UNIPHIER_PIN_DRV_4_8, + 95, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(111, "SBI0", UNIPHIER_PIN_IECTRL_NONE, + 96, UNIPHIER_PIN_DRV_4_8, + 96, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(112, "SBO1", UNIPHIER_PIN_IECTRL_NONE, + 97, UNIPHIER_PIN_DRV_4_8, + 97, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(113, "SBI1", UNIPHIER_PIN_IECTRL_NONE, + 98, UNIPHIER_PIN_DRV_4_8, + 98, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(114, "TXD1", UNIPHIER_PIN_IECTRL_NONE, + 99, UNIPHIER_PIN_DRV_4_8, + 99, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(115, "RXD1", UNIPHIER_PIN_IECTRL_NONE, + 100, UNIPHIER_PIN_DRV_4_8, + 100, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(116, "HIN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_5, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(117, "VIN", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_5, + -1, UNIPHIER_PIN_PULL_NONE), + UNIPHIER_PINCTRL_PIN(118, "TCON0", UNIPHIER_PIN_IECTRL_NONE, + 101, UNIPHIER_PIN_DRV_4_8, + 101, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(119, "TCON1", UNIPHIER_PIN_IECTRL_NONE, + 102, UNIPHIER_PIN_DRV_4_8, + 102, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(120, "TCON2", UNIPHIER_PIN_IECTRL_NONE, + 103, UNIPHIER_PIN_DRV_4_8, + 103, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(121, "TCON3", UNIPHIER_PIN_IECTRL_NONE, + 104, UNIPHIER_PIN_DRV_4_8, + 104, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(122, "TCON4", UNIPHIER_PIN_IECTRL_NONE, + 105, UNIPHIER_PIN_DRV_4_8, + 105, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(123, "TCON5", UNIPHIER_PIN_IECTRL_NONE, + 106, UNIPHIER_PIN_DRV_4_8, + 106, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(124, "TCON6", UNIPHIER_PIN_IECTRL_NONE, + 107, UNIPHIER_PIN_DRV_4_8, + 107, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(125, "TCON7", UNIPHIER_PIN_IECTRL_NONE, + 108, UNIPHIER_PIN_DRV_4_8, + 108, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(126, "TCON8", UNIPHIER_PIN_IECTRL_NONE, + 109, UNIPHIER_PIN_DRV_4_8, + 109, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(127, "PWMA", UNIPHIER_PIN_IECTRL_NONE, + 110, UNIPHIER_PIN_DRV_4_8, + 110, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(128, "XIRQ0", UNIPHIER_PIN_IECTRL_NONE, + 111, UNIPHIER_PIN_DRV_4_8, + 111, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(129, "XIRQ1", UNIPHIER_PIN_IECTRL_NONE, + 112, UNIPHIER_PIN_DRV_4_8, + 112, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(130, "XIRQ2", UNIPHIER_PIN_IECTRL_NONE, + 113, UNIPHIER_PIN_DRV_4_8, + 113, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(131, "XIRQ3", UNIPHIER_PIN_IECTRL_NONE, + 114, UNIPHIER_PIN_DRV_4_8, + 114, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(132, "XIRQ4", UNIPHIER_PIN_IECTRL_NONE, + 115, UNIPHIER_PIN_DRV_4_8, + 115, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(133, "XIRQ5", UNIPHIER_PIN_IECTRL_NONE, + 116, UNIPHIER_PIN_DRV_4_8, + 116, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(134, "XIRQ6", UNIPHIER_PIN_IECTRL_NONE, + 117, UNIPHIER_PIN_DRV_4_8, + 117, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(135, "XIRQ7", UNIPHIER_PIN_IECTRL_NONE, + 118, UNIPHIER_PIN_DRV_4_8, + 118, UNIPHIER_PIN_PULL_DOWN), +}; + +static const unsigned emmc_pins[] = {21, 22, 23, 24, 25, 26, 27}; +static const unsigned emmc_muxvals[] = {1, 1, 1, 1, 1, 1, 1}; +static const unsigned emmc_dat8_pins[] = {28, 29, 30, 31}; +static const unsigned emmc_dat8_muxvals[] = {1, 1, 1, 1}; +static const unsigned i2c0_pins[] = {102, 103}; +static const unsigned i2c0_muxvals[] = {0, 0}; +static const unsigned i2c1_pins[] = {104, 105}; +static const unsigned i2c1_muxvals[] = {0, 0}; +static const unsigned i2c2_pins[] = {108, 109}; +static const unsigned i2c2_muxvals[] = {2, 2}; +static const unsigned i2c3_pins[] = {108, 109}; +static const unsigned i2c3_muxvals[] = {3, 3}; +static const unsigned nand_pins[] = {15, 16, 17, 18, 19, 20, 21, 24, 25, 26, + 27, 28, 29, 30, 31}; +static const unsigned nand_muxvals[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0}; +static const unsigned nand_cs1_pins[] = {22, 23}; +static const unsigned nand_cs1_muxvals[] = {0, 0}; +static const unsigned uart0_pins[] = {70, 71}; +static const unsigned uart0_muxvals[] = {3, 3}; +static const unsigned uart1_pins[] = {114, 115}; +static const unsigned uart1_muxvals[] = {0, 0}; +static const unsigned uart2_pins[] = {112, 113}; +static const unsigned uart2_muxvals[] = {1, 1}; +static const unsigned uart3_pins[] = {110, 111}; +static const unsigned uart3_muxvals[] = {1, 1}; +static const unsigned usb0_pins[] = {41, 42}; +static const unsigned usb0_muxvals[] = {0, 0}; +static const unsigned usb1_pins[] = {43, 44}; +static const unsigned usb1_muxvals[] = {0, 0}; +static const unsigned usb2_pins[] = {114, 115}; +static const unsigned usb2_muxvals[] = {1, 1}; +static const unsigned port_range0_pins[] = { + 0, 1, 2, 3, 4, 5, 6, 7, /* PORT0x */ + 8, 9, 10, 11, 12, 13, 14, 15, /* PORT1x */ + 32, 33, 34, 35, 36, 37, 38, 39, /* PORT2x */ + 59, 60, 61, 62, 63, 64, 65, 66, /* PORT3x */ + 95, 96, 97, 98, 99, 100, 101, 57, /* PORT4x */ + 70, 71, 72, 73, 74, 75, 76, 77, /* PORT5x */ + 81, 83, 84, 85, 86, 89, 90, 91, /* PORT6x */ + 118, 119, 120, 121, 122, 53, 54, 55, /* PORT7x */ + 41, 42, 43, 44, 79, 80, 18, 19, /* PORT8x */ + 110, 111, 112, 113, 114, 115, 16, 17, /* PORT9x */ + 40, 67, 68, 69, 78, 92, 93, 94, /* PORT10x */ + 48, 49, 46, 45, 123, 124, 125, 126, /* PORT11x */ + 47, 127, 20, 56, 22, /* PORT120-124 */ +}; +static const unsigned port_range0_muxvals[] = { + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT0x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT1x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT2x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT3x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT4x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT5x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT6x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT7x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT8x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT9x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT10x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT11x */ + 15, 15, 15, 15, 15, /* PORT120-124 */ +}; +static const unsigned port_range1_pins[] = { + 116, 117, /* PORT130-131 */ +}; +static const unsigned port_range1_muxvals[] = { + 15, 15, /* PORT130-131 */ +}; +static const unsigned port_range2_pins[] = { + 102, 103, 104, 105, 106, 107, 108, 109, /* PORT14x */ +}; +static const unsigned port_range2_muxvals[] = { + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT14x */ +}; +static const unsigned port_range3_pins[] = { + 23, /* PORT166 */ +}; +static const unsigned port_range3_muxvals[] = { + 15, /* PORT166 */ +}; +static const unsigned xirq_range0_pins[] = { + 128, 129, 130, 131, 132, 133, 134, 135, /* XIRQ0-7 */ + 82, 87, 88, 50, 51, /* XIRQ8-12 */ +}; +static const unsigned xirq_range0_muxvals[] = { + 0, 0, 0, 0, 0, 0, 0, 0, /* XIRQ0-7 */ + 14, 14, 14, 14, 14, /* XIRQ8-12 */ +}; +static const unsigned xirq_range1_pins[] = { + 52, 58, /* XIRQ14-15 */ +}; +static const unsigned xirq_range1_muxvals[] = { + 14, 14, /* XIRQ14-15 */ +}; + +static const struct uniphier_pinctrl_group ph1_sld8_groups[] = { + UNIPHIER_PINCTRL_GROUP(emmc), + UNIPHIER_PINCTRL_GROUP(emmc_dat8), + UNIPHIER_PINCTRL_GROUP(i2c0), + UNIPHIER_PINCTRL_GROUP(i2c1), + UNIPHIER_PINCTRL_GROUP(i2c2), + UNIPHIER_PINCTRL_GROUP(i2c3), + UNIPHIER_PINCTRL_GROUP(nand), + UNIPHIER_PINCTRL_GROUP(nand_cs1), + UNIPHIER_PINCTRL_GROUP(uart0), + UNIPHIER_PINCTRL_GROUP(uart1), + UNIPHIER_PINCTRL_GROUP(uart2), + UNIPHIER_PINCTRL_GROUP(uart3), + UNIPHIER_PINCTRL_GROUP(usb0), + UNIPHIER_PINCTRL_GROUP(usb1), + UNIPHIER_PINCTRL_GROUP(usb2), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range0), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range1), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range2), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range3), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq_range0), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq_range1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port00, port_range0, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port01, port_range0, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port02, port_range0, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port03, port_range0, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port04, port_range0, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port05, port_range0, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port06, port_range0, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port07, port_range0, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port10, port_range0, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port11, port_range0, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port12, port_range0, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port13, port_range0, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port14, port_range0, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port15, port_range0, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port16, port_range0, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port17, port_range0, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port20, port_range0, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port21, port_range0, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port22, port_range0, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port23, port_range0, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port24, port_range0, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port25, port_range0, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port26, port_range0, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port27, port_range0, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port30, port_range0, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port31, port_range0, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port32, port_range0, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port33, port_range0, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port34, port_range0, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port35, port_range0, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port36, port_range0, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port37, port_range0, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port40, port_range0, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port41, port_range0, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port42, port_range0, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port43, port_range0, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port44, port_range0, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port45, port_range0, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port46, port_range0, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port47, port_range0, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port50, port_range0, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port51, port_range0, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port52, port_range0, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port53, port_range0, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port54, port_range0, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port55, port_range0, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port56, port_range0, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port57, port_range0, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port60, port_range0, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port61, port_range0, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port62, port_range0, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port63, port_range0, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port64, port_range0, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port65, port_range0, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port66, port_range0, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port67, port_range0, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port70, port_range0, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port71, port_range0, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port72, port_range0, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port73, port_range0, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port74, port_range0, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port75, port_range0, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port76, port_range0, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port77, port_range0, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port80, port_range0, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port81, port_range0, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port82, port_range0, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port83, port_range0, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port84, port_range0, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port85, port_range0, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port86, port_range0, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port87, port_range0, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port90, port_range0, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port91, port_range0, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port92, port_range0, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port93, port_range0, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port94, port_range0, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port95, port_range0, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port96, port_range0, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port97, port_range0, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port100, port_range0, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port101, port_range0, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port102, port_range0, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port103, port_range0, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port104, port_range0, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port105, port_range0, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port106, port_range0, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port107, port_range0, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port110, port_range0, 88), + UNIPHIER_PINCTRL_GROUP_SINGLE(port111, port_range0, 89), + UNIPHIER_PINCTRL_GROUP_SINGLE(port112, port_range0, 90), + UNIPHIER_PINCTRL_GROUP_SINGLE(port113, port_range0, 91), + UNIPHIER_PINCTRL_GROUP_SINGLE(port114, port_range0, 92), + UNIPHIER_PINCTRL_GROUP_SINGLE(port115, port_range0, 93), + UNIPHIER_PINCTRL_GROUP_SINGLE(port116, port_range0, 94), + UNIPHIER_PINCTRL_GROUP_SINGLE(port117, port_range0, 95), + UNIPHIER_PINCTRL_GROUP_SINGLE(port120, port_range0, 96), + UNIPHIER_PINCTRL_GROUP_SINGLE(port121, port_range0, 97), + UNIPHIER_PINCTRL_GROUP_SINGLE(port122, port_range0, 98), + UNIPHIER_PINCTRL_GROUP_SINGLE(port123, port_range0, 99), + UNIPHIER_PINCTRL_GROUP_SINGLE(port124, port_range0, 100), + UNIPHIER_PINCTRL_GROUP_SINGLE(port130, port_range1, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port131, port_range1, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port140, port_range2, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port141, port_range2, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port142, port_range2, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port143, port_range2, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port144, port_range2, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port145, port_range2, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port146, port_range2, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port147, port_range2, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port166, port_range3, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq0, xirq_range0, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq1, xirq_range0, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq2, xirq_range0, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq3, xirq_range0, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq4, xirq_range0, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq5, xirq_range0, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq6, xirq_range0, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq7, xirq_range0, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq8, xirq_range0, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq9, xirq_range0, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq10, xirq_range0, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq11, xirq_range0, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq12, xirq_range0, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq14, xirq_range1, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq15, xirq_range1, 1), +}; + +static const char * const emmc_groups[] = {"emmc", "emmc_dat8"}; +static const char * const i2c0_groups[] = {"i2c0"}; +static const char * const i2c1_groups[] = {"i2c1"}; +static const char * const i2c2_groups[] = {"i2c2"}; +static const char * const i2c3_groups[] = {"i2c3"}; +static const char * const nand_groups[] = {"nand", "nand_cs1"}; +static const char * const uart0_groups[] = {"uart0"}; +static const char * const uart1_groups[] = {"uart1"}; +static const char * const uart2_groups[] = {"uart2"}; +static const char * const uart3_groups[] = {"uart3"}; +static const char * const usb0_groups[] = {"usb0"}; +static const char * const usb1_groups[] = {"usb1"}; +static const char * const usb2_groups[] = {"usb2"}; +static const char * const port_groups[] = { + "port00", "port01", "port02", "port03", + "port04", "port05", "port06", "port07", + "port10", "port11", "port12", "port13", + "port14", "port15", "port16", "port17", + "port20", "port21", "port22", "port23", + "port24", "port25", "port26", "port27", + "port30", "port31", "port32", "port33", + "port34", "port35", "port36", "port37", + "port40", "port41", "port42", "port43", + "port44", "port45", "port46", "port47", + "port50", "port51", "port52", "port53", + "port54", "port55", "port56", "port57", + "port60", "port61", "port62", "port63", + "port64", "port65", "port66", "port67", + "port70", "port71", "port72", "port73", + "port74", "port75", "port76", "port77", + "port80", "port81", "port82", "port83", + "port84", "port85", "port86", "port87", + "port90", "port91", "port92", "port93", + "port94", "port95", "port96", "port97", + "port100", "port101", "port102", "port103", + "port104", "port105", "port106", "port107", + "port110", "port111", "port112", "port113", + "port114", "port115", "port116", "port117", + "port120", "port121", "port122", "port123", + "port124", "port125", "port126", "port127", + "port130", "port131", "port132", "port133", + "port134", "port135", "port136", "port137", + "port140", "port141", "port142", "port143", + "port144", "port145", "port146", "port147", + /* port150-164 missing */ + /* none */ "port165", +}; +static const char * const xirq_groups[] = { + "xirq0", "xirq1", "xirq2", "xirq3", + "xirq4", "xirq5", "xirq6", "xirq7", + "xirq8", "xirq9", "xirq10", "xirq11", + "xirq12", /* none*/ "xirq14", "xirq15", +}; + +static const struct uniphier_pinmux_function ph1_sld8_functions[] = { + UNIPHIER_PINMUX_FUNCTION(emmc), + UNIPHIER_PINMUX_FUNCTION(i2c0), + UNIPHIER_PINMUX_FUNCTION(i2c1), + UNIPHIER_PINMUX_FUNCTION(i2c2), + UNIPHIER_PINMUX_FUNCTION(i2c3), + UNIPHIER_PINMUX_FUNCTION(nand), + UNIPHIER_PINMUX_FUNCTION(uart0), + UNIPHIER_PINMUX_FUNCTION(uart1), + UNIPHIER_PINMUX_FUNCTION(uart2), + UNIPHIER_PINMUX_FUNCTION(uart3), + UNIPHIER_PINMUX_FUNCTION(usb0), + UNIPHIER_PINMUX_FUNCTION(usb1), + UNIPHIER_PINMUX_FUNCTION(usb2), + UNIPHIER_PINMUX_FUNCTION(port), + UNIPHIER_PINMUX_FUNCTION(xirq), +}; + +static struct uniphier_pinctrl_socdata ph1_sld8_pindata = { + .groups = ph1_sld8_groups, + .groups_count = ARRAY_SIZE(ph1_sld8_groups), + .functions = ph1_sld8_functions, + .functions_count = ARRAY_SIZE(ph1_sld8_functions), + .mux_bits = 8, + .reg_stride = 4, + .load_pinctrl = false, +}; + +static struct pinctrl_desc ph1_sld8_pinctrl_desc = { + .name = DRIVER_NAME, + .pins = ph1_sld8_pins, + .npins = ARRAY_SIZE(ph1_sld8_pins), + .owner = THIS_MODULE, +}; + +static int ph1_sld8_pinctrl_probe(struct platform_device *pdev) +{ + return uniphier_pinctrl_probe(pdev, &ph1_sld8_pinctrl_desc, + &ph1_sld8_pindata); +} + +static const struct of_device_id ph1_sld8_pinctrl_match[] = { + { .compatible = "socionext,ph1-sld8-pinctrl" }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, ph1_sld8_pinctrl_match); + +static struct platform_driver ph1_sld8_pinctrl_driver = { + .probe = ph1_sld8_pinctrl_probe, + .remove = uniphier_pinctrl_remove, + .driver = { + .name = DRIVER_NAME, + .of_match_table = ph1_sld8_pinctrl_match, + }, +}; +module_platform_driver(ph1_sld8_pinctrl_driver); + +MODULE_AUTHOR("Masahiro Yamada "); +MODULE_DESCRIPTION("UniPhier PH1-sLD8 pinctrl driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pinctrl/uniphier/pinctrl-proxstream2.c b/drivers/pinctrl/uniphier/pinctrl-proxstream2.c new file mode 100644 index 00000000000000..3f036e236ad902 --- /dev/null +++ b/drivers/pinctrl/uniphier/pinctrl-proxstream2.c @@ -0,0 +1,1269 @@ +/* + * Copyright (C) 2015 Masahiro Yamada + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include +#include +#include + +#include "pinctrl-uniphier.h" + +#define DRIVER_NAME "proxstream2-pinctrl" + +static const struct pinctrl_pin_desc proxstream2_pins[] = { + UNIPHIER_PINCTRL_PIN(0, "ED0", UNIPHIER_PIN_IECTRL_NONE, + 0, UNIPHIER_PIN_DRV_4_8, + 0, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(1, "ED1", UNIPHIER_PIN_IECTRL_NONE, + 1, UNIPHIER_PIN_DRV_4_8, + 1, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(2, "ED2", UNIPHIER_PIN_IECTRL_NONE, + 2, UNIPHIER_PIN_DRV_4_8, + 2, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(3, "ED3", UNIPHIER_PIN_IECTRL_NONE, + 3, UNIPHIER_PIN_DRV_4_8, + 3, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(4, "ED4", UNIPHIER_PIN_IECTRL_NONE, + 4, UNIPHIER_PIN_DRV_4_8, + 4, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(5, "ED5", UNIPHIER_PIN_IECTRL_NONE, + 5, UNIPHIER_PIN_DRV_4_8, + 5, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(6, "ED6", UNIPHIER_PIN_IECTRL_NONE, + 6, UNIPHIER_PIN_DRV_4_8, + 6, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(7, "ED7", UNIPHIER_PIN_IECTRL_NONE, + 7, UNIPHIER_PIN_DRV_4_8, + 7, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(8, "XERWE0", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_4_8, + 8, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(9, "XERWE1", UNIPHIER_PIN_IECTRL_NONE, + 9, UNIPHIER_PIN_DRV_4_8, + 9, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(10, "ERXW", UNIPHIER_PIN_IECTRL_NONE, + 10, UNIPHIER_PIN_DRV_4_8, + 10, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(11, "ES0", UNIPHIER_PIN_IECTRL_NONE, + 11, UNIPHIER_PIN_DRV_4_8, + 11, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(12, "ES1", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_4_8, + 12, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(13, "ES2", UNIPHIER_PIN_IECTRL_NONE, + 13, UNIPHIER_PIN_DRV_4_8, + 13, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(14, "XECS1", UNIPHIER_PIN_IECTRL_NONE, + 14, UNIPHIER_PIN_DRV_4_8, + 14, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(15, "SMTRST0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 15, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(16, "SMTCMD0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 16, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(17, "SMTD0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 17, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(18, "SMTSEL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 18, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(19, "SMTCLK0CG", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 19, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(20, "SMTDET0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 20, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(21, "SMTRST1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 21, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(22, "SMTCMD1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 22, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(23, "SMTD1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 23, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(24, "SMTSEL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 24, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(25, "SMTCLK1CG", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 25, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(26, "SMTDET1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 26, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(27, "XIRQ18", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 27, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(28, "XIRQ19", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 28, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(29, "XIRQ20", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 29, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(30, "XNFRE", UNIPHIER_PIN_IECTRL_NONE, + 30, UNIPHIER_PIN_DRV_4_8, + 30, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(31, "XNFWE", UNIPHIER_PIN_IECTRL_NONE, + 31, UNIPHIER_PIN_DRV_4_8, + 31, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(32, "NFALE", UNIPHIER_PIN_IECTRL_NONE, + 32, UNIPHIER_PIN_DRV_4_8, + 32, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(33, "NFCLE", UNIPHIER_PIN_IECTRL_NONE, + 33, UNIPHIER_PIN_DRV_4_8, + 33, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(34, "XNFWP", UNIPHIER_PIN_IECTRL_NONE, + 34, UNIPHIER_PIN_DRV_4_8, + 34, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(35, "XNFCE0", UNIPHIER_PIN_IECTRL_NONE, + 35, UNIPHIER_PIN_DRV_4_8, + 35, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(36, "NFRYBY0", UNIPHIER_PIN_IECTRL_NONE, + 36, UNIPHIER_PIN_DRV_4_8, + 36, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(37, "XNFCE1", UNIPHIER_PIN_IECTRL_NONE, + 37, UNIPHIER_PIN_DRV_4_8, + 37, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(38, "NFRYBY1", UNIPHIER_PIN_IECTRL_NONE, + 38, UNIPHIER_PIN_DRV_4_8, + 38, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(39, "NFD0", UNIPHIER_PIN_IECTRL_NONE, + 39, UNIPHIER_PIN_DRV_4_8, + 39, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(40, "NFD1", UNIPHIER_PIN_IECTRL_NONE, + 40, UNIPHIER_PIN_DRV_4_8, + 40, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(41, "NFD2", UNIPHIER_PIN_IECTRL_NONE, + 41, UNIPHIER_PIN_DRV_4_8, + 41, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(42, "NFD3", UNIPHIER_PIN_IECTRL_NONE, + 42, UNIPHIER_PIN_DRV_4_8, + 42, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(43, "NFD4", UNIPHIER_PIN_IECTRL_NONE, + 43, UNIPHIER_PIN_DRV_4_8, + 43, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(44, "NFD5", UNIPHIER_PIN_IECTRL_NONE, + 44, UNIPHIER_PIN_DRV_4_8, + 44, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(45, "NFD6", UNIPHIER_PIN_IECTRL_NONE, + 45, UNIPHIER_PIN_DRV_4_8, + 45, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(46, "NFD7", UNIPHIER_PIN_IECTRL_NONE, + 46, UNIPHIER_PIN_DRV_4_8, + 46, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(47, "SDCLK", UNIPHIER_PIN_IECTRL_NONE, + 0, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(48, "SDCMD", UNIPHIER_PIN_IECTRL_NONE, + 4, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(49, "SDDAT0", UNIPHIER_PIN_IECTRL_NONE, + 8, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(50, "SDDAT1", UNIPHIER_PIN_IECTRL_NONE, + 12, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(51, "SDDAT2", UNIPHIER_PIN_IECTRL_NONE, + 16, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(52, "SDDAT3", UNIPHIER_PIN_IECTRL_NONE, + 20, UNIPHIER_PIN_DRV_8_12_16_20, + -1, UNIPHIER_PIN_PULL_UP_FIXED), + UNIPHIER_PINCTRL_PIN(53, "SDCD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 53, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(54, "SDWP", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 54, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(55, "SDVOLC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 55, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(56, "USB0VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 56, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(57, "USB0OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 57, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(58, "USB1VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 58, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(59, "USB1OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 59, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(60, "USB2VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 60, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(61, "USB2OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 61, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(62, "USB3VBUS", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 62, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(63, "USB3OD", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 63, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(64, "CH0CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 64, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(65, "CH0PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 65, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(66, "CH0VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 66, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(67, "CH0DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 67, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(68, "CH1CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 68, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(69, "CH1PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 69, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(70, "CH1VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 70, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(71, "CH1DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 71, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(72, "XIRQ9", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 72, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(73, "XIRQ10", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 73, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(74, "XIRQ16", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 74, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(75, "CH4CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 75, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(76, "CH4PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 76, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(77, "CH4VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 77, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(78, "CH4DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 78, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(79, "CH5CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 79, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(80, "CH5PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 80, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(81, "CH5VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 81, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(82, "CH5DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 82, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(83, "CH6CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 83, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(84, "CH6PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 84, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(85, "CH6VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 85, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(86, "CH6DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 86, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(87, "STS0CLKO", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 87, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(88, "STS0SYNCO", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 88, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(89, "STS0VALO", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 89, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(90, "STS0DATAO", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 90, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(91, "XIRQ17", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 91, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(92, "PORT163", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 92, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(93, "PORT165", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 93, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(94, "PORT166", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 94, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(95, "PORT132", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 95, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(96, "PORT133", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 96, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(97, "AO2IEC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 97, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(98, "AI2ADCCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 98, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(99, "AI2BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 99, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(100, "AI2LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 100, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(101, "AI2D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 101, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(102, "AI2D1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 102, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(103, "AI2D2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 103, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(104, "AI2D3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 104, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(105, "AO3DACCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 105, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(106, "AO3BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 106, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(107, "AO3LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 107, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(108, "AO3DMIX", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 108, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(109, "SDA0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 109, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(110, "SCL0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 110, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(111, "SDA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 111, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(112, "SCL1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 112, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(113, "TXD2", 0, + 113, UNIPHIER_PIN_DRV_4_8, + 113, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(114, "RXD2", 0, + 114, UNIPHIER_PIN_DRV_4_8, + 114, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(115, "TXD1", 0, + 115, UNIPHIER_PIN_DRV_4_8, + 115, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(116, "RXD1", 0, + 116, UNIPHIER_PIN_DRV_4_8, + 116, UNIPHIER_PIN_PULL_UP), + UNIPHIER_PINCTRL_PIN(117, "PORT190", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 117, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(118, "VI1HSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 118, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(119, "VI1VSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 119, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(120, "VI1DE", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 120, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(121, "XIRQ3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 121, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(122, "XIRQ4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 122, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(123, "VI1G2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 123, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(124, "VI1G3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 124, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(125, "VI1G4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 125, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(126, "VI1G5", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 126, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(127, "VI1G6", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 127, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(128, "VI1G7", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 128, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(129, "VI1G8", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 129, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(130, "VI1G9", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 130, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(131, "VI1CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 131, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(132, "PORT05", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 132, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(133, "PORT06", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 133, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(134, "VI1R2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 134, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(135, "VI1R3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 135, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(136, "VI1R4", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 136, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(137, "VI1R5", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 137, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(138, "VI1R6", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 138, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(139, "VI1R7", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 139, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(140, "VI1R8", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 140, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(141, "VI1R9", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 141, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(142, "LPST", UNIPHIER_PIN_IECTRL_NONE, + 142, UNIPHIER_PIN_DRV_4_8, + 142, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(143, "MDC", 0, + 143, UNIPHIER_PIN_DRV_4_8, + 143, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(144, "MDIO", 0, + 144, UNIPHIER_PIN_DRV_4_8, + 144, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(145, "MDIO_INTL", 0, + 145, UNIPHIER_PIN_DRV_4_8, + 145, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(146, "PHYRSTL", 0, + 146, UNIPHIER_PIN_DRV_4_8, + 146, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(147, "RGMII_RXCLK", 0, + 147, UNIPHIER_PIN_DRV_4_8, + 147, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(148, "RGMII_RXD0", 0, + 148, UNIPHIER_PIN_DRV_4_8, + 148, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(149, "RGMII_RXD1", 0, + 149, UNIPHIER_PIN_DRV_4_8, + 149, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(150, "RGMII_RXD2", 0, + 150, UNIPHIER_PIN_DRV_4_8, + 150, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(151, "RGMII_RXD3", 0, + 151, UNIPHIER_PIN_DRV_4_8, + 151, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(152, "RGMII_RXCTL", 0, + 152, UNIPHIER_PIN_DRV_4_8, + 152, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(153, "RGMII_TXCLK", 0, + 153, UNIPHIER_PIN_DRV_4_8, + 153, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(154, "RGMII_TXD0", 0, + 154, UNIPHIER_PIN_DRV_4_8, + 154, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(155, "RGMII_TXD1", 0, + 155, UNIPHIER_PIN_DRV_4_8, + 155, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(156, "RGMII_TXD2", 0, + 156, UNIPHIER_PIN_DRV_4_8, + 156, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(157, "RGMII_TXD3", 0, + 157, UNIPHIER_PIN_DRV_4_8, + 157, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(158, "RGMII_TXCTL", 0, + 158, UNIPHIER_PIN_DRV_4_8, + 158, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(159, "SDA3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 159, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(160, "SCL3", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 160, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(161, "AI1ADCCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 161, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(162, "AI1BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 162, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(163, "CH2CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 163, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(164, "CH2PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 164, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(165, "CH2VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 165, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(166, "CH2DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 166, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(167, "CH3CLK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 167, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(168, "CH3PSYNC", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 168, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(169, "CH3VAL", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 169, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(170, "CH3DATA", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 170, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(171, "SDA2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 171, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(172, "SCL2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 172, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(173, "AI1LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 173, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(174, "AI1D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 174, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(175, "AO2LRCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 175, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(176, "AO2D0", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 176, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(177, "AO2DACCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 177, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(178, "AO2BCK", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 178, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(179, "PORT222", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 179, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(180, "PORT223", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 180, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(181, "PORT224", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 181, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(182, "PORT225", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 182, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(183, "PORT226", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 183, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(184, "PORT227", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 184, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(185, "PORT230", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 185, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(186, "FANPWM", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 186, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(187, "HRDDCSDA0", 0, + 187, UNIPHIER_PIN_DRV_4_8, + 187, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(188, "HRDDCSCL0", 0, + 188, UNIPHIER_PIN_DRV_4_8, + 188, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(189, "HRDDCSDA1", 0, + 189, UNIPHIER_PIN_DRV_4_8, + 189, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(190, "HRDDCSCL1", 0, + 190, UNIPHIER_PIN_DRV_4_8, + 190, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(191, "HTDDCSDA0", 0, + 191, UNIPHIER_PIN_DRV_4_8, + 191, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(192, "HTDDCSCL0", 0, + 192, UNIPHIER_PIN_DRV_4_8, + 192, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(193, "HTDDCSDA1", 0, + 193, UNIPHIER_PIN_DRV_4_8, + 193, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(194, "HTDDCSCL1", 0, + 194, UNIPHIER_PIN_DRV_4_8, + 194, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(195, "PORT241", 0, + 195, UNIPHIER_PIN_DRV_4_8, + 195, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(196, "PORT242", 0, + 196, UNIPHIER_PIN_DRV_4_8, + 196, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(197, "PORT243", 0, + 197, UNIPHIER_PIN_DRV_4_8, + 197, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(198, "MVSYNC", 0, + 198, UNIPHIER_PIN_DRV_4_8, + 198, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(199, "SPISYNC0", UNIPHIER_PIN_IECTRL_NONE, + 199, UNIPHIER_PIN_DRV_4_8, + 199, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(200, "SPISCLK0", UNIPHIER_PIN_IECTRL_NONE, + 200, UNIPHIER_PIN_DRV_4_8, + 200, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(201, "SPITXD0", UNIPHIER_PIN_IECTRL_NONE, + 201, UNIPHIER_PIN_DRV_4_8, + 201, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(202, "SPIRXD0", UNIPHIER_PIN_IECTRL_NONE, + 202, UNIPHIER_PIN_DRV_4_8, + 202, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(203, "CK54EXI", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 203, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(204, "AEXCKA1", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 204, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(205, "AEXCKA2", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 205, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(206, "CK27EXI", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_8, + 206, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(207, "STCDIN", 0, + 207, UNIPHIER_PIN_DRV_4_8, + 207, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(208, "PHSYNI", 0, + 208, UNIPHIER_PIN_DRV_4_8, + 208, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(209, "PVSYNI", 0, + 209, UNIPHIER_PIN_DRV_4_8, + 209, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(210, "MVSYN", UNIPHIER_PIN_IECTRL_NONE, + 210, UNIPHIER_PIN_DRV_4_8, + 210, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(211, "STCV", UNIPHIER_PIN_IECTRL_NONE, + 211, UNIPHIER_PIN_DRV_4_8, + 211, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(212, "PORT262", UNIPHIER_PIN_IECTRL_NONE, + 212, UNIPHIER_PIN_DRV_4_8, + 212, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(213, "USB0VBUS_IRQ", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 213, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(214, "USB1VBUS_IRQ", UNIPHIER_PIN_IECTRL_NONE, + -1, UNIPHIER_PIN_DRV_FIXED_4, + 214, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(215, "PORT265", UNIPHIER_PIN_IECTRL_NONE, + 215, UNIPHIER_PIN_DRV_4_8, + 215, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(216, "CK25O", 0, + 216, UNIPHIER_PIN_DRV_4_8, + 216, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(217, "TXD0", 0, + 217, UNIPHIER_PIN_DRV_4_8, + 217, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(218, "RXD0", 0, + 218, UNIPHIER_PIN_DRV_4_8, + 218, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(219, "TXD3", 0, + 219, UNIPHIER_PIN_DRV_4_8, + 219, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(220, "RXD3", 0, + 220, UNIPHIER_PIN_DRV_4_8, + 220, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(221, "PORT273", 0, + 221, UNIPHIER_PIN_DRV_4_8, + 221, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(222, "STCDOUTC", 0, + 222, UNIPHIER_PIN_DRV_4_8, + 222, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(223, "PORT274", 0, + 223, UNIPHIER_PIN_DRV_4_8, + 223, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(224, "PORT275", 0, + 224, UNIPHIER_PIN_DRV_4_8, + 224, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(225, "PORT276", 0, + 225, UNIPHIER_PIN_DRV_4_8, + 225, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(226, "PORT277", 0, + 226, UNIPHIER_PIN_DRV_4_8, + 226, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(227, "PORT280", 0, + 227, UNIPHIER_PIN_DRV_4_8, + 227, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(228, "PORT281", 0, + 228, UNIPHIER_PIN_DRV_4_8, + 228, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(229, "PORT282", 0, + 229, UNIPHIER_PIN_DRV_4_8, + 229, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(230, "PORT283", 0, + 230, UNIPHIER_PIN_DRV_4_8, + 230, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(231, "PORT284", 0, + 231, UNIPHIER_PIN_DRV_4_8, + 231, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(232, "PORT285", 0, + 232, UNIPHIER_PIN_DRV_4_8, + 232, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(233, "T0HPD", 0, + 233, UNIPHIER_PIN_DRV_4_8, + 233, UNIPHIER_PIN_PULL_DOWN), + UNIPHIER_PINCTRL_PIN(234, "T1HPD", 0, + 234, UNIPHIER_PIN_DRV_4_8, + 234, UNIPHIER_PIN_PULL_DOWN), +}; + +static const unsigned emmc_pins[] = {36, 37, 38, 39, 40, 41, 42}; +static const unsigned emmc_muxvals[] = {9, 9, 9, 9, 9, 9, 9}; +static const unsigned emmc_dat8_pins[] = {43, 44, 45, 46}; +static const unsigned emmc_dat8_muxvals[] = {9, 9, 9, 9}; +static const unsigned i2c0_pins[] = {109, 110}; +static const unsigned i2c0_muxvals[] = {8, 8}; +static const unsigned i2c1_pins[] = {111, 112}; +static const unsigned i2c1_muxvals[] = {8, 8}; +static const unsigned i2c2_pins[] = {171, 172}; +static const unsigned i2c2_muxvals[] = {8, 8}; +static const unsigned i2c3_pins[] = {159, 160}; +static const unsigned i2c3_muxvals[] = {8, 8}; +static const unsigned i2c5_pins[] = {183, 184}; +static const unsigned i2c5_muxvals[] = {11, 11}; +static const unsigned i2c6_pins[] = {185, 186}; +static const unsigned i2c6_muxvals[] = {11, 11}; +static const unsigned nand_pins[] = {30, 31, 32, 33, 34, 35, 36, 39, 40, 41, + 42, 43, 44, 45, 46}; +static const unsigned nand_muxvals[] = {8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, + 8, 8}; +static const unsigned nand_cs1_pins[] = {37, 38}; +static const unsigned nand_cs1_muxvals[] = {8, 8}; +static const unsigned uart0_pins[] = {217, 218}; +static const unsigned uart0_muxvals[] = {8, 8}; +static const unsigned uart0b_pins[] = {179, 180}; +static const unsigned uart0b_muxvals[] = {10, 10}; +static const unsigned uart1_pins[] = {115, 116}; +static const unsigned uart1_muxvals[] = {8, 8}; +static const unsigned uart2_pins[] = {113, 114}; +static const unsigned uart2_muxvals[] = {8, 8}; +static const unsigned uart3_pins[] = {219, 220}; +static const unsigned uart3_muxvals[] = {8, 8}; +static const unsigned uart3b_pins[] = {181, 182}; +static const unsigned uart3b_muxvals[] = {10, 10}; +static const unsigned usb0_pins[] = {56, 57}; +static const unsigned usb0_muxvals[] = {8, 8}; +static const unsigned usb1_pins[] = {58, 59}; +static const unsigned usb1_muxvals[] = {8, 8}; +static const unsigned usb2_pins[] = {60, 61}; +static const unsigned usb2_muxvals[] = {8, 8}; +static const unsigned usb3_pins[] = {62, 63}; +static const unsigned usb3_muxvals[] = {8, 8}; +static const unsigned port_range0_pins[] = { + 127, 128, 129, 130, 131, 132, 133, 134, /* PORT0x */ + 135, 136, 137, 138, 139, 140, 141, 142, /* PORT1x */ + 0, 1, 2, 3, 4, 5, 6, 7, /* PORT2x */ + 8, 9, 10, 11, 12, 13, 14, 15, /* PORT3x */ + 16, 17, 18, 19, 21, 22, 23, 24, /* PORT4x */ + 25, 30, 31, 32, 33, 34, 35, 36, /* PORT5x */ + 37, 38, 39, 40, 41, 42, 43, 44, /* PORT6x */ + 45, 46, 47, 48, 49, 50, 51, 52, /* PORT7x */ + 53, 54, 55, 56, 57, 58, 59, 60, /* PORT8x */ + 61, 62, 63, 64, 65, 66, 67, 68, /* PORT9x */ + 69, 70, 71, 76, 77, 78, 79, 80, /* PORT10x */ +}; +static const unsigned port_range0_muxvals[] = { + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT0x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT1x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT2x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT3x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT4x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT5x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT6x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT7x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT8x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT9x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT10x */ +}; +static const unsigned port_range1_pins[] = { + 81, 82, 83, 84, 85, 86, 87, 88, /* PORT12x */ + 89, 90, 95, 96, 97, 98, 99, 100, /* PORT13x */ + 101, 102, 103, 104, 105, 106, 107, 108, /* PORT14x */ + 118, 119, 120, 121, 122, 123, 124, 125, /* PORT15x */ + 126, 72, 73, 92, 177, 93, 94, 176, /* PORT16x */ + 74, 91, 27, 28, 29, 75, 20, 26, /* PORT17x */ + 109, 110, 111, 112, 113, 114, 115, 116, /* PORT18x */ + 117, 143, 144, 145, 146, 147, 148, 149, /* PORT19x */ + 150, 151, 152, 153, 154, 155, 156, 157, /* PORT20x */ + 158, 159, 160, 161, 162, 163, 164, 165, /* PORT21x */ + 166, 178, 179, 180, 181, 182, 183, 184, /* PORT22x */ + 185, 187, 188, 189, 190, 191, 192, 193, /* PORT23x */ + 194, 195, 196, 197, 198, 199, 200, 201, /* PORT24x */ + 202, 203, 204, 205, 206, 207, 208, 209, /* PORT25x */ + 210, 211, 212, 213, 214, 215, 216, 217, /* PORT26x */ + 218, 219, 220, 221, 223, 224, 225, 226, /* PORT27x */ + 227, 228, 229, 230, 231, 232, 233, 234, /* PORT28x */ +}; +static const unsigned port_range1_muxvals[] = { + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT12x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT13x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT14x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT15x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT16x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT17x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT18x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT19x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT20x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT21x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT22x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT23x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT24x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT25x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT26x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT27x */ + 15, 15, 15, 15, 15, 15, 15, 15, /* PORT28x */ +}; +static const unsigned xirq_pins[] = { + 118, 119, 120, 121, 122, 123, 124, 125, /* XIRQ0-7 */ + 126, 72, 73, 92, 177, 93, 94, 176, /* XIRQ8-15 */ + 74, 91, 27, 28, 29, 75, 20, 26, /* XIRQ16-23 */ +}; +static const unsigned xirq_muxvals[] = { + 14, 14, 14, 14, 14, 14, 14, 14, /* XIRQ0-7 */ + 14, 14, 14, 14, 14, 14, 14, 14, /* XIRQ8-15 */ + 14, 14, 14, 14, 14, 14, 14, 14, /* XIRQ16-23 */ +}; + +static const struct uniphier_pinctrl_group proxstream2_groups[] = { + UNIPHIER_PINCTRL_GROUP(emmc), + UNIPHIER_PINCTRL_GROUP(emmc_dat8), + UNIPHIER_PINCTRL_GROUP(i2c0), + UNIPHIER_PINCTRL_GROUP(i2c1), + UNIPHIER_PINCTRL_GROUP(i2c2), + UNIPHIER_PINCTRL_GROUP(i2c3), + UNIPHIER_PINCTRL_GROUP(i2c5), + UNIPHIER_PINCTRL_GROUP(i2c6), + UNIPHIER_PINCTRL_GROUP(nand), + UNIPHIER_PINCTRL_GROUP(nand_cs1), + UNIPHIER_PINCTRL_GROUP(uart0), + UNIPHIER_PINCTRL_GROUP(uart0b), + UNIPHIER_PINCTRL_GROUP(uart1), + UNIPHIER_PINCTRL_GROUP(uart2), + UNIPHIER_PINCTRL_GROUP(uart3), + UNIPHIER_PINCTRL_GROUP(uart3b), + UNIPHIER_PINCTRL_GROUP(usb0), + UNIPHIER_PINCTRL_GROUP(usb1), + UNIPHIER_PINCTRL_GROUP(usb2), + UNIPHIER_PINCTRL_GROUP(usb3), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range0), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(port_range1), + UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(xirq), + UNIPHIER_PINCTRL_GROUP_SINGLE(port00, port_range0, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port01, port_range0, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port02, port_range0, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port03, port_range0, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port04, port_range0, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port05, port_range0, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port06, port_range0, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port07, port_range0, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port10, port_range0, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port11, port_range0, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port12, port_range0, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port13, port_range0, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port14, port_range0, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port15, port_range0, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port16, port_range0, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port17, port_range0, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port20, port_range0, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port21, port_range0, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port22, port_range0, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port23, port_range0, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port24, port_range0, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port25, port_range0, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port26, port_range0, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port27, port_range0, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port30, port_range0, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port31, port_range0, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port32, port_range0, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port33, port_range0, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port34, port_range0, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port35, port_range0, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port36, port_range0, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port37, port_range0, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port40, port_range0, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port41, port_range0, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port42, port_range0, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port43, port_range0, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port44, port_range0, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port45, port_range0, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port46, port_range0, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port47, port_range0, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port50, port_range0, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port51, port_range0, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port52, port_range0, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port53, port_range0, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port54, port_range0, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port55, port_range0, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port56, port_range0, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port57, port_range0, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port60, port_range0, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port61, port_range0, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port62, port_range0, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port63, port_range0, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port64, port_range0, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port65, port_range0, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port66, port_range0, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port67, port_range0, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port70, port_range0, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port71, port_range0, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port72, port_range0, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port73, port_range0, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port74, port_range0, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port75, port_range0, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port76, port_range0, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port77, port_range0, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port80, port_range0, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port81, port_range0, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port82, port_range0, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port83, port_range0, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port84, port_range0, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port85, port_range0, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port86, port_range0, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port87, port_range0, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port90, port_range0, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port91, port_range0, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port92, port_range0, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port93, port_range0, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port94, port_range0, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port95, port_range0, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port96, port_range0, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port97, port_range0, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port100, port_range0, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port101, port_range0, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port102, port_range0, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port103, port_range0, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port104, port_range0, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port105, port_range0, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port106, port_range0, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port107, port_range0, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port120, port_range1, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(port121, port_range1, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(port122, port_range1, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(port123, port_range1, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(port124, port_range1, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(port125, port_range1, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(port126, port_range1, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(port127, port_range1, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(port130, port_range1, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(port131, port_range1, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(port132, port_range1, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(port133, port_range1, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(port134, port_range1, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(port135, port_range1, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(port136, port_range1, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(port137, port_range1, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(port140, port_range1, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(port141, port_range1, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(port142, port_range1, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(port143, port_range1, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(port144, port_range1, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(port145, port_range1, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(port146, port_range1, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(port147, port_range1, 23), + UNIPHIER_PINCTRL_GROUP_SINGLE(port150, port_range1, 24), + UNIPHIER_PINCTRL_GROUP_SINGLE(port151, port_range1, 25), + UNIPHIER_PINCTRL_GROUP_SINGLE(port152, port_range1, 26), + UNIPHIER_PINCTRL_GROUP_SINGLE(port153, port_range1, 27), + UNIPHIER_PINCTRL_GROUP_SINGLE(port154, port_range1, 28), + UNIPHIER_PINCTRL_GROUP_SINGLE(port155, port_range1, 29), + UNIPHIER_PINCTRL_GROUP_SINGLE(port156, port_range1, 30), + UNIPHIER_PINCTRL_GROUP_SINGLE(port157, port_range1, 31), + UNIPHIER_PINCTRL_GROUP_SINGLE(port160, port_range1, 32), + UNIPHIER_PINCTRL_GROUP_SINGLE(port161, port_range1, 33), + UNIPHIER_PINCTRL_GROUP_SINGLE(port162, port_range1, 34), + UNIPHIER_PINCTRL_GROUP_SINGLE(port163, port_range1, 35), + UNIPHIER_PINCTRL_GROUP_SINGLE(port164, port_range1, 36), + UNIPHIER_PINCTRL_GROUP_SINGLE(port165, port_range1, 37), + UNIPHIER_PINCTRL_GROUP_SINGLE(port166, port_range1, 38), + UNIPHIER_PINCTRL_GROUP_SINGLE(port167, port_range1, 39), + UNIPHIER_PINCTRL_GROUP_SINGLE(port170, port_range1, 40), + UNIPHIER_PINCTRL_GROUP_SINGLE(port171, port_range1, 41), + UNIPHIER_PINCTRL_GROUP_SINGLE(port172, port_range1, 42), + UNIPHIER_PINCTRL_GROUP_SINGLE(port173, port_range1, 43), + UNIPHIER_PINCTRL_GROUP_SINGLE(port174, port_range1, 44), + UNIPHIER_PINCTRL_GROUP_SINGLE(port175, port_range1, 45), + UNIPHIER_PINCTRL_GROUP_SINGLE(port176, port_range1, 46), + UNIPHIER_PINCTRL_GROUP_SINGLE(port177, port_range1, 47), + UNIPHIER_PINCTRL_GROUP_SINGLE(port180, port_range1, 48), + UNIPHIER_PINCTRL_GROUP_SINGLE(port181, port_range1, 49), + UNIPHIER_PINCTRL_GROUP_SINGLE(port182, port_range1, 50), + UNIPHIER_PINCTRL_GROUP_SINGLE(port183, port_range1, 51), + UNIPHIER_PINCTRL_GROUP_SINGLE(port184, port_range1, 52), + UNIPHIER_PINCTRL_GROUP_SINGLE(port185, port_range1, 53), + UNIPHIER_PINCTRL_GROUP_SINGLE(port186, port_range1, 54), + UNIPHIER_PINCTRL_GROUP_SINGLE(port187, port_range1, 55), + UNIPHIER_PINCTRL_GROUP_SINGLE(port190, port_range1, 56), + UNIPHIER_PINCTRL_GROUP_SINGLE(port191, port_range1, 57), + UNIPHIER_PINCTRL_GROUP_SINGLE(port192, port_range1, 58), + UNIPHIER_PINCTRL_GROUP_SINGLE(port193, port_range1, 59), + UNIPHIER_PINCTRL_GROUP_SINGLE(port194, port_range1, 60), + UNIPHIER_PINCTRL_GROUP_SINGLE(port195, port_range1, 61), + UNIPHIER_PINCTRL_GROUP_SINGLE(port196, port_range1, 62), + UNIPHIER_PINCTRL_GROUP_SINGLE(port197, port_range1, 63), + UNIPHIER_PINCTRL_GROUP_SINGLE(port200, port_range1, 64), + UNIPHIER_PINCTRL_GROUP_SINGLE(port201, port_range1, 65), + UNIPHIER_PINCTRL_GROUP_SINGLE(port202, port_range1, 66), + UNIPHIER_PINCTRL_GROUP_SINGLE(port203, port_range1, 67), + UNIPHIER_PINCTRL_GROUP_SINGLE(port204, port_range1, 68), + UNIPHIER_PINCTRL_GROUP_SINGLE(port205, port_range1, 69), + UNIPHIER_PINCTRL_GROUP_SINGLE(port206, port_range1, 70), + UNIPHIER_PINCTRL_GROUP_SINGLE(port207, port_range1, 71), + UNIPHIER_PINCTRL_GROUP_SINGLE(port210, port_range1, 72), + UNIPHIER_PINCTRL_GROUP_SINGLE(port211, port_range1, 73), + UNIPHIER_PINCTRL_GROUP_SINGLE(port212, port_range1, 74), + UNIPHIER_PINCTRL_GROUP_SINGLE(port213, port_range1, 75), + UNIPHIER_PINCTRL_GROUP_SINGLE(port214, port_range1, 76), + UNIPHIER_PINCTRL_GROUP_SINGLE(port215, port_range1, 77), + UNIPHIER_PINCTRL_GROUP_SINGLE(port216, port_range1, 78), + UNIPHIER_PINCTRL_GROUP_SINGLE(port217, port_range1, 79), + UNIPHIER_PINCTRL_GROUP_SINGLE(port220, port_range1, 80), + UNIPHIER_PINCTRL_GROUP_SINGLE(port221, port_range1, 81), + UNIPHIER_PINCTRL_GROUP_SINGLE(port222, port_range1, 82), + UNIPHIER_PINCTRL_GROUP_SINGLE(port223, port_range1, 83), + UNIPHIER_PINCTRL_GROUP_SINGLE(port224, port_range1, 84), + UNIPHIER_PINCTRL_GROUP_SINGLE(port225, port_range1, 85), + UNIPHIER_PINCTRL_GROUP_SINGLE(port226, port_range1, 86), + UNIPHIER_PINCTRL_GROUP_SINGLE(port227, port_range1, 87), + UNIPHIER_PINCTRL_GROUP_SINGLE(port230, port_range1, 88), + UNIPHIER_PINCTRL_GROUP_SINGLE(port231, port_range1, 89), + UNIPHIER_PINCTRL_GROUP_SINGLE(port232, port_range1, 90), + UNIPHIER_PINCTRL_GROUP_SINGLE(port233, port_range1, 91), + UNIPHIER_PINCTRL_GROUP_SINGLE(port234, port_range1, 92), + UNIPHIER_PINCTRL_GROUP_SINGLE(port235, port_range1, 93), + UNIPHIER_PINCTRL_GROUP_SINGLE(port236, port_range1, 94), + UNIPHIER_PINCTRL_GROUP_SINGLE(port237, port_range1, 95), + UNIPHIER_PINCTRL_GROUP_SINGLE(port240, port_range1, 96), + UNIPHIER_PINCTRL_GROUP_SINGLE(port241, port_range1, 97), + UNIPHIER_PINCTRL_GROUP_SINGLE(port242, port_range1, 98), + UNIPHIER_PINCTRL_GROUP_SINGLE(port243, port_range1, 99), + UNIPHIER_PINCTRL_GROUP_SINGLE(port244, port_range1, 100), + UNIPHIER_PINCTRL_GROUP_SINGLE(port245, port_range1, 101), + UNIPHIER_PINCTRL_GROUP_SINGLE(port246, port_range1, 102), + UNIPHIER_PINCTRL_GROUP_SINGLE(port247, port_range1, 103), + UNIPHIER_PINCTRL_GROUP_SINGLE(port250, port_range1, 104), + UNIPHIER_PINCTRL_GROUP_SINGLE(port251, port_range1, 105), + UNIPHIER_PINCTRL_GROUP_SINGLE(port252, port_range1, 106), + UNIPHIER_PINCTRL_GROUP_SINGLE(port253, port_range1, 107), + UNIPHIER_PINCTRL_GROUP_SINGLE(port254, port_range1, 108), + UNIPHIER_PINCTRL_GROUP_SINGLE(port255, port_range1, 109), + UNIPHIER_PINCTRL_GROUP_SINGLE(port256, port_range1, 110), + UNIPHIER_PINCTRL_GROUP_SINGLE(port257, port_range1, 111), + UNIPHIER_PINCTRL_GROUP_SINGLE(port260, port_range1, 112), + UNIPHIER_PINCTRL_GROUP_SINGLE(port261, port_range1, 113), + UNIPHIER_PINCTRL_GROUP_SINGLE(port262, port_range1, 114), + UNIPHIER_PINCTRL_GROUP_SINGLE(port263, port_range1, 115), + UNIPHIER_PINCTRL_GROUP_SINGLE(port264, port_range1, 116), + UNIPHIER_PINCTRL_GROUP_SINGLE(port265, port_range1, 117), + UNIPHIER_PINCTRL_GROUP_SINGLE(port266, port_range1, 118), + UNIPHIER_PINCTRL_GROUP_SINGLE(port267, port_range1, 119), + UNIPHIER_PINCTRL_GROUP_SINGLE(port270, port_range1, 120), + UNIPHIER_PINCTRL_GROUP_SINGLE(port271, port_range1, 121), + UNIPHIER_PINCTRL_GROUP_SINGLE(port272, port_range1, 122), + UNIPHIER_PINCTRL_GROUP_SINGLE(port273, port_range1, 123), + UNIPHIER_PINCTRL_GROUP_SINGLE(port274, port_range1, 124), + UNIPHIER_PINCTRL_GROUP_SINGLE(port275, port_range1, 125), + UNIPHIER_PINCTRL_GROUP_SINGLE(port276, port_range1, 126), + UNIPHIER_PINCTRL_GROUP_SINGLE(port277, port_range1, 127), + UNIPHIER_PINCTRL_GROUP_SINGLE(port280, port_range1, 128), + UNIPHIER_PINCTRL_GROUP_SINGLE(port281, port_range1, 129), + UNIPHIER_PINCTRL_GROUP_SINGLE(port282, port_range1, 130), + UNIPHIER_PINCTRL_GROUP_SINGLE(port283, port_range1, 131), + UNIPHIER_PINCTRL_GROUP_SINGLE(port284, port_range1, 132), + UNIPHIER_PINCTRL_GROUP_SINGLE(port285, port_range1, 133), + UNIPHIER_PINCTRL_GROUP_SINGLE(port286, port_range1, 134), + UNIPHIER_PINCTRL_GROUP_SINGLE(port287, port_range1, 135), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq0, xirq, 0), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq1, xirq, 1), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq2, xirq, 2), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq3, xirq, 3), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq4, xirq, 4), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq5, xirq, 5), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq6, xirq, 6), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq7, xirq, 7), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq8, xirq, 8), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq9, xirq, 9), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq10, xirq, 10), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq11, xirq, 11), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq12, xirq, 12), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq13, xirq, 13), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq14, xirq, 14), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq15, xirq, 15), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq16, xirq, 16), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq17, xirq, 17), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq18, xirq, 18), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq19, xirq, 19), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq20, xirq, 20), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq21, xirq, 21), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq22, xirq, 22), + UNIPHIER_PINCTRL_GROUP_SINGLE(xirq23, xirq, 23), +}; + +static const char * const emmc_groups[] = {"emmc", "emmc_dat8"}; +static const char * const i2c0_groups[] = {"i2c0"}; +static const char * const i2c1_groups[] = {"i2c1"}; +static const char * const i2c2_groups[] = {"i2c2"}; +static const char * const i2c3_groups[] = {"i2c3"}; +static const char * const i2c5_groups[] = {"i2c5"}; +static const char * const i2c6_groups[] = {"i2c6"}; +static const char * const nand_groups[] = {"nand", "nand_cs1"}; +static const char * const uart0_groups[] = {"uart0", "uart0b"}; +static const char * const uart1_groups[] = {"uart1"}; +static const char * const uart2_groups[] = {"uart2"}; +static const char * const uart3_groups[] = {"uart3", "uart3b"}; +static const char * const usb0_groups[] = {"usb0"}; +static const char * const usb1_groups[] = {"usb1"}; +static const char * const usb2_groups[] = {"usb2"}; +static const char * const usb3_groups[] = {"usb3"}; +static const char * const port_groups[] = { + "port00", "port01", "port02", "port03", + "port04", "port05", "port06", "port07", + "port10", "port11", "port12", "port13", + "port14", "port15", "port16", "port17", + "port20", "port21", "port22", "port23", + "port24", "port25", "port26", "port27", + "port30", "port31", "port32", "port33", + "port34", "port35", "port36", "port37", + "port40", "port41", "port42", "port43", + "port44", "port45", "port46", "port47", + "port50", "port51", "port52", "port53", + "port54", "port55", "port56", "port57", + "port60", "port61", "port62", "port63", + "port64", "port65", "port66", "port67", + "port70", "port71", "port72", "port73", + "port74", "port75", "port76", "port77", + "port80", "port81", "port82", "port83", + "port84", "port85", "port86", "port87", + "port90", "port91", "port92", "port93", + "port94", "port95", "port96", "port97", + "port100", "port101", "port102", "port103", + "port104", "port105", "port106", "port107", + /* port110-117 missing */ + "port120", "port121", "port122", "port123", + "port124", "port125", "port126", "port127", + "port130", "port131", "port132", "port133", + "port134", "port135", "port136", "port137", + "port140", "port141", "port142", "port143", + "port144", "port145", "port146", "port147", + "port150", "port151", "port152", "port153", + "port154", "port155", "port156", "port157", + "port160", "port161", "port162", "port163", + "port164", "port165", "port166", "port167", + "port170", "port171", "port172", "port173", + "port174", "port175", "port176", "port177", + "port180", "port181", "port182", "port183", + "port184", "port185", "port186", "port187", + "port190", "port191", "port192", "port193", + "port194", "port195", "port196", "port197", + "port200", "port201", "port202", "port203", + "port204", "port205", "port206", "port207", + "port210", "port211", "port212", "port213", + "port214", "port215", "port216", "port217", + "port220", "port221", "port222", "port223", + "port224", "port225", "port226", "port227", + "port230", "port231", "port232", "port233", + "port234", "port235", "port236", "port237", + "port240", "port241", "port242", "port243", + "port244", "port245", "port246", "port247", + "port250", "port251", "port252", "port253", + "port254", "port255", "port256", "port257", + "port260", "port261", "port262", "port263", + "port264", "port265", "port266", "port267", + "port270", "port271", "port272", "port273", + "port274", "port275", "port276", "port277", + "port280", "port281", "port282", "port283", + "port284", "port285", "port286", "port287", +}; +static const char * const xirq_groups[] = { + "xirq0", "xirq1", "xirq2", "xirq3", + "xirq4", "xirq5", "xirq6", "xirq7", + "xirq8", "xirq9", "xirq10", "xirq11", + "xirq12", "xirq13", "xirq14", "xirq15", + "xirq16", "xirq17", "xirq18", "xirq19", + "xirq20", "xirq21", "xirq22", "xirq23", +}; + +static const struct uniphier_pinmux_function proxstream2_functions[] = { + UNIPHIER_PINMUX_FUNCTION(emmc), + UNIPHIER_PINMUX_FUNCTION(i2c0), + UNIPHIER_PINMUX_FUNCTION(i2c1), + UNIPHIER_PINMUX_FUNCTION(i2c2), + UNIPHIER_PINMUX_FUNCTION(i2c3), + UNIPHIER_PINMUX_FUNCTION(i2c5), + UNIPHIER_PINMUX_FUNCTION(i2c6), + UNIPHIER_PINMUX_FUNCTION(nand), + UNIPHIER_PINMUX_FUNCTION(uart0), + UNIPHIER_PINMUX_FUNCTION(uart1), + UNIPHIER_PINMUX_FUNCTION(uart2), + UNIPHIER_PINMUX_FUNCTION(uart3), + UNIPHIER_PINMUX_FUNCTION(usb0), + UNIPHIER_PINMUX_FUNCTION(usb1), + UNIPHIER_PINMUX_FUNCTION(usb2), + UNIPHIER_PINMUX_FUNCTION(usb3), + UNIPHIER_PINMUX_FUNCTION(port), + UNIPHIER_PINMUX_FUNCTION(xirq), +}; + +static struct uniphier_pinctrl_socdata proxstream2_pindata = { + .groups = proxstream2_groups, + .groups_count = ARRAY_SIZE(proxstream2_groups), + .functions = proxstream2_functions, + .functions_count = ARRAY_SIZE(proxstream2_functions), + .mux_bits = 8, + .reg_stride = 4, + .load_pinctrl = false, +}; + +static struct pinctrl_desc proxstream2_pinctrl_desc = { + .name = DRIVER_NAME, + .pins = proxstream2_pins, + .npins = ARRAY_SIZE(proxstream2_pins), + .owner = THIS_MODULE, +}; + +static int proxstream2_pinctrl_probe(struct platform_device *pdev) +{ + return uniphier_pinctrl_probe(pdev, &proxstream2_pinctrl_desc, + &proxstream2_pindata); +} + +static const struct of_device_id proxstream2_pinctrl_match[] = { + { .compatible = "socionext,proxstream2-pinctrl" }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, proxstream2_pinctrl_match); + +static struct platform_driver proxstream2_pinctrl_driver = { + .probe = proxstream2_pinctrl_probe, + .remove = uniphier_pinctrl_remove, + .driver = { + .name = DRIVER_NAME, + .of_match_table = proxstream2_pinctrl_match, + }, +}; +module_platform_driver(proxstream2_pinctrl_driver); + +MODULE_AUTHOR("Masahiro Yamada "); +MODULE_DESCRIPTION("UniPhier ProXstream2 pinctrl driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier-core.c b/drivers/pinctrl/uniphier/pinctrl-uniphier-core.c new file mode 100644 index 00000000000000..918f3b643f1b00 --- /dev/null +++ b/drivers/pinctrl/uniphier/pinctrl-uniphier-core.c @@ -0,0 +1,684 @@ +/* + * Copyright (C) 2015 Masahiro Yamada + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../core.h" +#include "../pinctrl-utils.h" +#include "pinctrl-uniphier.h" + +struct uniphier_pinctrl_priv { + struct pinctrl_dev *pctldev; + struct regmap *regmap; + struct uniphier_pinctrl_socdata *socdata; +}; + +static int uniphier_pctl_get_groups_count(struct pinctrl_dev *pctldev) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + + return priv->socdata->groups_count; +} + +static const char *uniphier_pctl_get_group_name(struct pinctrl_dev *pctldev, + unsigned selector) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + + return priv->socdata->groups[selector].name; +} + +static int uniphier_pctl_get_group_pins(struct pinctrl_dev *pctldev, + unsigned selector, + const unsigned **pins, + unsigned *num_pins) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + + *pins = priv->socdata->groups[selector].pins; + *num_pins = priv->socdata->groups[selector].num_pins; + + return 0; +} + +#ifdef CONFIG_DEBUG_FS +static void uniphier_pctl_pin_dbg_show(struct pinctrl_dev *pctldev, + struct seq_file *s, unsigned offset) +{ + const struct pinctrl_pin_desc *pin = &pctldev->desc->pins[offset]; + const char *pull_dir, *drv_str; + + switch (uniphier_pin_get_pull_dir(pin->drv_data)) { + case UNIPHIER_PIN_PULL_UP: + pull_dir = "UP"; + break; + case UNIPHIER_PIN_PULL_DOWN: + pull_dir = "DOWN"; + break; + case UNIPHIER_PIN_PULL_NONE: + pull_dir = "NONE"; + break; + default: + BUG(); + } + + switch (uniphier_pin_get_drv_str(pin->drv_data)) { + case UNIPHIER_PIN_DRV_4_8: + drv_str = "4/8(mA)"; + break; + case UNIPHIER_PIN_DRV_8_12_16_20: + drv_str = "8/12/16/20(mA)"; + break; + case UNIPHIER_PIN_DRV_FIXED_4: + drv_str = "4(mA)"; + break; + case UNIPHIER_PIN_DRV_FIXED_5: + drv_str = "5(mA)"; + break; + case UNIPHIER_PIN_DRV_FIXED_8: + drv_str = "8(mA)"; + break; + case UNIPHIER_PIN_DRV_NONE: + drv_str = "NONE"; + break; + default: + BUG(); + } + + seq_printf(s, " PULL_DIR=%s DRV_STR=%s", pull_dir, drv_str); +} +#endif + +static const struct pinctrl_ops uniphier_pctlops = { + .get_groups_count = uniphier_pctl_get_groups_count, + .get_group_name = uniphier_pctl_get_group_name, + .get_group_pins = uniphier_pctl_get_group_pins, +#ifdef CONFIG_DEBUG_FS + .pin_dbg_show = uniphier_pctl_pin_dbg_show, +#endif + .dt_node_to_map = pinconf_generic_dt_node_to_map_all, + .dt_free_map = pinctrl_utils_dt_free_map, +}; + +static int uniphier_conf_pin_bias_get(struct pinctrl_dev *pctldev, + const struct pinctrl_pin_desc *pin, + enum pin_config_param param) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + enum uniphier_pin_pull_dir pull_dir = + uniphier_pin_get_pull_dir(pin->drv_data); + unsigned int pupdctrl, reg, shift, val; + unsigned int expected = 1; + int ret; + + switch (param) { + case PIN_CONFIG_BIAS_DISABLE: + if (pull_dir == UNIPHIER_PIN_PULL_NONE) + return 0; + if (pull_dir == UNIPHIER_PIN_PULL_UP_FIXED || + pull_dir == UNIPHIER_PIN_PULL_DOWN_FIXED) + return -EINVAL; + expected = 0; + break; + case PIN_CONFIG_BIAS_PULL_UP: + if (pull_dir == UNIPHIER_PIN_PULL_UP_FIXED) + return 0; + if (pull_dir != UNIPHIER_PIN_PULL_UP) + return -EINVAL; + break; + case PIN_CONFIG_BIAS_PULL_DOWN: + if (pull_dir == UNIPHIER_PIN_PULL_DOWN_FIXED) + return 0; + if (pull_dir != UNIPHIER_PIN_PULL_DOWN) + return -EINVAL; + break; + default: + BUG(); + } + + pupdctrl = uniphier_pin_get_pupdctrl(pin->drv_data); + + reg = UNIPHIER_PINCTRL_PUPDCTRL_BASE + pupdctrl / 32 * 4; + shift = pupdctrl % 32; + + ret = regmap_read(priv->regmap, reg, &val); + if (ret) + return ret; + + val = (val >> shift) & 1; + + return (val == expected) ? 0 : -EINVAL; +} + +static int uniphier_conf_pin_drive_get(struct pinctrl_dev *pctldev, + const struct pinctrl_pin_desc *pin, + u16 *strength) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + enum uniphier_pin_drv_str drv_str = + uniphier_pin_get_drv_str(pin->drv_data); + const unsigned int strength_4_8[] = {4, 8}; + const unsigned int strength_8_12_16_20[] = {8, 12, 16, 20}; + const unsigned int *supported_strength; + unsigned int drvctrl, reg, shift, mask, width, val; + int ret; + + switch (drv_str) { + case UNIPHIER_PIN_DRV_4_8: + supported_strength = strength_4_8; + width = 1; + break; + case UNIPHIER_PIN_DRV_8_12_16_20: + supported_strength = strength_8_12_16_20; + width = 2; + break; + case UNIPHIER_PIN_DRV_FIXED_4: + *strength = 4; + return 0; + case UNIPHIER_PIN_DRV_FIXED_5: + *strength = 5; + return 0; + case UNIPHIER_PIN_DRV_FIXED_8: + *strength = 8; + return 0; + default: + /* drive strength control is not supported for this pin */ + return -EINVAL; + } + + drvctrl = uniphier_pin_get_drvctrl(pin->drv_data); + drvctrl *= width; + + reg = (width == 2) ? UNIPHIER_PINCTRL_DRV2CTRL_BASE : + UNIPHIER_PINCTRL_DRVCTRL_BASE; + + reg += drvctrl / 32 * 4; + shift = drvctrl % 32; + mask = (1U << width) - 1; + + ret = regmap_read(priv->regmap, reg, &val); + if (ret) + return ret; + + *strength = supported_strength[(val >> shift) & mask]; + + return 0; +} + +static int uniphier_conf_pin_input_enable_get(struct pinctrl_dev *pctldev, + const struct pinctrl_pin_desc *pin) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + unsigned int iectrl = uniphier_pin_get_iectrl(pin->drv_data); + unsigned int val; + int ret; + + if (iectrl == UNIPHIER_PIN_IECTRL_NONE) + /* This pin is always input-enabled. */ + return 0; + + ret = regmap_read(priv->regmap, UNIPHIER_PINCTRL_IECTRL, &val); + if (ret) + return ret; + + return val & BIT(iectrl) ? 0 : -EINVAL; +} + +static int uniphier_conf_pin_config_get(struct pinctrl_dev *pctldev, + unsigned pin, + unsigned long *configs) +{ + const struct pinctrl_pin_desc *pin_desc = &pctldev->desc->pins[pin]; + enum pin_config_param param = pinconf_to_config_param(*configs); + bool has_arg = false; + u16 arg; + int ret; + + switch (param) { + case PIN_CONFIG_BIAS_DISABLE: + case PIN_CONFIG_BIAS_PULL_UP: + case PIN_CONFIG_BIAS_PULL_DOWN: + ret = uniphier_conf_pin_bias_get(pctldev, pin_desc, param); + break; + case PIN_CONFIG_DRIVE_STRENGTH: + ret = uniphier_conf_pin_drive_get(pctldev, pin_desc, &arg); + has_arg = true; + break; + case PIN_CONFIG_INPUT_ENABLE: + ret = uniphier_conf_pin_input_enable_get(pctldev, pin_desc); + break; + default: + /* unsupported parameter */ + ret = -EINVAL; + break; + } + + if (ret == 0 && has_arg) + *configs = pinconf_to_config_packed(param, arg); + + return ret; +} + +static int uniphier_conf_pin_bias_set(struct pinctrl_dev *pctldev, + const struct pinctrl_pin_desc *pin, + enum pin_config_param param, + u16 arg) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + enum uniphier_pin_pull_dir pull_dir = + uniphier_pin_get_pull_dir(pin->drv_data); + unsigned int pupdctrl, reg, shift; + unsigned int val = 1; + + switch (param) { + case PIN_CONFIG_BIAS_DISABLE: + if (pull_dir == UNIPHIER_PIN_PULL_NONE) + return 0; + if (pull_dir == UNIPHIER_PIN_PULL_UP_FIXED || + pull_dir == UNIPHIER_PIN_PULL_DOWN_FIXED) { + dev_err(pctldev->dev, + "can not disable pull register for pin %u (%s)\n", + pin->number, pin->name); + return -EINVAL; + } + val = 0; + break; + case PIN_CONFIG_BIAS_PULL_UP: + if (pull_dir == UNIPHIER_PIN_PULL_UP_FIXED && arg != 0) + return 0; + if (pull_dir != UNIPHIER_PIN_PULL_UP) { + dev_err(pctldev->dev, + "pull-up is unsupported for pin %u (%s)\n", + pin->number, pin->name); + return -EINVAL; + } + if (arg == 0) { + dev_err(pctldev->dev, "pull-up can not be total\n"); + return -EINVAL; + } + break; + case PIN_CONFIG_BIAS_PULL_DOWN: + if (pull_dir == UNIPHIER_PIN_PULL_DOWN_FIXED && arg != 0) + return 0; + if (pull_dir != UNIPHIER_PIN_PULL_DOWN) { + dev_err(pctldev->dev, + "pull-down is unsupported for pin %u (%s)\n", + pin->number, pin->name); + return -EINVAL; + } + if (arg == 0) { + dev_err(pctldev->dev, "pull-down can not be total\n"); + return -EINVAL; + } + break; + case PIN_CONFIG_BIAS_PULL_PIN_DEFAULT: + if (pull_dir == UNIPHIER_PIN_PULL_NONE) { + dev_err(pctldev->dev, + "pull-up/down is unsupported for pin %u (%s)\n", + pin->number, pin->name); + return -EINVAL; + } + + if (arg == 0) + return 0; /* configuration ingored */ + break; + default: + BUG(); + } + + pupdctrl = uniphier_pin_get_pupdctrl(pin->drv_data); + + reg = UNIPHIER_PINCTRL_PUPDCTRL_BASE + pupdctrl / 32 * 4; + shift = pupdctrl % 32; + + return regmap_update_bits(priv->regmap, reg, 1 << shift, val << shift); +} + +static int uniphier_conf_pin_drive_set(struct pinctrl_dev *pctldev, + const struct pinctrl_pin_desc *pin, + u16 strength) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + enum uniphier_pin_drv_str drv_str = + uniphier_pin_get_drv_str(pin->drv_data); + const unsigned int strength_4_8[] = {4, 8, -1}; + const unsigned int strength_8_12_16_20[] = {8, 12, 16, 20, -1}; + const unsigned int *supported_strength; + unsigned int drvctrl, reg, shift, mask, width, val; + + switch (drv_str) { + case UNIPHIER_PIN_DRV_4_8: + supported_strength = strength_4_8; + width = 1; + break; + case UNIPHIER_PIN_DRV_8_12_16_20: + supported_strength = strength_8_12_16_20; + width = 2; + break; + default: + dev_err(pctldev->dev, + "cannot change drive strength for pin %u (%s)\n", + pin->number, pin->name); + return -EINVAL; + } + + for (val = 0; supported_strength[val] > 0; val++) { + if (supported_strength[val] > strength) + break; + } + + if (val == 0) { + dev_err(pctldev->dev, + "unsupported drive strength %u mA for pin %u (%s)\n", + strength, pin->number, pin->name); + return -EINVAL; + } + + val--; + + drvctrl = uniphier_pin_get_drvctrl(pin->drv_data); + drvctrl *= width; + + reg = (width == 2) ? UNIPHIER_PINCTRL_DRV2CTRL_BASE : + UNIPHIER_PINCTRL_DRVCTRL_BASE; + + reg += drvctrl / 32 * 4; + shift = drvctrl % 32; + mask = (1U << width) - 1; + + return regmap_update_bits(priv->regmap, reg, + mask << shift, val << shift); +} + +static int uniphier_conf_pin_input_enable(struct pinctrl_dev *pctldev, + const struct pinctrl_pin_desc *pin, + u16 enable) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + unsigned int iectrl = uniphier_pin_get_iectrl(pin->drv_data); + + if (enable == 0) { + /* + * Multiple pins share one input enable, so per-pin disabling + * is impossible. + */ + dev_err(pctldev->dev, "unable to disable input\n"); + return -EINVAL; + } + + if (iectrl == UNIPHIER_PIN_IECTRL_NONE) + /* This pin is always input-enabled. nothing to do. */ + return 0; + + return regmap_update_bits(priv->regmap, UNIPHIER_PINCTRL_IECTRL, + BIT(iectrl), BIT(iectrl)); +} + +static int uniphier_conf_pin_config_set(struct pinctrl_dev *pctldev, + unsigned pin, + unsigned long *configs, + unsigned num_configs) +{ + const struct pinctrl_pin_desc *pin_desc = &pctldev->desc->pins[pin]; + int i, ret; + + for (i = 0; i < num_configs; i++) { + enum pin_config_param param = + pinconf_to_config_param(configs[i]); + u16 arg = pinconf_to_config_argument(configs[i]); + + switch (param) { + case PIN_CONFIG_BIAS_DISABLE: + case PIN_CONFIG_BIAS_PULL_UP: + case PIN_CONFIG_BIAS_PULL_DOWN: + case PIN_CONFIG_BIAS_PULL_PIN_DEFAULT: + ret = uniphier_conf_pin_bias_set(pctldev, pin_desc, + param, arg); + break; + case PIN_CONFIG_DRIVE_STRENGTH: + ret = uniphier_conf_pin_drive_set(pctldev, pin_desc, + arg); + break; + case PIN_CONFIG_INPUT_ENABLE: + ret = uniphier_conf_pin_input_enable(pctldev, + pin_desc, arg); + break; + default: + dev_err(pctldev->dev, + "unsupported configuration parameter %u\n", + param); + return -EINVAL; + } + + if (ret) + return ret; + } + + return 0; +} + +static int uniphier_conf_pin_config_group_set(struct pinctrl_dev *pctldev, + unsigned selector, + unsigned long *configs, + unsigned num_configs) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + const unsigned *pins = priv->socdata->groups[selector].pins; + unsigned num_pins = priv->socdata->groups[selector].num_pins; + int i, ret; + + for (i = 0; i < num_pins; i++) { + ret = uniphier_conf_pin_config_set(pctldev, pins[i], + configs, num_configs); + if (ret) + return ret; + } + + return 0; +} + +static const struct pinconf_ops uniphier_confops = { + .is_generic = true, + .pin_config_get = uniphier_conf_pin_config_get, + .pin_config_set = uniphier_conf_pin_config_set, + .pin_config_group_set = uniphier_conf_pin_config_group_set, +}; + +static int uniphier_pmx_get_functions_count(struct pinctrl_dev *pctldev) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + + return priv->socdata->functions_count; +} + +static const char *uniphier_pmx_get_function_name(struct pinctrl_dev *pctldev, + unsigned selector) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + + return priv->socdata->functions[selector].name; +} + +static int uniphier_pmx_get_function_groups(struct pinctrl_dev *pctldev, + unsigned selector, + const char * const **groups, + unsigned *num_groups) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + + *groups = priv->socdata->functions[selector].groups; + *num_groups = priv->socdata->functions[selector].num_groups; + + return 0; +} + +static int uniphier_pmx_set_one_mux(struct pinctrl_dev *pctldev, unsigned pin, + unsigned muxval) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + unsigned mux_bits = priv->socdata->mux_bits; + unsigned reg_stride = priv->socdata->reg_stride; + unsigned reg, reg_end, shift, mask; + int ret; + + reg = UNIPHIER_PINCTRL_PINMUX_BASE + pin * mux_bits / 32 * reg_stride; + reg_end = reg + reg_stride; + shift = pin * mux_bits % 32; + mask = (1U << mux_bits) - 1; + + /* + * If reg_stride is greater than 4, the MSB of each pinsel shall be + * stored in the offset+4. + */ + for (; reg < reg_end; reg += 4) { + ret = regmap_update_bits(priv->regmap, reg, + mask << shift, muxval << shift); + if (ret) + return ret; + muxval >>= mux_bits; + } + + if (priv->socdata->load_pinctrl) { + ret = regmap_write(priv->regmap, + UNIPHIER_PINCTRL_LOAD_PINMUX, 1); + if (ret) + return ret; + } + + /* some pins need input-enabling */ + return uniphier_conf_pin_input_enable(pctldev, + &pctldev->desc->pins[pin], 1); +} + +static int uniphier_pmx_set_mux(struct pinctrl_dev *pctldev, + unsigned func_selector, + unsigned group_selector) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + const struct uniphier_pinctrl_group *grp = + &priv->socdata->groups[group_selector]; + int i; + int ret; + + for (i = 0; i < grp->num_pins; i++) { + ret = uniphier_pmx_set_one_mux(pctldev, grp->pins[i], + grp->muxvals[i]); + if (ret) + return ret; + } + + return 0; +} + +static int uniphier_pmx_gpio_request_enable(struct pinctrl_dev *pctldev, + struct pinctrl_gpio_range *range, + unsigned offset) +{ + struct uniphier_pinctrl_priv *priv = pinctrl_dev_get_drvdata(pctldev); + const struct uniphier_pinctrl_group *groups = priv->socdata->groups; + int groups_count = priv->socdata->groups_count; + enum uniphier_pinmux_gpio_range_type range_type; + int i, j; + + if (strstr(range->name, "irq")) + range_type = UNIPHIER_PINMUX_GPIO_RANGE_IRQ; + else + range_type = UNIPHIER_PINMUX_GPIO_RANGE_PORT; + + for (i = 0; i < groups_count; i++) { + if (groups[i].range_type != range_type) + continue; + + for (j = 0; j < groups[i].num_pins; j++) + if (groups[i].pins[j] == offset) + goto found; + } + + dev_err(pctldev->dev, "pin %u does not support GPIO\n", offset); + return -EINVAL; + +found: + return uniphier_pmx_set_one_mux(pctldev, offset, groups[i].muxvals[j]); +} + +static const struct pinmux_ops uniphier_pmxops = { + .get_functions_count = uniphier_pmx_get_functions_count, + .get_function_name = uniphier_pmx_get_function_name, + .get_function_groups = uniphier_pmx_get_function_groups, + .set_mux = uniphier_pmx_set_mux, + .gpio_request_enable = uniphier_pmx_gpio_request_enable, + .strict = true, +}; + +int uniphier_pinctrl_probe(struct platform_device *pdev, + struct pinctrl_desc *desc, + struct uniphier_pinctrl_socdata *socdata) +{ + struct device *dev = &pdev->dev; + struct uniphier_pinctrl_priv *priv; + + if (!socdata || + !socdata->groups || + !socdata->groups_count || + !socdata->functions || + !socdata->functions_count || + !socdata->mux_bits || + !socdata->reg_stride) { + dev_err(dev, "pinctrl socdata lacks necessary members\n"); + return -EINVAL; + } + + priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL); + if (!priv) + return -ENOMEM; + + priv->regmap = syscon_node_to_regmap(dev->of_node); + if (IS_ERR(priv->regmap)) { + dev_err(dev, "failed to get regmap\n"); + return PTR_ERR(priv->regmap); + } + + priv->socdata = socdata; + desc->pctlops = &uniphier_pctlops; + desc->pmxops = &uniphier_pmxops; + desc->confops = &uniphier_confops; + + priv->pctldev = pinctrl_register(desc, dev, priv); + if (IS_ERR(priv->pctldev)) { + dev_err(dev, "failed to register UniPhier pinctrl driver\n"); + return PTR_ERR(priv->pctldev); + } + + platform_set_drvdata(pdev, priv); + + return 0; +} +EXPORT_SYMBOL_GPL(uniphier_pinctrl_probe); + +int uniphier_pinctrl_remove(struct platform_device *pdev) +{ + struct uniphier_pinctrl_priv *priv = platform_get_drvdata(pdev); + + pinctrl_unregister(priv->pctldev); + + return 0; +} +EXPORT_SYMBOL_GPL(uniphier_pinctrl_remove); diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier.h b/drivers/pinctrl/uniphier/pinctrl-uniphier.h new file mode 100644 index 00000000000000..e1e98b868be501 --- /dev/null +++ b/drivers/pinctrl/uniphier/pinctrl-uniphier.h @@ -0,0 +1,217 @@ +/* + * Copyright (C) 2015 Masahiro Yamada + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#ifndef __PINCTRL_UNIPHIER_H__ +#define __PINCTRL_UNIPHIER_H__ + +#include +#include +#include + +#define UNIPHIER_PINCTRL_PINMUX_BASE 0x0 +#define UNIPHIER_PINCTRL_LOAD_PINMUX 0x700 +#define UNIPHIER_PINCTRL_DRVCTRL_BASE 0x800 +#define UNIPHIER_PINCTRL_DRV2CTRL_BASE 0x900 +#define UNIPHIER_PINCTRL_PUPDCTRL_BASE 0xa00 +#define UNIPHIER_PINCTRL_IECTRL 0xd00 + +/* input enable control register bit */ +#define UNIPHIER_PIN_IECTRL_SHIFT 0 +#define UNIPHIER_PIN_IECTRL_BITS 8 +#define UNIPHIER_PIN_IECTRL_MASK ((1UL << (UNIPHIER_PIN_IECTRL_BITS)) \ + - 1) + +/* drive strength control register number */ +#define UNIPHIER_PIN_DRVCTRL_SHIFT ((UNIPHIER_PIN_IECTRL_SHIFT) + \ + (UNIPHIER_PIN_IECTRL_BITS)) +#define UNIPHIER_PIN_DRVCTRL_BITS 9 +#define UNIPHIER_PIN_DRVCTRL_MASK ((1UL << (UNIPHIER_PIN_DRVCTRL_BITS)) \ + - 1) + +/* supported drive strength (mA) */ +#define UNIPHIER_PIN_DRV_STR_SHIFT ((UNIPHIER_PIN_DRVCTRL_SHIFT) + \ + (UNIPHIER_PIN_DRVCTRL_BITS)) +#define UNIPHIER_PIN_DRV_STR_BITS 3 +#define UNIPHIER_PIN_DRV_STR_MASK ((1UL << (UNIPHIER_PIN_DRV_STR_BITS)) \ + - 1) + +/* pull-up / pull-down register number */ +#define UNIPHIER_PIN_PUPDCTRL_SHIFT ((UNIPHIER_PIN_DRV_STR_SHIFT) + \ + (UNIPHIER_PIN_DRV_STR_BITS)) +#define UNIPHIER_PIN_PUPDCTRL_BITS 9 +#define UNIPHIER_PIN_PUPDCTRL_MASK ((1UL << (UNIPHIER_PIN_PUPDCTRL_BITS))\ + - 1) + +/* direction of pull register */ +#define UNIPHIER_PIN_PULL_DIR_SHIFT ((UNIPHIER_PIN_PUPDCTRL_SHIFT) + \ + (UNIPHIER_PIN_PUPDCTRL_BITS)) +#define UNIPHIER_PIN_PULL_DIR_BITS 3 +#define UNIPHIER_PIN_PULL_DIR_MASK ((1UL << (UNIPHIER_PIN_PULL_DIR_BITS))\ + - 1) + +#if UNIPHIER_PIN_PULL_DIR_SHIFT + UNIPHIER_PIN_PULL_DIR_BITS > BITS_PER_LONG +#error "unable to pack pin attributes." +#endif + +#define UNIPHIER_PIN_IECTRL_NONE (UNIPHIER_PIN_IECTRL_MASK) + +/* selectable drive strength */ +enum uniphier_pin_drv_str { + UNIPHIER_PIN_DRV_4_8, /* 2 level control: 4/8 mA */ + UNIPHIER_PIN_DRV_8_12_16_20, /* 4 level control: 8/12/16/20 mA */ + UNIPHIER_PIN_DRV_FIXED_4, /* fixed to 4mA */ + UNIPHIER_PIN_DRV_FIXED_5, /* fixed to 5mA */ + UNIPHIER_PIN_DRV_FIXED_8, /* fixed to 8mA */ + UNIPHIER_PIN_DRV_NONE, /* no support (input only pin) */ +}; + +/* direction of pull register (no pin supports bi-directional pull biasing) */ +enum uniphier_pin_pull_dir { + UNIPHIER_PIN_PULL_UP, /* pull-up or disabled */ + UNIPHIER_PIN_PULL_DOWN, /* pull-down or disabled */ + UNIPHIER_PIN_PULL_UP_FIXED, /* always pull-up */ + UNIPHIER_PIN_PULL_DOWN_FIXED, /* always pull-down */ + UNIPHIER_PIN_PULL_NONE, /* no pull register */ +}; + +#define UNIPHIER_PIN_IECTRL(x) \ + (((x) & (UNIPHIER_PIN_IECTRL_MASK)) << (UNIPHIER_PIN_IECTRL_SHIFT)) +#define UNIPHIER_PIN_DRVCTRL(x) \ + (((x) & (UNIPHIER_PIN_DRVCTRL_MASK)) << (UNIPHIER_PIN_DRVCTRL_SHIFT)) +#define UNIPHIER_PIN_DRV_STR(x) \ + (((x) & (UNIPHIER_PIN_DRV_STR_MASK)) << (UNIPHIER_PIN_DRV_STR_SHIFT)) +#define UNIPHIER_PIN_PUPDCTRL(x) \ + (((x) & (UNIPHIER_PIN_PUPDCTRL_MASK)) << (UNIPHIER_PIN_PUPDCTRL_SHIFT)) +#define UNIPHIER_PIN_PULL_DIR(x) \ + (((x) & (UNIPHIER_PIN_PULL_DIR_MASK)) << (UNIPHIER_PIN_PULL_DIR_SHIFT)) + +#define UNIPHIER_PIN_ATTR_PACKED(iectrl, drvctrl, drv_str, pupdctrl, pull_dir)\ + (UNIPHIER_PIN_IECTRL(iectrl) | \ + UNIPHIER_PIN_DRVCTRL(drvctrl) | \ + UNIPHIER_PIN_DRV_STR(drv_str) | \ + UNIPHIER_PIN_PUPDCTRL(pupdctrl) | \ + UNIPHIER_PIN_PULL_DIR(pull_dir)) + +static inline unsigned int uniphier_pin_get_iectrl(void *drv_data) +{ + return ((unsigned long)drv_data >> UNIPHIER_PIN_IECTRL_SHIFT) & + UNIPHIER_PIN_IECTRL_MASK; +} + +static inline unsigned int uniphier_pin_get_drvctrl(void *drv_data) +{ + return ((unsigned long)drv_data >> UNIPHIER_PIN_DRVCTRL_SHIFT) & + UNIPHIER_PIN_DRVCTRL_MASK; +} + +static inline unsigned int uniphier_pin_get_drv_str(void *drv_data) +{ + return ((unsigned long)drv_data >> UNIPHIER_PIN_DRV_STR_SHIFT) & + UNIPHIER_PIN_DRV_STR_MASK; +} + +static inline unsigned int uniphier_pin_get_pupdctrl(void *drv_data) +{ + return ((unsigned long)drv_data >> UNIPHIER_PIN_PUPDCTRL_SHIFT) & + UNIPHIER_PIN_PUPDCTRL_MASK; +} + +static inline unsigned int uniphier_pin_get_pull_dir(void *drv_data) +{ + return ((unsigned long)drv_data >> UNIPHIER_PIN_PULL_DIR_SHIFT) & + UNIPHIER_PIN_PULL_DIR_MASK; +} + +enum uniphier_pinmux_gpio_range_type { + UNIPHIER_PINMUX_GPIO_RANGE_PORT, + UNIPHIER_PINMUX_GPIO_RANGE_IRQ, + UNIPHIER_PINMUX_GPIO_RANGE_NONE, +}; + +struct uniphier_pinctrl_group { + const char *name; + const unsigned *pins; + unsigned num_pins; + const unsigned *muxvals; + enum uniphier_pinmux_gpio_range_type range_type; +}; + +struct uniphier_pinmux_function { + const char *name; + const char * const *groups; + unsigned num_groups; +}; + +struct uniphier_pinctrl_socdata { + const struct uniphier_pinctrl_group *groups; + int groups_count; + const struct uniphier_pinmux_function *functions; + int functions_count; + unsigned mux_bits; + unsigned reg_stride; + bool load_pinctrl; +}; + +#define UNIPHIER_PINCTRL_PIN(a, b, c, d, e, f, g) \ +{ \ + .number = a, \ + .name = b, \ + .drv_data = (void *)UNIPHIER_PIN_ATTR_PACKED(c, d, e, f, g), \ +} + +#define __UNIPHIER_PINCTRL_GROUP(grp, type) \ + { \ + .name = #grp, \ + .pins = grp##_pins, \ + .num_pins = ARRAY_SIZE(grp##_pins), \ + .muxvals = grp##_muxvals + \ + BUILD_BUG_ON_ZERO(ARRAY_SIZE(grp##_pins) != \ + ARRAY_SIZE(grp##_muxvals)), \ + .range_type = type, \ + } + +#define UNIPHIER_PINCTRL_GROUP(grp) \ + __UNIPHIER_PINCTRL_GROUP(grp, UNIPHIER_PINMUX_GPIO_RANGE_NONE) + +#define UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_PORT(grp) \ + __UNIPHIER_PINCTRL_GROUP(grp, UNIPHIER_PINMUX_GPIO_RANGE_PORT) + +#define UNIPHIER_PINCTRL_GROUP_GPIO_RANGE_IRQ(grp) \ + __UNIPHIER_PINCTRL_GROUP(grp, UNIPHIER_PINMUX_GPIO_RANGE_IRQ) + +#define UNIPHIER_PINCTRL_GROUP_SINGLE(grp, array, ofst) \ + { \ + .name = #grp, \ + .pins = array##_pins + ofst, \ + .num_pins = 1, \ + .muxvals = array##_muxvals + ofst, \ + } + +#define UNIPHIER_PINMUX_FUNCTION(func) \ + { \ + .name = #func, \ + .groups = func##_groups, \ + .num_groups = ARRAY_SIZE(func##_groups), \ + } + +struct platform_device; +struct pinctrl_desc; + +int uniphier_pinctrl_probe(struct platform_device *pdev, + struct pinctrl_desc *desc, + struct uniphier_pinctrl_socdata *socdata); + +int uniphier_pinctrl_remove(struct platform_device *pdev); + +#endif /* __PINCTRL_UNIPHIER_H__ */ diff --git a/drivers/s390/scsi/zfcp_aux.c b/drivers/s390/scsi/zfcp_aux.c index 01a73395a0171f..c00ac4650dce29 100644 --- a/drivers/s390/scsi/zfcp_aux.c +++ b/drivers/s390/scsi/zfcp_aux.c @@ -529,7 +529,7 @@ struct zfcp_port *zfcp_port_enqueue(struct zfcp_adapter *adapter, u64 wwpn, list_add_tail(&port->list, &adapter->port_list); write_unlock_irq(&adapter->port_list_lock); - atomic_set_mask(status | ZFCP_STATUS_COMMON_RUNNING, &port->status); + atomic_or(status | ZFCP_STATUS_COMMON_RUNNING, &port->status); return port; diff --git a/drivers/s390/scsi/zfcp_erp.c b/drivers/s390/scsi/zfcp_erp.c index acde3f5d6e9e7f..3fb410977014f8 100644 --- a/drivers/s390/scsi/zfcp_erp.c +++ b/drivers/s390/scsi/zfcp_erp.c @@ -190,7 +190,7 @@ static struct zfcp_erp_action *zfcp_erp_setup_act(int need, u32 act_status, if (!(act_status & ZFCP_STATUS_ERP_NO_REF)) if (scsi_device_get(sdev)) return NULL; - atomic_set_mask(ZFCP_STATUS_COMMON_ERP_INUSE, + atomic_or(ZFCP_STATUS_COMMON_ERP_INUSE, &zfcp_sdev->status); erp_action = &zfcp_sdev->erp_action; memset(erp_action, 0, sizeof(struct zfcp_erp_action)); @@ -206,7 +206,7 @@ static struct zfcp_erp_action *zfcp_erp_setup_act(int need, u32 act_status, if (!get_device(&port->dev)) return NULL; zfcp_erp_action_dismiss_port(port); - atomic_set_mask(ZFCP_STATUS_COMMON_ERP_INUSE, &port->status); + atomic_or(ZFCP_STATUS_COMMON_ERP_INUSE, &port->status); erp_action = &port->erp_action; memset(erp_action, 0, sizeof(struct zfcp_erp_action)); erp_action->port = port; @@ -217,7 +217,7 @@ static struct zfcp_erp_action *zfcp_erp_setup_act(int need, u32 act_status, case ZFCP_ERP_ACTION_REOPEN_ADAPTER: kref_get(&adapter->ref); zfcp_erp_action_dismiss_adapter(adapter); - atomic_set_mask(ZFCP_STATUS_COMMON_ERP_INUSE, &adapter->status); + atomic_or(ZFCP_STATUS_COMMON_ERP_INUSE, &adapter->status); erp_action = &adapter->erp_action; memset(erp_action, 0, sizeof(struct zfcp_erp_action)); if (!(atomic_read(&adapter->status) & @@ -254,7 +254,7 @@ static int zfcp_erp_action_enqueue(int want, struct zfcp_adapter *adapter, act = zfcp_erp_setup_act(need, act_status, adapter, port, sdev); if (!act) goto out; - atomic_set_mask(ZFCP_STATUS_ADAPTER_ERP_PENDING, &adapter->status); + atomic_or(ZFCP_STATUS_ADAPTER_ERP_PENDING, &adapter->status); ++adapter->erp_total_count; list_add_tail(&act->list, &adapter->erp_ready_head); wake_up(&adapter->erp_ready_wq); @@ -486,14 +486,14 @@ static void zfcp_erp_adapter_unblock(struct zfcp_adapter *adapter) { if (status_change_set(ZFCP_STATUS_COMMON_UNBLOCKED, &adapter->status)) zfcp_dbf_rec_run("eraubl1", &adapter->erp_action); - atomic_set_mask(ZFCP_STATUS_COMMON_UNBLOCKED, &adapter->status); + atomic_or(ZFCP_STATUS_COMMON_UNBLOCKED, &adapter->status); } static void zfcp_erp_port_unblock(struct zfcp_port *port) { if (status_change_set(ZFCP_STATUS_COMMON_UNBLOCKED, &port->status)) zfcp_dbf_rec_run("erpubl1", &port->erp_action); - atomic_set_mask(ZFCP_STATUS_COMMON_UNBLOCKED, &port->status); + atomic_or(ZFCP_STATUS_COMMON_UNBLOCKED, &port->status); } static void zfcp_erp_lun_unblock(struct scsi_device *sdev) @@ -502,7 +502,7 @@ static void zfcp_erp_lun_unblock(struct scsi_device *sdev) if (status_change_set(ZFCP_STATUS_COMMON_UNBLOCKED, &zfcp_sdev->status)) zfcp_dbf_rec_run("erlubl1", &sdev_to_zfcp(sdev)->erp_action); - atomic_set_mask(ZFCP_STATUS_COMMON_UNBLOCKED, &zfcp_sdev->status); + atomic_or(ZFCP_STATUS_COMMON_UNBLOCKED, &zfcp_sdev->status); } static void zfcp_erp_action_to_running(struct zfcp_erp_action *erp_action) @@ -642,7 +642,7 @@ static void zfcp_erp_wakeup(struct zfcp_adapter *adapter) read_lock_irqsave(&adapter->erp_lock, flags); if (list_empty(&adapter->erp_ready_head) && list_empty(&adapter->erp_running_head)) { - atomic_clear_mask(ZFCP_STATUS_ADAPTER_ERP_PENDING, + atomic_andnot(ZFCP_STATUS_ADAPTER_ERP_PENDING, &adapter->status); wake_up(&adapter->erp_done_wqh); } @@ -665,16 +665,16 @@ static int zfcp_erp_adapter_strat_fsf_xconf(struct zfcp_erp_action *erp_action) int sleep = 1; struct zfcp_adapter *adapter = erp_action->adapter; - atomic_clear_mask(ZFCP_STATUS_ADAPTER_XCONFIG_OK, &adapter->status); + atomic_andnot(ZFCP_STATUS_ADAPTER_XCONFIG_OK, &adapter->status); for (retries = 7; retries; retries--) { - atomic_clear_mask(ZFCP_STATUS_ADAPTER_HOST_CON_INIT, + atomic_andnot(ZFCP_STATUS_ADAPTER_HOST_CON_INIT, &adapter->status); write_lock_irq(&adapter->erp_lock); zfcp_erp_action_to_running(erp_action); write_unlock_irq(&adapter->erp_lock); if (zfcp_fsf_exchange_config_data(erp_action)) { - atomic_clear_mask(ZFCP_STATUS_ADAPTER_HOST_CON_INIT, + atomic_andnot(ZFCP_STATUS_ADAPTER_HOST_CON_INIT, &adapter->status); return ZFCP_ERP_FAILED; } @@ -692,7 +692,7 @@ static int zfcp_erp_adapter_strat_fsf_xconf(struct zfcp_erp_action *erp_action) sleep *= 2; } - atomic_clear_mask(ZFCP_STATUS_ADAPTER_HOST_CON_INIT, + atomic_andnot(ZFCP_STATUS_ADAPTER_HOST_CON_INIT, &adapter->status); if (!(atomic_read(&adapter->status) & ZFCP_STATUS_ADAPTER_XCONFIG_OK)) @@ -764,7 +764,7 @@ static void zfcp_erp_adapter_strategy_close(struct zfcp_erp_action *act) /* all ports and LUNs are closed */ zfcp_erp_clear_adapter_status(adapter, ZFCP_STATUS_COMMON_OPEN); - atomic_clear_mask(ZFCP_STATUS_ADAPTER_XCONFIG_OK | + atomic_andnot(ZFCP_STATUS_ADAPTER_XCONFIG_OK | ZFCP_STATUS_ADAPTER_LINK_UNPLUGGED, &adapter->status); } @@ -773,7 +773,7 @@ static int zfcp_erp_adapter_strategy_open(struct zfcp_erp_action *act) struct zfcp_adapter *adapter = act->adapter; if (zfcp_qdio_open(adapter->qdio)) { - atomic_clear_mask(ZFCP_STATUS_ADAPTER_XCONFIG_OK | + atomic_andnot(ZFCP_STATUS_ADAPTER_XCONFIG_OK | ZFCP_STATUS_ADAPTER_LINK_UNPLUGGED, &adapter->status); return ZFCP_ERP_FAILED; @@ -784,7 +784,7 @@ static int zfcp_erp_adapter_strategy_open(struct zfcp_erp_action *act) return ZFCP_ERP_FAILED; } - atomic_set_mask(ZFCP_STATUS_COMMON_OPEN, &adapter->status); + atomic_or(ZFCP_STATUS_COMMON_OPEN, &adapter->status); return ZFCP_ERP_SUCCEEDED; } @@ -948,7 +948,7 @@ static void zfcp_erp_lun_strategy_clearstati(struct scsi_device *sdev) { struct zfcp_scsi_dev *zfcp_sdev = sdev_to_zfcp(sdev); - atomic_clear_mask(ZFCP_STATUS_COMMON_ACCESS_DENIED, + atomic_andnot(ZFCP_STATUS_COMMON_ACCESS_DENIED, &zfcp_sdev->status); } @@ -1187,18 +1187,18 @@ static void zfcp_erp_action_dequeue(struct zfcp_erp_action *erp_action) switch (erp_action->action) { case ZFCP_ERP_ACTION_REOPEN_LUN: zfcp_sdev = sdev_to_zfcp(erp_action->sdev); - atomic_clear_mask(ZFCP_STATUS_COMMON_ERP_INUSE, + atomic_andnot(ZFCP_STATUS_COMMON_ERP_INUSE, &zfcp_sdev->status); break; case ZFCP_ERP_ACTION_REOPEN_PORT_FORCED: case ZFCP_ERP_ACTION_REOPEN_PORT: - atomic_clear_mask(ZFCP_STATUS_COMMON_ERP_INUSE, + atomic_andnot(ZFCP_STATUS_COMMON_ERP_INUSE, &erp_action->port->status); break; case ZFCP_ERP_ACTION_REOPEN_ADAPTER: - atomic_clear_mask(ZFCP_STATUS_COMMON_ERP_INUSE, + atomic_andnot(ZFCP_STATUS_COMMON_ERP_INUSE, &erp_action->adapter->status); break; } @@ -1422,19 +1422,19 @@ void zfcp_erp_set_adapter_status(struct zfcp_adapter *adapter, u32 mask) unsigned long flags; u32 common_mask = mask & ZFCP_COMMON_FLAGS; - atomic_set_mask(mask, &adapter->status); + atomic_or(mask, &adapter->status); if (!common_mask) return; read_lock_irqsave(&adapter->port_list_lock, flags); list_for_each_entry(port, &adapter->port_list, list) - atomic_set_mask(common_mask, &port->status); + atomic_or(common_mask, &port->status); read_unlock_irqrestore(&adapter->port_list_lock, flags); spin_lock_irqsave(adapter->scsi_host->host_lock, flags); __shost_for_each_device(sdev, adapter->scsi_host) - atomic_set_mask(common_mask, &sdev_to_zfcp(sdev)->status); + atomic_or(common_mask, &sdev_to_zfcp(sdev)->status); spin_unlock_irqrestore(adapter->scsi_host->host_lock, flags); } @@ -1453,7 +1453,7 @@ void zfcp_erp_clear_adapter_status(struct zfcp_adapter *adapter, u32 mask) u32 common_mask = mask & ZFCP_COMMON_FLAGS; u32 clear_counter = mask & ZFCP_STATUS_COMMON_ERP_FAILED; - atomic_clear_mask(mask, &adapter->status); + atomic_andnot(mask, &adapter->status); if (!common_mask) return; @@ -1463,7 +1463,7 @@ void zfcp_erp_clear_adapter_status(struct zfcp_adapter *adapter, u32 mask) read_lock_irqsave(&adapter->port_list_lock, flags); list_for_each_entry(port, &adapter->port_list, list) { - atomic_clear_mask(common_mask, &port->status); + atomic_andnot(common_mask, &port->status); if (clear_counter) atomic_set(&port->erp_counter, 0); } @@ -1471,7 +1471,7 @@ void zfcp_erp_clear_adapter_status(struct zfcp_adapter *adapter, u32 mask) spin_lock_irqsave(adapter->scsi_host->host_lock, flags); __shost_for_each_device(sdev, adapter->scsi_host) { - atomic_clear_mask(common_mask, &sdev_to_zfcp(sdev)->status); + atomic_andnot(common_mask, &sdev_to_zfcp(sdev)->status); if (clear_counter) atomic_set(&sdev_to_zfcp(sdev)->erp_counter, 0); } @@ -1491,7 +1491,7 @@ void zfcp_erp_set_port_status(struct zfcp_port *port, u32 mask) u32 common_mask = mask & ZFCP_COMMON_FLAGS; unsigned long flags; - atomic_set_mask(mask, &port->status); + atomic_or(mask, &port->status); if (!common_mask) return; @@ -1499,7 +1499,7 @@ void zfcp_erp_set_port_status(struct zfcp_port *port, u32 mask) spin_lock_irqsave(port->adapter->scsi_host->host_lock, flags); __shost_for_each_device(sdev, port->adapter->scsi_host) if (sdev_to_zfcp(sdev)->port == port) - atomic_set_mask(common_mask, + atomic_or(common_mask, &sdev_to_zfcp(sdev)->status); spin_unlock_irqrestore(port->adapter->scsi_host->host_lock, flags); } @@ -1518,7 +1518,7 @@ void zfcp_erp_clear_port_status(struct zfcp_port *port, u32 mask) u32 clear_counter = mask & ZFCP_STATUS_COMMON_ERP_FAILED; unsigned long flags; - atomic_clear_mask(mask, &port->status); + atomic_andnot(mask, &port->status); if (!common_mask) return; @@ -1529,7 +1529,7 @@ void zfcp_erp_clear_port_status(struct zfcp_port *port, u32 mask) spin_lock_irqsave(port->adapter->scsi_host->host_lock, flags); __shost_for_each_device(sdev, port->adapter->scsi_host) if (sdev_to_zfcp(sdev)->port == port) { - atomic_clear_mask(common_mask, + atomic_andnot(common_mask, &sdev_to_zfcp(sdev)->status); if (clear_counter) atomic_set(&sdev_to_zfcp(sdev)->erp_counter, 0); @@ -1546,7 +1546,7 @@ void zfcp_erp_set_lun_status(struct scsi_device *sdev, u32 mask) { struct zfcp_scsi_dev *zfcp_sdev = sdev_to_zfcp(sdev); - atomic_set_mask(mask, &zfcp_sdev->status); + atomic_or(mask, &zfcp_sdev->status); } /** @@ -1558,7 +1558,7 @@ void zfcp_erp_clear_lun_status(struct scsi_device *sdev, u32 mask) { struct zfcp_scsi_dev *zfcp_sdev = sdev_to_zfcp(sdev); - atomic_clear_mask(mask, &zfcp_sdev->status); + atomic_andnot(mask, &zfcp_sdev->status); if (mask & ZFCP_STATUS_COMMON_ERP_FAILED) atomic_set(&zfcp_sdev->erp_counter, 0); diff --git a/drivers/s390/scsi/zfcp_fc.c b/drivers/s390/scsi/zfcp_fc.c index 25d49f32ca6380..237688af179b94 100644 --- a/drivers/s390/scsi/zfcp_fc.c +++ b/drivers/s390/scsi/zfcp_fc.c @@ -508,7 +508,7 @@ static void zfcp_fc_adisc_handler(void *data) /* port is good, unblock rport without going through erp */ zfcp_scsi_schedule_rport_register(port); out: - atomic_clear_mask(ZFCP_STATUS_PORT_LINK_TEST, &port->status); + atomic_andnot(ZFCP_STATUS_PORT_LINK_TEST, &port->status); put_device(&port->dev); kmem_cache_free(zfcp_fc_req_cache, fc_req); } @@ -564,14 +564,14 @@ void zfcp_fc_link_test_work(struct work_struct *work) if (atomic_read(&port->status) & ZFCP_STATUS_PORT_LINK_TEST) goto out; - atomic_set_mask(ZFCP_STATUS_PORT_LINK_TEST, &port->status); + atomic_or(ZFCP_STATUS_PORT_LINK_TEST, &port->status); retval = zfcp_fc_adisc(port); if (retval == 0) return; /* send of ADISC was not possible */ - atomic_clear_mask(ZFCP_STATUS_PORT_LINK_TEST, &port->status); + atomic_andnot(ZFCP_STATUS_PORT_LINK_TEST, &port->status); zfcp_erp_port_forced_reopen(port, 0, "fcltwk1"); out: @@ -640,7 +640,7 @@ static void zfcp_fc_validate_port(struct zfcp_port *port, struct list_head *lh) if (!(atomic_read(&port->status) & ZFCP_STATUS_COMMON_NOESC)) return; - atomic_clear_mask(ZFCP_STATUS_COMMON_NOESC, &port->status); + atomic_andnot(ZFCP_STATUS_COMMON_NOESC, &port->status); if ((port->supported_classes != 0) || !list_empty(&port->unit_list)) diff --git a/drivers/s390/scsi/zfcp_fsf.c b/drivers/s390/scsi/zfcp_fsf.c index 4ac73e047c114b..522a633c866a8b 100644 --- a/drivers/s390/scsi/zfcp_fsf.c +++ b/drivers/s390/scsi/zfcp_fsf.c @@ -114,7 +114,7 @@ static void zfcp_fsf_link_down_info_eval(struct zfcp_fsf_req *req, if (atomic_read(&adapter->status) & ZFCP_STATUS_ADAPTER_LINK_UNPLUGGED) return; - atomic_set_mask(ZFCP_STATUS_ADAPTER_LINK_UNPLUGGED, &adapter->status); + atomic_or(ZFCP_STATUS_ADAPTER_LINK_UNPLUGGED, &adapter->status); zfcp_scsi_schedule_rports_block(adapter); @@ -345,7 +345,7 @@ static void zfcp_fsf_protstatus_eval(struct zfcp_fsf_req *req) zfcp_erp_adapter_shutdown(adapter, 0, "fspse_3"); break; case FSF_PROT_HOST_CONNECTION_INITIALIZING: - atomic_set_mask(ZFCP_STATUS_ADAPTER_HOST_CON_INIT, + atomic_or(ZFCP_STATUS_ADAPTER_HOST_CON_INIT, &adapter->status); break; case FSF_PROT_DUPLICATE_REQUEST_ID: @@ -554,7 +554,7 @@ static void zfcp_fsf_exchange_config_data_handler(struct zfcp_fsf_req *req) zfcp_erp_adapter_shutdown(adapter, 0, "fsecdh1"); return; } - atomic_set_mask(ZFCP_STATUS_ADAPTER_XCONFIG_OK, + atomic_or(ZFCP_STATUS_ADAPTER_XCONFIG_OK, &adapter->status); break; case FSF_EXCHANGE_CONFIG_DATA_INCOMPLETE: @@ -567,7 +567,7 @@ static void zfcp_fsf_exchange_config_data_handler(struct zfcp_fsf_req *req) /* avoids adapter shutdown to be able to recognize * events such as LINK UP */ - atomic_set_mask(ZFCP_STATUS_ADAPTER_XCONFIG_OK, + atomic_or(ZFCP_STATUS_ADAPTER_XCONFIG_OK, &adapter->status); zfcp_fsf_link_down_info_eval(req, &qtcb->header.fsf_status_qual.link_down_info); @@ -1394,9 +1394,9 @@ static void zfcp_fsf_open_port_handler(struct zfcp_fsf_req *req) break; case FSF_GOOD: port->handle = header->port_handle; - atomic_set_mask(ZFCP_STATUS_COMMON_OPEN | + atomic_or(ZFCP_STATUS_COMMON_OPEN | ZFCP_STATUS_PORT_PHYS_OPEN, &port->status); - atomic_clear_mask(ZFCP_STATUS_COMMON_ACCESS_BOXED, + atomic_andnot(ZFCP_STATUS_COMMON_ACCESS_BOXED, &port->status); /* check whether D_ID has changed during open */ /* @@ -1677,10 +1677,10 @@ static void zfcp_fsf_close_physical_port_handler(struct zfcp_fsf_req *req) case FSF_PORT_BOXED: /* can't use generic zfcp_erp_modify_port_status because * ZFCP_STATUS_COMMON_OPEN must not be reset for the port */ - atomic_clear_mask(ZFCP_STATUS_PORT_PHYS_OPEN, &port->status); + atomic_andnot(ZFCP_STATUS_PORT_PHYS_OPEN, &port->status); shost_for_each_device(sdev, port->adapter->scsi_host) if (sdev_to_zfcp(sdev)->port == port) - atomic_clear_mask(ZFCP_STATUS_COMMON_OPEN, + atomic_andnot(ZFCP_STATUS_COMMON_OPEN, &sdev_to_zfcp(sdev)->status); zfcp_erp_set_port_status(port, ZFCP_STATUS_COMMON_ACCESS_BOXED); zfcp_erp_port_reopen(port, ZFCP_STATUS_COMMON_ERP_FAILED, @@ -1700,10 +1700,10 @@ static void zfcp_fsf_close_physical_port_handler(struct zfcp_fsf_req *req) /* can't use generic zfcp_erp_modify_port_status because * ZFCP_STATUS_COMMON_OPEN must not be reset for the port */ - atomic_clear_mask(ZFCP_STATUS_PORT_PHYS_OPEN, &port->status); + atomic_andnot(ZFCP_STATUS_PORT_PHYS_OPEN, &port->status); shost_for_each_device(sdev, port->adapter->scsi_host) if (sdev_to_zfcp(sdev)->port == port) - atomic_clear_mask(ZFCP_STATUS_COMMON_OPEN, + atomic_andnot(ZFCP_STATUS_COMMON_OPEN, &sdev_to_zfcp(sdev)->status); break; } @@ -1766,7 +1766,7 @@ static void zfcp_fsf_open_lun_handler(struct zfcp_fsf_req *req) zfcp_sdev = sdev_to_zfcp(sdev); - atomic_clear_mask(ZFCP_STATUS_COMMON_ACCESS_DENIED | + atomic_andnot(ZFCP_STATUS_COMMON_ACCESS_DENIED | ZFCP_STATUS_COMMON_ACCESS_BOXED, &zfcp_sdev->status); @@ -1822,7 +1822,7 @@ static void zfcp_fsf_open_lun_handler(struct zfcp_fsf_req *req) case FSF_GOOD: zfcp_sdev->lun_handle = header->lun_handle; - atomic_set_mask(ZFCP_STATUS_COMMON_OPEN, &zfcp_sdev->status); + atomic_or(ZFCP_STATUS_COMMON_OPEN, &zfcp_sdev->status); break; } } @@ -1913,7 +1913,7 @@ static void zfcp_fsf_close_lun_handler(struct zfcp_fsf_req *req) } break; case FSF_GOOD: - atomic_clear_mask(ZFCP_STATUS_COMMON_OPEN, &zfcp_sdev->status); + atomic_andnot(ZFCP_STATUS_COMMON_OPEN, &zfcp_sdev->status); break; } } diff --git a/drivers/s390/scsi/zfcp_qdio.c b/drivers/s390/scsi/zfcp_qdio.c index 495e1cb3afa600..dbf2b54703f721 100644 --- a/drivers/s390/scsi/zfcp_qdio.c +++ b/drivers/s390/scsi/zfcp_qdio.c @@ -349,7 +349,7 @@ void zfcp_qdio_close(struct zfcp_qdio *qdio) /* clear QDIOUP flag, thus do_QDIO is not called during qdio_shutdown */ spin_lock_irq(&qdio->req_q_lock); - atomic_clear_mask(ZFCP_STATUS_ADAPTER_QDIOUP, &adapter->status); + atomic_andnot(ZFCP_STATUS_ADAPTER_QDIOUP, &adapter->status); spin_unlock_irq(&qdio->req_q_lock); wake_up(&qdio->req_q_wq); @@ -384,7 +384,7 @@ int zfcp_qdio_open(struct zfcp_qdio *qdio) if (atomic_read(&adapter->status) & ZFCP_STATUS_ADAPTER_QDIOUP) return -EIO; - atomic_clear_mask(ZFCP_STATUS_ADAPTER_SIOSL_ISSUED, + atomic_andnot(ZFCP_STATUS_ADAPTER_SIOSL_ISSUED, &qdio->adapter->status); zfcp_qdio_setup_init_data(&init_data, qdio); @@ -396,14 +396,14 @@ int zfcp_qdio_open(struct zfcp_qdio *qdio) goto failed_qdio; if (ssqd.qdioac2 & CHSC_AC2_DATA_DIV_ENABLED) - atomic_set_mask(ZFCP_STATUS_ADAPTER_DATA_DIV_ENABLED, + atomic_or(ZFCP_STATUS_ADAPTER_DATA_DIV_ENABLED, &qdio->adapter->status); if (ssqd.qdioac2 & CHSC_AC2_MULTI_BUFFER_ENABLED) { - atomic_set_mask(ZFCP_STATUS_ADAPTER_MB_ACT, &adapter->status); + atomic_or(ZFCP_STATUS_ADAPTER_MB_ACT, &adapter->status); qdio->max_sbale_per_sbal = QDIO_MAX_ELEMENTS_PER_BUFFER; } else { - atomic_clear_mask(ZFCP_STATUS_ADAPTER_MB_ACT, &adapter->status); + atomic_andnot(ZFCP_STATUS_ADAPTER_MB_ACT, &adapter->status); qdio->max_sbale_per_sbal = QDIO_MAX_ELEMENTS_PER_BUFFER - 1; } @@ -427,7 +427,7 @@ int zfcp_qdio_open(struct zfcp_qdio *qdio) /* set index of first available SBALS / number of available SBALS */ qdio->req_q_idx = 0; atomic_set(&qdio->req_q_free, QDIO_MAX_BUFFERS_PER_Q); - atomic_set_mask(ZFCP_STATUS_ADAPTER_QDIOUP, &qdio->adapter->status); + atomic_or(ZFCP_STATUS_ADAPTER_QDIOUP, &qdio->adapter->status); if (adapter->scsi_host) { adapter->scsi_host->sg_tablesize = qdio->max_sbale_per_req; @@ -499,6 +499,6 @@ void zfcp_qdio_siosl(struct zfcp_adapter *adapter) rc = ccw_device_siosl(adapter->ccw_device); if (!rc) - atomic_set_mask(ZFCP_STATUS_ADAPTER_SIOSL_ISSUED, + atomic_or(ZFCP_STATUS_ADAPTER_SIOSL_ISSUED, &adapter->status); } diff --git a/drivers/staging/octeon/ethernet-rx.c b/drivers/staging/octeon/ethernet-rx.c index 22853d33da0525..d1a33a927f6d12 100644 --- a/drivers/staging/octeon/ethernet-rx.c +++ b/drivers/staging/octeon/ethernet-rx.c @@ -70,7 +70,14 @@ static irqreturn_t cvm_oct_do_interrupt(int cpl, void *dev_id) */ static inline int cvm_oct_check_rcv_error(cvmx_wqe_t *work) { - if ((work->word2.snoip.err_code == 10) && (work->len <= 64)) { + int port; + + if (octeon_has_feature(OCTEON_FEATURE_PKND)) + port = work->word0.pip.cn68xx.pknd; + else + port = work->word1.cn38xx.ipprt; + + if ((work->word2.snoip.err_code == 10) && (work->word1.len <= 64)) { /* * Ignore length errors on min size packets. Some * equipment incorrectly pads packets to 64+4FCS @@ -87,8 +94,8 @@ static inline int cvm_oct_check_rcv_error(cvmx_wqe_t *work) * packet to determine if we can remove a non spec * preamble and generate a correct packet. */ - int interface = cvmx_helper_get_interface_num(work->ipprt); - int index = cvmx_helper_get_interface_index_num(work->ipprt); + int interface = cvmx_helper_get_interface_num(port); + int index = cvmx_helper_get_interface_index_num(port); union cvmx_gmxx_rxx_frm_ctl gmxx_rxx_frm_ctl; gmxx_rxx_frm_ctl.u64 = @@ -99,7 +106,7 @@ static inline int cvm_oct_check_rcv_error(cvmx_wqe_t *work) cvmx_phys_to_ptr(work->packet_ptr.s.addr); int i = 0; - while (i < work->len - 1) { + while (i < work->word1.len - 1) { if (*ptr != 0x55) break; ptr++; @@ -109,18 +116,18 @@ static inline int cvm_oct_check_rcv_error(cvmx_wqe_t *work) if (*ptr == 0xd5) { /* printk_ratelimited("Port %d received 0xd5 preamble\n", - work->ipprt); + port); */ work->packet_ptr.s.addr += i + 1; - work->len -= i + 5; + work->word1.len -= i + 5; } else if ((*ptr & 0xf) == 0xd) { /* printk_ratelimited("Port %d received 0x?d preamble\n", - work->ipprt); + port); */ work->packet_ptr.s.addr += i; - work->len -= i + 4; - for (i = 0; i < work->len; i++) { + work->word1.len -= i + 4; + for (i = 0; i < work->word1.len; i++) { *ptr = ((*ptr & 0xf0) >> 4) | ((*(ptr + 1) & 0xf) << 4); @@ -128,7 +135,7 @@ static inline int cvm_oct_check_rcv_error(cvmx_wqe_t *work) } } else { printk_ratelimited("Port %d unknown preamble, packet dropped\n", - work->ipprt); + port); /* cvmx_helper_dump_packet(work); */ @@ -138,7 +145,7 @@ static inline int cvm_oct_check_rcv_error(cvmx_wqe_t *work) } } else { printk_ratelimited("Port %d receive error code %d, packet dropped\n", - work->ipprt, work->word2.snoip.err_code); + port, work->word2.snoip.err_code); cvm_oct_free_work(work); return 1; } @@ -172,9 +179,16 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) } /* Only allow work for our group (and preserve priorities) */ - old_group_mask = cvmx_read_csr(CVMX_POW_PP_GRP_MSKX(coreid)); - cvmx_write_csr(CVMX_POW_PP_GRP_MSKX(coreid), - (old_group_mask & ~0xFFFFull) | 1 << pow_receive_group); + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) { + old_group_mask = cvmx_read_csr(CVMX_SSO_PPX_GRP_MSK(coreid)); + cvmx_write_csr(CVMX_SSO_PPX_GRP_MSK(coreid), + 1ull << pow_receive_group); + cvmx_read_csr(CVMX_SSO_PPX_GRP_MSK(coreid)); /* Flush */ + } else { + old_group_mask = cvmx_read_csr(CVMX_POW_PP_GRP_MSKX(coreid)); + cvmx_write_csr(CVMX_POW_PP_GRP_MSKX(coreid), + (old_group_mask & ~0xFFFFull) | 1 << pow_receive_group); + } if (USE_ASYNC_IOBDMA) { cvmx_pow_work_request_async(CVMX_SCR_SCRATCH, CVMX_POW_NO_WAIT); @@ -186,6 +200,7 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) struct sk_buff **pskb = NULL; int skb_in_hw; cvmx_wqe_t *work; + int port; if (USE_ASYNC_IOBDMA && did_work_request) work = cvmx_pow_work_response_async(CVMX_SCR_SCRATCH); @@ -195,12 +210,19 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) prefetch(work); did_work_request = 0; if (work == NULL) { - union cvmx_pow_wq_int wq_int; + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) { + cvmx_write_csr(CVMX_SSO_WQ_IQ_DIS, + 1ull << pow_receive_group); + cvmx_write_csr(CVMX_SSO_WQ_INT, + 1ull << pow_receive_group); + } else { + union cvmx_pow_wq_int wq_int; - wq_int.u64 = 0; - wq_int.s.iq_dis = 1 << pow_receive_group; - wq_int.s.wq_int = 1 << pow_receive_group; - cvmx_write_csr(CVMX_POW_WQ_INT, wq_int.u64); + wq_int.u64 = 0; + wq_int.s.iq_dis = 1 << pow_receive_group; + wq_int.s.wq_int = 1 << pow_receive_group; + cvmx_write_csr(CVMX_POW_WQ_INT, wq_int.u64); + } break; } pskb = (struct sk_buff **)(cvm_oct_get_buffer_ptr(work->packet_ptr) - @@ -220,7 +242,13 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) prefetch(&skb->head); prefetch(&skb->len); } - prefetch(cvm_oct_device[work->ipprt]); + + if (octeon_has_feature(OCTEON_FEATURE_PKND)) + port = work->word0.pip.cn68xx.pknd; + else + port = work->word1.cn38xx.ipprt; + + prefetch(cvm_oct_device[port]); /* Immediately throw away all packets with receive errors */ if (unlikely(work->word2.snoip.rcv_error)) { @@ -237,7 +265,7 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) skb->data = skb->head + work->packet_ptr.s.addr - cvmx_ptr_to_phys(skb->head); prefetch(skb->data); - skb->len = work->len; + skb->len = work->word1.len; skb_set_tail_pointer(skb, skb->len); packet_not_copied = 1; } else { @@ -245,7 +273,7 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) * We have to copy the packet. First allocate * an skbuff for it. */ - skb = dev_alloc_skb(work->len); + skb = dev_alloc_skb(work->word1.len); if (!skb) { cvm_oct_free_work(work); continue; @@ -268,13 +296,14 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) else ptr += 6; } - memcpy(skb_put(skb, work->len), ptr, work->len); + memcpy(skb_put(skb, work->word1.len), ptr, + work->word1.len); /* No packet buffers to free */ } else { int segments = work->word2.s.bufs; union cvmx_buf_ptr segment_ptr = work->packet_ptr; - int len = work->len; + int len = work->word1.len; while (segments--) { union cvmx_buf_ptr next_ptr = @@ -310,10 +339,9 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) } packet_not_copied = 0; } - - if (likely((work->ipprt < TOTAL_NUMBER_OF_PORTS) && - cvm_oct_device[work->ipprt])) { - struct net_device *dev = cvm_oct_device[work->ipprt]; + if (likely((port < TOTAL_NUMBER_OF_PORTS) && + cvm_oct_device[port])) { + struct net_device *dev = cvm_oct_device[port]; struct octeon_ethernet *priv = netdev_priv(dev); /* @@ -333,7 +361,7 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) skb->ip_summed = CHECKSUM_UNNECESSARY; /* Increment RX stats for virtual ports */ - if (work->ipprt >= CVMX_PIP_NUM_INPUT_PORTS) { + if (port >= CVMX_PIP_NUM_INPUT_PORTS) { #ifdef CONFIG_64BIT atomic64_add(1, (atomic64_t *)&priv->stats.rx_packets); @@ -368,7 +396,7 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) * doesn't exist. */ printk_ratelimited("Port %d not controlled by Linux, packet dropped\n", - work->ipprt); + port); dev_kfree_skb_irq(skb); } /* @@ -390,7 +418,13 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget) } } /* Restore the original POW group mask */ - cvmx_write_csr(CVMX_POW_PP_GRP_MSKX(coreid), old_group_mask); + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) { + cvmx_write_csr(CVMX_SSO_PPX_GRP_MSK(coreid), old_group_mask); + cvmx_read_csr(CVMX_SSO_PPX_GRP_MSK(coreid)); /* Flush */ + } else { + cvmx_write_csr(CVMX_POW_PP_GRP_MSKX(coreid), old_group_mask); + } + if (USE_ASYNC_IOBDMA) { /* Restore the scratch area */ cvmx_scratch_write64(CVMX_SCR_SCRATCH, old_scratch); @@ -422,8 +456,6 @@ void cvm_oct_rx_initialize(void) { int i; struct net_device *dev_for_napi = NULL; - union cvmx_pow_wq_int_thrx int_thr; - union cvmx_pow_wq_int_pc int_pc; for (i = 0; i < TOTAL_NUMBER_OF_PORTS; i++) { if (cvm_oct_device[i]) { @@ -449,15 +481,34 @@ void cvm_oct_rx_initialize(void) disable_irq_nosync(OCTEON_IRQ_WORKQ0 + pow_receive_group); - int_thr.u64 = 0; - int_thr.s.tc_en = 1; - int_thr.s.tc_thr = 1; /* Enable POW interrupt when our port has at least one packet */ - cvmx_write_csr(CVMX_POW_WQ_INT_THRX(pow_receive_group), int_thr.u64); - - int_pc.u64 = 0; - int_pc.s.pc_thr = 5; - cvmx_write_csr(CVMX_POW_WQ_INT_PC, int_pc.u64); + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) { + union cvmx_sso_wq_int_thrx int_thr; + union cvmx_pow_wq_int_pc int_pc; + + int_thr.u64 = 0; + int_thr.s.tc_en = 1; + int_thr.s.tc_thr = 1; + cvmx_write_csr(CVMX_SSO_WQ_INT_THRX(pow_receive_group), + int_thr.u64); + + int_pc.u64 = 0; + int_pc.s.pc_thr = 5; + cvmx_write_csr(CVMX_SSO_WQ_INT_PC, int_pc.u64); + } else { + union cvmx_pow_wq_int_thrx int_thr; + union cvmx_pow_wq_int_pc int_pc; + + int_thr.u64 = 0; + int_thr.s.tc_en = 1; + int_thr.s.tc_thr = 1; + cvmx_write_csr(CVMX_POW_WQ_INT_THRX(pow_receive_group), + int_thr.u64); + + int_pc.u64 = 0; + int_pc.s.pc_thr = 5; + cvmx_write_csr(CVMX_POW_WQ_INT_PC, int_pc.u64); + } /* Schedule NAPI now. This will indirectly enable the interrupt. */ napi_schedule(&cvm_oct_napi); diff --git a/drivers/staging/octeon/ethernet-tx.c b/drivers/staging/octeon/ethernet-tx.c index e2df041ca82d4e..9e2116f4c91538 100644 --- a/drivers/staging/octeon/ethernet-tx.c +++ b/drivers/staging/octeon/ethernet-tx.c @@ -591,13 +591,14 @@ int cvm_oct_xmit_pow(struct sk_buff *skb, struct net_device *dev) * Fill in some of the work queue fields. We may need to add * more if the software at the other end needs them. */ - work->hw_chksum = skb->csum; - work->len = skb->len; - work->ipprt = priv->port; - work->qos = priv->port & 0x7; - work->grp = pow_send_group; - work->tag_type = CVMX_HELPER_INPUT_TAG_TYPE; - work->tag = pow_send_group; /* FIXME */ + if (!OCTEON_IS_MODEL(OCTEON_CN68XX)) + work->word0.pip.cn38xx.hw_chksum = skb->csum; + work->word1.len = skb->len; + cvmx_wqe_set_port(work, priv->port); + cvmx_wqe_set_qos(work, priv->port & 0x7); + cvmx_wqe_set_grp(work, pow_send_group); + work->word1.tag_type = CVMX_HELPER_INPUT_TAG_TYPE; + work->word1.tag = pow_send_group; /* FIXME */ /* Default to zero. Sets of zero later are commented out */ work->word2.u64 = 0; work->word2.s.bufs = 1; @@ -677,8 +678,8 @@ int cvm_oct_xmit_pow(struct sk_buff *skb, struct net_device *dev) } /* Submit the packet to the POW */ - cvmx_pow_work_submit(work, work->tag, work->tag_type, work->qos, - work->grp); + cvmx_pow_work_submit(work, work->word1.tag, work->word1.tag_type, + cvmx_wqe_get_qos(work), cvmx_wqe_get_grp(work)); priv->stats.tx_packets++; priv->stats.tx_bytes += skb->len; dev_consume_skb_any(skb); diff --git a/drivers/staging/octeon/ethernet-util.h b/drivers/staging/octeon/ethernet-util.h index 1ba789a7741bd2..45f024bc5e33f7 100644 --- a/drivers/staging/octeon/ethernet-util.h +++ b/drivers/staging/octeon/ethernet-util.h @@ -8,6 +8,10 @@ * published by the Free Software Foundation. */ +#include +#include +#include + /** * cvm_oct_get_buffer_ptr - convert packet data address to pointer * @packet_ptr: Packet data hardware address @@ -28,14 +32,12 @@ static inline void *cvm_oct_get_buffer_ptr(union cvmx_buf_ptr packet_ptr) */ static inline int INTERFACE(int ipd_port) { - if (ipd_port < 32) /* Interface 0 or 1 for RGMII,GMII,SPI, etc */ - return ipd_port >> 4; - else if (ipd_port < 36) /* Interface 2 for NPI */ - return 2; - else if (ipd_port < 40) /* Interface 3 for loopback */ - return 3; - else if (ipd_port == 40) /* Non existent interface for POW0 */ - return 4; + int interface = cvmx_helper_get_interface_num(ipd_port); + + if (interface >= 0) + return interface; + else if (ipd_port == CVMX_PIP_NUM_INPUT_PORTS) + return 10; panic("Illegal ipd_port %d passed to INTERFACE\n", ipd_port); } @@ -47,7 +49,5 @@ static inline int INTERFACE(int ipd_port) */ static inline int INDEX(int ipd_port) { - if (ipd_port < 32) - return ipd_port & 15; - return ipd_port & 3; + return cvmx_helper_get_interface_index_num(ipd_port); } diff --git a/drivers/staging/octeon/ethernet.c b/drivers/staging/octeon/ethernet.c index 0718b35f733575..7274fda0b77b9f 100644 --- a/drivers/staging/octeon/ethernet.c +++ b/drivers/staging/octeon/ethernet.c @@ -152,7 +152,7 @@ static void cvm_oct_configure_common_hw(void) num_packet_buffers); if (CVMX_FPA_OUTPUT_BUFFER_POOL != CVMX_FPA_PACKET_POOL) cvm_oct_mem_fill_fpa(CVMX_FPA_OUTPUT_BUFFER_POOL, - CVMX_FPA_OUTPUT_BUFFER_POOL_SIZE, 128); + CVMX_FPA_OUTPUT_BUFFER_POOL_SIZE, 1024); #ifdef __LITTLE_ENDIAN { @@ -860,7 +860,10 @@ static int cvm_oct_remove(struct platform_device *pdev) int port; /* Disable POW interrupt */ - cvmx_write_csr(CVMX_POW_WQ_INT_THRX(pow_receive_group), 0); + if (OCTEON_IS_MODEL(OCTEON_CN68XX)) + cvmx_write_csr(CVMX_SSO_WQ_INT_THRX(pow_receive_group), 0); + else + cvmx_write_csr(CVMX_POW_WQ_INT_THRX(pow_receive_group), 0); cvmx_ipd_disable(); diff --git a/drivers/tty/hvc/hvsi.c b/drivers/tty/hvc/hvsi.c index 41901997c0d628..a75146f600cb7c 100644 --- a/drivers/tty/hvc/hvsi.c +++ b/drivers/tty/hvc/hvsi.c @@ -240,9 +240,9 @@ static void hvsi_recv_control(struct hvsi_struct *hp, uint8_t *packet, { struct hvsi_control *header = (struct hvsi_control *)packet; - switch (header->verb) { + switch (be16_to_cpu(header->verb)) { case VSV_MODEM_CTL_UPDATE: - if ((header->word & HVSI_TSCD) == 0) { + if ((be32_to_cpu(header->word) & HVSI_TSCD) == 0) { /* CD went away; no more connection */ pr_debug("hvsi%i: CD dropped\n", hp->index); hp->mctrl &= TIOCM_CD; @@ -267,6 +267,7 @@ static void hvsi_recv_control(struct hvsi_struct *hp, uint8_t *packet, static void hvsi_recv_response(struct hvsi_struct *hp, uint8_t *packet) { struct hvsi_query_response *resp = (struct hvsi_query_response *)packet; + uint32_t mctrl_word; switch (hp->state) { case HVSI_WAIT_FOR_VER_RESPONSE: @@ -274,9 +275,10 @@ static void hvsi_recv_response(struct hvsi_struct *hp, uint8_t *packet) break; case HVSI_WAIT_FOR_MCTRL_RESPONSE: hp->mctrl = 0; - if (resp->u.mctrl_word & HVSI_TSDTR) + mctrl_word = be32_to_cpu(resp->u.mctrl_word); + if (mctrl_word & HVSI_TSDTR) hp->mctrl |= TIOCM_DTR; - if (resp->u.mctrl_word & HVSI_TSCD) + if (mctrl_word & HVSI_TSCD) hp->mctrl |= TIOCM_CD; __set_state(hp, HVSI_OPEN); break; @@ -295,10 +297,10 @@ static int hvsi_version_respond(struct hvsi_struct *hp, uint16_t query_seqno) packet.hdr.type = VS_QUERY_RESPONSE_PACKET_HEADER; packet.hdr.len = sizeof(struct hvsi_query_response); - packet.hdr.seqno = atomic_inc_return(&hp->seqno); - packet.verb = VSV_SEND_VERSION_NUMBER; + packet.hdr.seqno = cpu_to_be16(atomic_inc_return(&hp->seqno)); + packet.verb = cpu_to_be16(VSV_SEND_VERSION_NUMBER); packet.u.version = HVSI_VERSION; - packet.query_seqno = query_seqno+1; + packet.query_seqno = cpu_to_be16(query_seqno+1); pr_debug("%s: sending %i bytes\n", __func__, packet.hdr.len); dbg_dump_hex((uint8_t*)&packet, packet.hdr.len); @@ -319,7 +321,7 @@ static void hvsi_recv_query(struct hvsi_struct *hp, uint8_t *packet) switch (hp->state) { case HVSI_WAIT_FOR_VER_QUERY: - hvsi_version_respond(hp, query->hdr.seqno); + hvsi_version_respond(hp, be16_to_cpu(query->hdr.seqno)); __set_state(hp, HVSI_OPEN); break; default: @@ -555,8 +557,8 @@ static int hvsi_query(struct hvsi_struct *hp, uint16_t verb) packet.hdr.type = VS_QUERY_PACKET_HEADER; packet.hdr.len = sizeof(struct hvsi_query); - packet.hdr.seqno = atomic_inc_return(&hp->seqno); - packet.verb = verb; + packet.hdr.seqno = cpu_to_be16(atomic_inc_return(&hp->seqno)); + packet.verb = cpu_to_be16(verb); pr_debug("%s: sending %i bytes\n", __func__, packet.hdr.len); dbg_dump_hex((uint8_t*)&packet, packet.hdr.len); @@ -596,14 +598,14 @@ static int hvsi_set_mctrl(struct hvsi_struct *hp, uint16_t mctrl) struct hvsi_control packet __ALIGNED__; int wrote; - packet.hdr.type = VS_CONTROL_PACKET_HEADER, - packet.hdr.seqno = atomic_inc_return(&hp->seqno); + packet.hdr.type = VS_CONTROL_PACKET_HEADER; + packet.hdr.seqno = cpu_to_be16(atomic_inc_return(&hp->seqno)); packet.hdr.len = sizeof(struct hvsi_control); - packet.verb = VSV_SET_MODEM_CTL; - packet.mask = HVSI_TSDTR; + packet.verb = cpu_to_be16(VSV_SET_MODEM_CTL); + packet.mask = cpu_to_be32(HVSI_TSDTR); if (mctrl & TIOCM_DTR) - packet.word = HVSI_TSDTR; + packet.word = cpu_to_be32(HVSI_TSDTR); pr_debug("%s: sending %i bytes\n", __func__, packet.hdr.len); dbg_dump_hex((uint8_t*)&packet, packet.hdr.len); @@ -680,7 +682,7 @@ static int hvsi_put_chars(struct hvsi_struct *hp, const char *buf, int count) BUG_ON(count > HVSI_MAX_OUTGOING_DATA); packet.hdr.type = VS_DATA_PACKET_HEADER; - packet.hdr.seqno = atomic_inc_return(&hp->seqno); + packet.hdr.seqno = cpu_to_be16(atomic_inc_return(&hp->seqno)); packet.hdr.len = count + sizeof(struct hvsi_header); memcpy(&packet.data, buf, count); @@ -697,9 +699,9 @@ static void hvsi_close_protocol(struct hvsi_struct *hp) struct hvsi_control packet __ALIGNED__; packet.hdr.type = VS_CONTROL_PACKET_HEADER; - packet.hdr.seqno = atomic_inc_return(&hp->seqno); + packet.hdr.seqno = cpu_to_be16(atomic_inc_return(&hp->seqno)); packet.hdr.len = 6; - packet.verb = VSV_CLOSE_PROTOCOL; + packet.verb = cpu_to_be16(VSV_CLOSE_PROTOCOL); pr_debug("%s: sending %i bytes\n", __func__, packet.hdr.len); dbg_dump_hex((uint8_t*)&packet, packet.hdr.len); @@ -1180,7 +1182,7 @@ static int __init hvsi_console_init(void) /* search device tree for vty nodes */ for_each_compatible_node(vty, "serial", "hvterm-protocol") { struct hvsi_struct *hp; - const uint32_t *vtermno, *irq; + const __be32 *vtermno, *irq; vtermno = of_get_property(vty, "reg", NULL); irq = of_get_property(vty, "interrupts", NULL); @@ -1202,11 +1204,11 @@ static int __init hvsi_console_init(void) hp->index = hvsi_count; hp->inbuf_end = hp->inbuf; hp->state = HVSI_CLOSED; - hp->vtermno = *vtermno; - hp->virq = irq_create_mapping(NULL, irq[0]); + hp->vtermno = be32_to_cpup(vtermno); + hp->virq = irq_create_mapping(NULL, be32_to_cpup(irq)); if (hp->virq == 0) { printk(KERN_ERR "%s: couldn't create irq mapping for 0x%x\n", - __func__, irq[0]); + __func__, be32_to_cpup(irq)); tty_port_destroy(&hp->port); continue; } diff --git a/drivers/tty/mips_ejtag_fdc.c b/drivers/tty/mips_ejtag_fdc.c index 358323c83b4f34..a8c8cfd52a23b8 100644 --- a/drivers/tty/mips_ejtag_fdc.c +++ b/drivers/tty/mips_ejtag_fdc.c @@ -879,6 +879,11 @@ static const struct tty_operations mips_ejtag_fdc_tty_ops = { .chars_in_buffer = mips_ejtag_fdc_tty_chars_in_buffer, }; +int __weak get_c0_fdc_int(void) +{ + return -1; +} + static int mips_ejtag_fdc_tty_probe(struct mips_cdmm_device *dev) { int ret, nport; @@ -967,9 +972,7 @@ static int mips_ejtag_fdc_tty_probe(struct mips_cdmm_device *dev) wake_up_process(priv->thread); /* Look for an FDC IRQ */ - priv->irq = -1; - if (get_c0_fdc_int) - priv->irq = get_c0_fdc_int(); + priv->irq = get_c0_fdc_int(); /* Try requesting the IRQ */ if (priv->irq >= 0) { diff --git a/fs/Kconfig b/fs/Kconfig index 011f43365d7b1e..da3f32f1a4e4bd 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -11,18 +11,15 @@ config DCACHE_WORD_ACCESS if BLOCK source "fs/ext2/Kconfig" -source "fs/ext3/Kconfig" source "fs/ext4/Kconfig" -source "fs/jbd/Kconfig" source "fs/jbd2/Kconfig" config FS_MBCACHE # Meta block cache for Extended Attributes (ext2/ext3/ext4) tristate default y if EXT2_FS=y && EXT2_FS_XATTR - default y if EXT3_FS=y && EXT3_FS_XATTR default y if EXT4_FS=y - default m if EXT2_FS_XATTR || EXT3_FS_XATTR || EXT4_FS + default m if EXT2_FS_XATTR || EXT4_FS source "fs/reiserfs/Kconfig" source "fs/jfs/Kconfig" diff --git a/fs/Makefile b/fs/Makefile index cb20e4bf230398..09e051fefc5b4e 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -62,12 +62,10 @@ obj-$(CONFIG_DLM) += dlm/ # Do not add any filesystems before this line obj-$(CONFIG_FSCACHE) += fscache/ obj-$(CONFIG_REISERFS_FS) += reiserfs/ -obj-$(CONFIG_EXT3_FS) += ext3/ # Before ext2 so root fs can be ext3 obj-$(CONFIG_EXT2_FS) += ext2/ # We place ext4 after ext2 so plain ext2 root fs's are mounted using ext2 # unless explicitly requested by rootfstype obj-$(CONFIG_EXT4_FS) += ext4/ -obj-$(CONFIG_JBD) += jbd/ obj-$(CONFIG_JBD2) += jbd2/ obj-$(CONFIG_CRAMFS) += cramfs/ obj-$(CONFIG_SQUASHFS) += squashfs/ diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index 754fd6c0b7470b..87e9d796cf7dd9 100644 --- a/fs/dlm/lowcomms.c +++ b/fs/dlm/lowcomms.c @@ -120,12 +120,11 @@ struct connection { struct cbuf cb; int retries; #define MAX_CONNECT_RETRIES 3 - int sctp_assoc; struct hlist_node list; struct connection *othercon; struct work_struct rwork; /* Receive workqueue */ struct work_struct swork; /* Send workqueue */ - bool try_new_addr; + void (*orig_error_report)(struct sock *sk); }; #define sock2con(x) ((struct connection *)(x)->sk_user_data) @@ -252,26 +251,6 @@ static struct connection *nodeid2con(int nodeid, gfp_t allocation) return con; } -/* This is a bit drastic, but only called when things go wrong */ -static struct connection *assoc2con(int assoc_id) -{ - int i; - struct connection *con; - - mutex_lock(&connections_lock); - - for (i = 0 ; i < CONN_HASH_SIZE; i++) { - hlist_for_each_entry(con, &connection_hash[i], list) { - if (con->sctp_assoc == assoc_id) { - mutex_unlock(&connections_lock); - return con; - } - } - } - mutex_unlock(&connections_lock); - return NULL; -} - static struct dlm_node_addr *find_node_addr(int nodeid) { struct dlm_node_addr *na; @@ -322,14 +301,14 @@ static int nodeid_to_addr(int nodeid, struct sockaddr_storage *sas_out, spin_lock(&dlm_node_addrs_spin); na = find_node_addr(nodeid); if (na && na->addr_count) { + memcpy(&sas, na->addr[na->curr_addr_index], + sizeof(struct sockaddr_storage)); + if (try_new_addr) { na->curr_addr_index++; if (na->curr_addr_index == na->addr_count) na->curr_addr_index = 0; } - - memcpy(&sas, na->addr[na->curr_addr_index ], - sizeof(struct sockaddr_storage)); } spin_unlock(&dlm_node_addrs_spin); @@ -459,18 +438,23 @@ static inline void lowcomms_connect_sock(struct connection *con) static void lowcomms_state_change(struct sock *sk) { - if (sk->sk_state == TCP_ESTABLISHED) + /* SCTP layer is not calling sk_data_ready when the connection + * is done, so we catch the signal through here. Also, it + * doesn't switch socket state when entering shutdown, so we + * skip the write in that case. + */ + if (sk->sk_shutdown) { + if (sk->sk_shutdown == RCV_SHUTDOWN) + lowcomms_data_ready(sk); + } else if (sk->sk_state == TCP_ESTABLISHED) { lowcomms_write_space(sk); + } } int dlm_lowcomms_connect_node(int nodeid) { struct connection *con; - /* with sctp there's no connecting without sending */ - if (dlm_config.ci_protocol != 0) - return 0; - if (nodeid == dlm_our_nodeid()) return 0; @@ -481,6 +465,43 @@ int dlm_lowcomms_connect_node(int nodeid) return 0; } +static void lowcomms_error_report(struct sock *sk) +{ + struct connection *con = sock2con(sk); + struct sockaddr_storage saddr; + + if (nodeid_to_addr(con->nodeid, &saddr, NULL, false)) { + printk_ratelimited(KERN_ERR "dlm: node %d: socket error " + "sending to node %d, port %d, " + "sk_err=%d/%d\n", dlm_our_nodeid(), + con->nodeid, dlm_config.ci_tcp_port, + sk->sk_err, sk->sk_err_soft); + return; + } else if (saddr.ss_family == AF_INET) { + struct sockaddr_in *sin4 = (struct sockaddr_in *)&saddr; + + printk_ratelimited(KERN_ERR "dlm: node %d: socket error " + "sending to node %d at %pI4, port %d, " + "sk_err=%d/%d\n", dlm_our_nodeid(), + con->nodeid, &sin4->sin_addr.s_addr, + dlm_config.ci_tcp_port, sk->sk_err, + sk->sk_err_soft); + } else { + struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&saddr; + + printk_ratelimited(KERN_ERR "dlm: node %d: socket error " + "sending to node %d at %u.%u.%u.%u, " + "port %d, sk_err=%d/%d\n", dlm_our_nodeid(), + con->nodeid, sin6->sin6_addr.s6_addr32[0], + sin6->sin6_addr.s6_addr32[1], + sin6->sin6_addr.s6_addr32[2], + sin6->sin6_addr.s6_addr32[3], + dlm_config.ci_tcp_port, sk->sk_err, + sk->sk_err_soft); + } + con->orig_error_report(sk); +} + /* Make a socket active */ static void add_sock(struct socket *sock, struct connection *con) { @@ -492,6 +513,8 @@ static void add_sock(struct socket *sock, struct connection *con) con->sock->sk->sk_state_change = lowcomms_state_change; con->sock->sk->sk_user_data = con; con->sock->sk->sk_allocation = GFP_NOFS; + con->orig_error_report = con->sock->sk->sk_error_report; + con->sock->sk->sk_error_report = lowcomms_error_report; } /* Add the port number to an IPv6 or 4 sockaddr and return the address @@ -514,17 +537,24 @@ static void make_sockaddr(struct sockaddr_storage *saddr, uint16_t port, } /* Close a remote connection and tidy up */ -static void close_connection(struct connection *con, bool and_other) +static void close_connection(struct connection *con, bool and_other, + bool tx, bool rx) { - mutex_lock(&con->sock_mutex); + clear_bit(CF_CONNECT_PENDING, &con->flags); + clear_bit(CF_WRITE_PENDING, &con->flags); + if (tx && cancel_work_sync(&con->swork)) + log_print("canceled swork for node %d", con->nodeid); + if (rx && cancel_work_sync(&con->rwork)) + log_print("canceled rwork for node %d", con->nodeid); + mutex_lock(&con->sock_mutex); if (con->sock) { sock_release(con->sock); con->sock = NULL; } if (con->othercon && and_other) { /* Will only re-enter once. */ - close_connection(con->othercon, false); + close_connection(con->othercon, false, true, true); } if (con->rx_page) { __free_page(con->rx_page); @@ -535,254 +565,6 @@ static void close_connection(struct connection *con, bool and_other) mutex_unlock(&con->sock_mutex); } -/* We only send shutdown messages to nodes that are not part of the cluster */ -static void sctp_send_shutdown(sctp_assoc_t associd) -{ - static char outcmsg[CMSG_SPACE(sizeof(struct sctp_sndrcvinfo))]; - struct msghdr outmessage; - struct cmsghdr *cmsg; - struct sctp_sndrcvinfo *sinfo; - int ret; - struct connection *con; - - con = nodeid2con(0,0); - BUG_ON(con == NULL); - - outmessage.msg_name = NULL; - outmessage.msg_namelen = 0; - outmessage.msg_control = outcmsg; - outmessage.msg_controllen = sizeof(outcmsg); - outmessage.msg_flags = MSG_EOR; - - cmsg = CMSG_FIRSTHDR(&outmessage); - cmsg->cmsg_level = IPPROTO_SCTP; - cmsg->cmsg_type = SCTP_SNDRCV; - cmsg->cmsg_len = CMSG_LEN(sizeof(struct sctp_sndrcvinfo)); - outmessage.msg_controllen = cmsg->cmsg_len; - sinfo = CMSG_DATA(cmsg); - memset(sinfo, 0x00, sizeof(struct sctp_sndrcvinfo)); - - sinfo->sinfo_flags |= MSG_EOF; - sinfo->sinfo_assoc_id = associd; - - ret = kernel_sendmsg(con->sock, &outmessage, NULL, 0, 0); - - if (ret != 0) - log_print("send EOF to node failed: %d", ret); -} - -static void sctp_init_failed_foreach(struct connection *con) -{ - - /* - * Don't try to recover base con and handle race where the - * other node's assoc init creates a assoc and we get that - * notification, then we get a notification that our attempt - * failed due. This happens when we are still trying the primary - * address, but the other node has already tried secondary addrs - * and found one that worked. - */ - if (!con->nodeid || con->sctp_assoc) - return; - - log_print("Retrying SCTP association init for node %d\n", con->nodeid); - - con->try_new_addr = true; - con->sctp_assoc = 0; - if (test_and_clear_bit(CF_INIT_PENDING, &con->flags)) { - if (!test_and_set_bit(CF_WRITE_PENDING, &con->flags)) - queue_work(send_workqueue, &con->swork); - } -} - -/* INIT failed but we don't know which node... - restart INIT on all pending nodes */ -static void sctp_init_failed(void) -{ - mutex_lock(&connections_lock); - - foreach_conn(sctp_init_failed_foreach); - - mutex_unlock(&connections_lock); -} - -static void retry_failed_sctp_send(struct connection *recv_con, - struct sctp_send_failed *sn_send_failed, - char *buf) -{ - int len = sn_send_failed->ssf_length - sizeof(struct sctp_send_failed); - struct dlm_mhandle *mh; - struct connection *con; - char *retry_buf; - int nodeid = sn_send_failed->ssf_info.sinfo_ppid; - - log_print("Retry sending %d bytes to node id %d", len, nodeid); - - if (!nodeid) { - log_print("Shouldn't resend data via listening connection."); - return; - } - - con = nodeid2con(nodeid, 0); - if (!con) { - log_print("Could not look up con for nodeid %d\n", - nodeid); - return; - } - - mh = dlm_lowcomms_get_buffer(nodeid, len, GFP_NOFS, &retry_buf); - if (!mh) { - log_print("Could not allocate buf for retry."); - return; - } - memcpy(retry_buf, buf + sizeof(struct sctp_send_failed), len); - dlm_lowcomms_commit_buffer(mh); - - /* - * If we got a assoc changed event before the send failed event then - * we only need to retry the send. - */ - if (con->sctp_assoc) { - if (!test_and_set_bit(CF_WRITE_PENDING, &con->flags)) - queue_work(send_workqueue, &con->swork); - } else - sctp_init_failed_foreach(con); -} - -/* Something happened to an association */ -static void process_sctp_notification(struct connection *con, - struct msghdr *msg, char *buf) -{ - union sctp_notification *sn = (union sctp_notification *)buf; - struct linger linger; - - switch (sn->sn_header.sn_type) { - case SCTP_SEND_FAILED: - retry_failed_sctp_send(con, &sn->sn_send_failed, buf); - break; - case SCTP_ASSOC_CHANGE: - switch (sn->sn_assoc_change.sac_state) { - case SCTP_COMM_UP: - case SCTP_RESTART: - { - /* Check that the new node is in the lockspace */ - struct sctp_prim prim; - int nodeid; - int prim_len, ret; - int addr_len; - struct connection *new_con; - - /* - * We get this before any data for an association. - * We verify that the node is in the cluster and - * then peel off a socket for it. - */ - if ((int)sn->sn_assoc_change.sac_assoc_id <= 0) { - log_print("COMM_UP for invalid assoc ID %d", - (int)sn->sn_assoc_change.sac_assoc_id); - sctp_init_failed(); - return; - } - memset(&prim, 0, sizeof(struct sctp_prim)); - prim_len = sizeof(struct sctp_prim); - prim.ssp_assoc_id = sn->sn_assoc_change.sac_assoc_id; - - ret = kernel_getsockopt(con->sock, - IPPROTO_SCTP, - SCTP_PRIMARY_ADDR, - (char*)&prim, - &prim_len); - if (ret < 0) { - log_print("getsockopt/sctp_primary_addr on " - "new assoc %d failed : %d", - (int)sn->sn_assoc_change.sac_assoc_id, - ret); - - /* Retry INIT later */ - new_con = assoc2con(sn->sn_assoc_change.sac_assoc_id); - if (new_con) - clear_bit(CF_CONNECT_PENDING, &con->flags); - return; - } - make_sockaddr(&prim.ssp_addr, 0, &addr_len); - if (addr_to_nodeid(&prim.ssp_addr, &nodeid)) { - unsigned char *b=(unsigned char *)&prim.ssp_addr; - log_print("reject connect from unknown addr"); - print_hex_dump_bytes("ss: ", DUMP_PREFIX_NONE, - b, sizeof(struct sockaddr_storage)); - sctp_send_shutdown(prim.ssp_assoc_id); - return; - } - - new_con = nodeid2con(nodeid, GFP_NOFS); - if (!new_con) - return; - - /* Peel off a new sock */ - lock_sock(con->sock->sk); - ret = sctp_do_peeloff(con->sock->sk, - sn->sn_assoc_change.sac_assoc_id, - &new_con->sock); - release_sock(con->sock->sk); - if (ret < 0) { - log_print("Can't peel off a socket for " - "connection %d to node %d: err=%d", - (int)sn->sn_assoc_change.sac_assoc_id, - nodeid, ret); - return; - } - add_sock(new_con->sock, new_con); - - linger.l_onoff = 1; - linger.l_linger = 0; - ret = kernel_setsockopt(new_con->sock, SOL_SOCKET, SO_LINGER, - (char *)&linger, sizeof(linger)); - if (ret < 0) - log_print("set socket option SO_LINGER failed"); - - log_print("connecting to %d sctp association %d", - nodeid, (int)sn->sn_assoc_change.sac_assoc_id); - - new_con->sctp_assoc = sn->sn_assoc_change.sac_assoc_id; - new_con->try_new_addr = false; - /* Send any pending writes */ - clear_bit(CF_CONNECT_PENDING, &new_con->flags); - clear_bit(CF_INIT_PENDING, &new_con->flags); - if (!test_and_set_bit(CF_WRITE_PENDING, &new_con->flags)) { - queue_work(send_workqueue, &new_con->swork); - } - if (!test_and_set_bit(CF_READ_PENDING, &new_con->flags)) - queue_work(recv_workqueue, &new_con->rwork); - } - break; - - case SCTP_COMM_LOST: - case SCTP_SHUTDOWN_COMP: - { - con = assoc2con(sn->sn_assoc_change.sac_assoc_id); - if (con) { - con->sctp_assoc = 0; - } - } - break; - - case SCTP_CANT_STR_ASSOC: - { - /* Will retry init when we get the send failed notification */ - log_print("Can't start SCTP association - retrying"); - } - break; - - default: - log_print("unexpected SCTP assoc change id=%d state=%d", - (int)sn->sn_assoc_change.sac_assoc_id, - sn->sn_assoc_change.sac_state); - } - default: - ; /* fall through */ - } -} - /* Data received from remote end */ static int receive_from_sock(struct connection *con) { @@ -793,7 +575,6 @@ static int receive_from_sock(struct connection *con) int r; int call_again_soon = 0; int nvec; - char incmsg[CMSG_SPACE(sizeof(struct sctp_sndrcvinfo))]; mutex_lock(&con->sock_mutex); @@ -801,6 +582,10 @@ static int receive_from_sock(struct connection *con) ret = -EAGAIN; goto out_close; } + if (con->nodeid == 0) { + ret = -EINVAL; + goto out_close; + } if (con->rx_page == NULL) { /* @@ -813,11 +598,6 @@ static int receive_from_sock(struct connection *con) cbuf_init(&con->cb, PAGE_CACHE_SIZE); } - /* Only SCTP needs these really */ - memset(&incmsg, 0, sizeof(incmsg)); - msg.msg_control = incmsg; - msg.msg_controllen = sizeof(incmsg); - /* * iov[0] is the bit of the circular buffer between the current end * point (cb.base + cb.len) and the end of the buffer. @@ -843,31 +623,18 @@ static int receive_from_sock(struct connection *con) MSG_DONTWAIT | MSG_NOSIGNAL); if (ret <= 0) goto out_close; - - /* Process SCTP notifications */ - if (msg.msg_flags & MSG_NOTIFICATION) { - msg.msg_control = incmsg; - msg.msg_controllen = sizeof(incmsg); - - process_sctp_notification(con, &msg, - page_address(con->rx_page) + con->cb.base); - mutex_unlock(&con->sock_mutex); - return 0; - } - BUG_ON(con->nodeid == 0); - - if (ret == len) + else if (ret == len) call_again_soon = 1; + cbuf_add(&con->cb, ret); ret = dlm_process_incoming_buffer(con->nodeid, page_address(con->rx_page), con->cb.base, con->cb.len, PAGE_CACHE_SIZE); if (ret == -EBADMSG) { - log_print("lowcomms: addr=%p, base=%u, len=%u, " - "iov_len=%u, iov_base[0]=%p, read=%d", - page_address(con->rx_page), con->cb.base, con->cb.len, - len, iov[0].iov_base, r); + log_print("lowcomms: addr=%p, base=%u, len=%u, read=%d", + page_address(con->rx_page), con->cb.base, + con->cb.len, r); } if (ret < 0) goto out_close; @@ -892,7 +659,7 @@ static int receive_from_sock(struct connection *con) out_close: mutex_unlock(&con->sock_mutex); if (ret != -EAGAIN) { - close_connection(con, false); + close_connection(con, false, true, false); /* Reconnect when there is something to send */ } /* Don't return success if we really got EOF */ @@ -1033,6 +800,120 @@ static int tcp_accept_from_sock(struct connection *con) return result; } +static int sctp_accept_from_sock(struct connection *con) +{ + /* Check that the new node is in the lockspace */ + struct sctp_prim prim; + int nodeid; + int prim_len, ret; + int addr_len; + struct connection *newcon; + struct connection *addcon; + struct socket *newsock; + + mutex_lock(&connections_lock); + if (!dlm_allow_conn) { + mutex_unlock(&connections_lock); + return -1; + } + mutex_unlock(&connections_lock); + + mutex_lock_nested(&con->sock_mutex, 0); + + ret = kernel_accept(con->sock, &newsock, O_NONBLOCK); + if (ret < 0) + goto accept_err; + + memset(&prim, 0, sizeof(struct sctp_prim)); + prim_len = sizeof(struct sctp_prim); + + ret = kernel_getsockopt(newsock, IPPROTO_SCTP, SCTP_PRIMARY_ADDR, + (char *)&prim, &prim_len); + if (ret < 0) { + log_print("getsockopt/sctp_primary_addr failed: %d", ret); + goto accept_err; + } + + make_sockaddr(&prim.ssp_addr, 0, &addr_len); + if (addr_to_nodeid(&prim.ssp_addr, &nodeid)) { + unsigned char *b = (unsigned char *)&prim.ssp_addr; + + log_print("reject connect from unknown addr"); + print_hex_dump_bytes("ss: ", DUMP_PREFIX_NONE, + b, sizeof(struct sockaddr_storage)); + goto accept_err; + } + + newcon = nodeid2con(nodeid, GFP_NOFS); + if (!newcon) { + ret = -ENOMEM; + goto accept_err; + } + + mutex_lock_nested(&newcon->sock_mutex, 1); + + if (newcon->sock) { + struct connection *othercon = newcon->othercon; + + if (!othercon) { + othercon = kmem_cache_zalloc(con_cache, GFP_NOFS); + if (!othercon) { + log_print("failed to allocate incoming socket"); + mutex_unlock(&newcon->sock_mutex); + ret = -ENOMEM; + goto accept_err; + } + othercon->nodeid = nodeid; + othercon->rx_action = receive_from_sock; + mutex_init(&othercon->sock_mutex); + INIT_WORK(&othercon->swork, process_send_sockets); + INIT_WORK(&othercon->rwork, process_recv_sockets); + set_bit(CF_IS_OTHERCON, &othercon->flags); + } + if (!othercon->sock) { + newcon->othercon = othercon; + othercon->sock = newsock; + newsock->sk->sk_user_data = othercon; + add_sock(newsock, othercon); + addcon = othercon; + } else { + printk("Extra connection from node %d attempted\n", nodeid); + ret = -EAGAIN; + mutex_unlock(&newcon->sock_mutex); + goto accept_err; + } + } else { + newsock->sk->sk_user_data = newcon; + newcon->rx_action = receive_from_sock; + add_sock(newsock, newcon); + addcon = newcon; + } + + log_print("connected to %d", nodeid); + + mutex_unlock(&newcon->sock_mutex); + + /* + * Add it to the active queue in case we got data + * between processing the accept adding the socket + * to the read_sockets list + */ + if (!test_and_set_bit(CF_READ_PENDING, &addcon->flags)) + queue_work(recv_workqueue, &addcon->rwork); + mutex_unlock(&con->sock_mutex); + + return 0; + +accept_err: + mutex_unlock(&con->sock_mutex); + if (newsock) + sock_release(newsock); + if (ret != -EAGAIN) + log_print("error accepting connection from node: %d", ret); + + return ret; +} + static void free_entry(struct writequeue_entry *e) { __free_page(e->page); @@ -1057,97 +938,129 @@ static void writequeue_entry_complete(struct writequeue_entry *e, int completed) } } +/* + * sctp_bind_addrs - bind a SCTP socket to all our addresses + */ +static int sctp_bind_addrs(struct connection *con, uint16_t port) +{ + struct sockaddr_storage localaddr; + int i, addr_len, result = 0; + + for (i = 0; i < dlm_local_count; i++) { + memcpy(&localaddr, dlm_local_addr[i], sizeof(localaddr)); + make_sockaddr(&localaddr, port, &addr_len); + + if (!i) + result = kernel_bind(con->sock, + (struct sockaddr *)&localaddr, + addr_len); + else + result = kernel_setsockopt(con->sock, SOL_SCTP, + SCTP_SOCKOPT_BINDX_ADD, + (char *)&localaddr, addr_len); + + if (result < 0) { + log_print("Can't bind to %d addr number %d, %d.\n", + port, i + 1, result); + break; + } + } + return result; +} + /* Initiate an SCTP association. This is a special case of send_to_sock() in that we don't yet have a peeled-off socket for this association, so we use the listening socket and add the primary IP address of the remote node. */ -static void sctp_init_assoc(struct connection *con) +static void sctp_connect_to_sock(struct connection *con) { - struct sockaddr_storage rem_addr; - char outcmsg[CMSG_SPACE(sizeof(struct sctp_sndrcvinfo))]; - struct msghdr outmessage; - struct cmsghdr *cmsg; - struct sctp_sndrcvinfo *sinfo; - struct connection *base_con; - struct writequeue_entry *e; - int len, offset; - int ret; - int addrlen; - struct kvec iov[1]; + struct sockaddr_storage daddr; + int one = 1; + int result; + int addr_len; + struct socket *sock; + + if (con->nodeid == 0) { + log_print("attempt to connect sock 0 foiled"); + return; + } mutex_lock(&con->sock_mutex); - if (test_and_set_bit(CF_INIT_PENDING, &con->flags)) - goto unlock; - if (nodeid_to_addr(con->nodeid, NULL, (struct sockaddr *)&rem_addr, - con->try_new_addr)) { + /* Some odd races can cause double-connects, ignore them */ + if (con->retries++ > MAX_CONNECT_RETRIES) + goto out; + + if (con->sock) { + log_print("node %d already connected.", con->nodeid); + goto out; + } + + memset(&daddr, 0, sizeof(daddr)); + result = nodeid_to_addr(con->nodeid, &daddr, NULL, true); + if (result < 0) { log_print("no address for nodeid %d", con->nodeid); - goto unlock; + goto out; } - base_con = nodeid2con(0, 0); - BUG_ON(base_con == NULL); - make_sockaddr(&rem_addr, dlm_config.ci_tcp_port, &addrlen); + /* Create a socket to communicate with */ + result = sock_create_kern(&init_net, dlm_local_addr[0]->ss_family, + SOCK_STREAM, IPPROTO_SCTP, &sock); + if (result < 0) + goto socket_err; - outmessage.msg_name = &rem_addr; - outmessage.msg_namelen = addrlen; - outmessage.msg_control = outcmsg; - outmessage.msg_controllen = sizeof(outcmsg); - outmessage.msg_flags = MSG_EOR; + sock->sk->sk_user_data = con; + con->rx_action = receive_from_sock; + con->connect_action = sctp_connect_to_sock; + add_sock(sock, con); - spin_lock(&con->writequeue_lock); + /* Bind to all addresses. */ + if (sctp_bind_addrs(con, 0)) + goto bind_err; - if (list_empty(&con->writequeue)) { - spin_unlock(&con->writequeue_lock); - log_print("writequeue empty for nodeid %d", con->nodeid); - goto unlock; - } + make_sockaddr(&daddr, dlm_config.ci_tcp_port, &addr_len); - e = list_first_entry(&con->writequeue, struct writequeue_entry, list); - len = e->len; - offset = e->offset; + log_print("connecting to %d", con->nodeid); - /* Send the first block off the write queue */ - iov[0].iov_base = page_address(e->page)+offset; - iov[0].iov_len = len; - spin_unlock(&con->writequeue_lock); + /* Turn off Nagle's algorithm */ + kernel_setsockopt(sock, SOL_TCP, TCP_NODELAY, (char *)&one, + sizeof(one)); - if (rem_addr.ss_family == AF_INET) { - struct sockaddr_in *sin = (struct sockaddr_in *)&rem_addr; - log_print("Trying to connect to %pI4", &sin->sin_addr.s_addr); - } else { - struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&rem_addr; - log_print("Trying to connect to %pI6", &sin6->sin6_addr); - } + result = sock->ops->connect(sock, (struct sockaddr *)&daddr, addr_len, + O_NONBLOCK); + if (result == -EINPROGRESS) + result = 0; + if (result == 0) + goto out; - cmsg = CMSG_FIRSTHDR(&outmessage); - cmsg->cmsg_level = IPPROTO_SCTP; - cmsg->cmsg_type = SCTP_SNDRCV; - cmsg->cmsg_len = CMSG_LEN(sizeof(struct sctp_sndrcvinfo)); - sinfo = CMSG_DATA(cmsg); - memset(sinfo, 0x00, sizeof(struct sctp_sndrcvinfo)); - sinfo->sinfo_ppid = cpu_to_le32(con->nodeid); - outmessage.msg_controllen = cmsg->cmsg_len; - sinfo->sinfo_flags |= SCTP_ADDR_OVER; - ret = kernel_sendmsg(base_con->sock, &outmessage, iov, 1, len); - if (ret < 0) { - log_print("Send first packet to node %d failed: %d", - con->nodeid, ret); +bind_err: + con->sock = NULL; + sock_release(sock); - /* Try again later */ +socket_err: + /* + * Some errors are fatal and this list might need adjusting. For other + * errors we try again until the max number of retries is reached. + */ + if (result != -EHOSTUNREACH && + result != -ENETUNREACH && + result != -ENETDOWN && + result != -EINVAL && + result != -EPROTONOSUPPORT) { + log_print("connect %d try %d error %d", con->nodeid, + con->retries, result); + mutex_unlock(&con->sock_mutex); + msleep(1000); clear_bit(CF_CONNECT_PENDING, &con->flags); - clear_bit(CF_INIT_PENDING, &con->flags); - } - else { - spin_lock(&con->writequeue_lock); - writequeue_entry_complete(e, ret); - spin_unlock(&con->writequeue_lock); + lowcomms_connect_sock(con); + return; } -unlock: +out: mutex_unlock(&con->sock_mutex); + set_bit(CF_WRITE_PENDING, &con->flags); } /* Connect a new socket to its peer */ @@ -1236,11 +1149,13 @@ static void tcp_connect_to_sock(struct connection *con) con->retries, result); mutex_unlock(&con->sock_mutex); msleep(1000); + clear_bit(CF_CONNECT_PENDING, &con->flags); lowcomms_connect_sock(con); return; } out: mutex_unlock(&con->sock_mutex); + set_bit(CF_WRITE_PENDING, &con->flags); return; } @@ -1325,37 +1240,11 @@ static void init_local(void) } } -/* Bind to an IP address. SCTP allows multiple address so it can do - multi-homing */ -static int add_sctp_bind_addr(struct connection *sctp_con, - struct sockaddr_storage *addr, - int addr_len, int num) -{ - int result = 0; - - if (num == 1) - result = kernel_bind(sctp_con->sock, - (struct sockaddr *) addr, - addr_len); - else - result = kernel_setsockopt(sctp_con->sock, SOL_SCTP, - SCTP_SOCKOPT_BINDX_ADD, - (char *)addr, addr_len); - - if (result < 0) - log_print("Can't bind to port %d addr number %d", - dlm_config.ci_tcp_port, num); - - return result; -} - /* Initialise SCTP socket and bind to all interfaces */ static int sctp_listen_for_all(void) { struct socket *sock = NULL; - struct sockaddr_storage localaddr; - struct sctp_event_subscribe subscribe; - int result = -EINVAL, num = 1, i, addr_len; + int result = -EINVAL; struct connection *con = nodeid2con(0, GFP_NOFS); int bufsize = NEEDED_RMEM; int one = 1; @@ -1366,33 +1255,17 @@ static int sctp_listen_for_all(void) log_print("Using SCTP for communications"); result = sock_create_kern(&init_net, dlm_local_addr[0]->ss_family, - SOCK_SEQPACKET, IPPROTO_SCTP, &sock); + SOCK_STREAM, IPPROTO_SCTP, &sock); if (result < 0) { log_print("Can't create comms socket, check SCTP is loaded"); goto out; } - /* Listen for events */ - memset(&subscribe, 0, sizeof(subscribe)); - subscribe.sctp_data_io_event = 1; - subscribe.sctp_association_event = 1; - subscribe.sctp_send_failure_event = 1; - subscribe.sctp_shutdown_event = 1; - subscribe.sctp_partial_delivery_event = 1; - result = kernel_setsockopt(sock, SOL_SOCKET, SO_RCVBUFFORCE, (char *)&bufsize, sizeof(bufsize)); if (result) log_print("Error increasing buffer space on socket %d", result); - result = kernel_setsockopt(sock, SOL_SCTP, SCTP_EVENTS, - (char *)&subscribe, sizeof(subscribe)); - if (result < 0) { - log_print("Failed to set SCTP_EVENTS on socket: result=%d", - result); - goto create_delsock; - } - result = kernel_setsockopt(sock, SOL_SCTP, SCTP_NODELAY, (char *)&one, sizeof(one)); if (result < 0) @@ -1402,19 +1275,12 @@ static int sctp_listen_for_all(void) sock->sk->sk_user_data = con; con->sock = sock; con->sock->sk->sk_data_ready = lowcomms_data_ready; - con->rx_action = receive_from_sock; - con->connect_action = sctp_init_assoc; + con->rx_action = sctp_accept_from_sock; + con->connect_action = sctp_connect_to_sock; - /* Bind to all interfaces. */ - for (i = 0; i < dlm_local_count; i++) { - memcpy(&localaddr, dlm_local_addr[i], sizeof(localaddr)); - make_sockaddr(&localaddr, dlm_config.ci_tcp_port, &addr_len); - - result = add_sctp_bind_addr(con, &localaddr, addr_len, num); - if (result) - goto create_delsock; - ++num; - } + /* Bind to all addresses. */ + if (sctp_bind_addrs(con, dlm_config.ci_tcp_port)) + goto create_delsock; result = sock->ops->listen(sock, 5); if (result < 0) { @@ -1612,14 +1478,13 @@ static void send_to_sock(struct connection *con) send_error: mutex_unlock(&con->sock_mutex); - close_connection(con, false); + close_connection(con, false, false, true); lowcomms_connect_sock(con); return; out_connect: mutex_unlock(&con->sock_mutex); - if (!test_bit(CF_INIT_PENDING, &con->flags)) - lowcomms_connect_sock(con); + lowcomms_connect_sock(con); } static void clean_one_writequeue(struct connection *con) @@ -1644,15 +1509,9 @@ int dlm_lowcomms_close(int nodeid) log_print("closing connection to node %d", nodeid); con = nodeid2con(nodeid, 0); if (con) { - clear_bit(CF_CONNECT_PENDING, &con->flags); - clear_bit(CF_WRITE_PENDING, &con->flags); set_bit(CF_CLOSE, &con->flags); - if (cancel_work_sync(&con->swork)) - log_print("canceled swork for node %d", nodeid); - if (cancel_work_sync(&con->rwork)) - log_print("canceled rwork for node %d", nodeid); + close_connection(con, true, true, true); clean_one_writequeue(con); - close_connection(con, true); } spin_lock(&dlm_node_addrs_spin); @@ -1685,10 +1544,8 @@ static void process_send_sockets(struct work_struct *work) { struct connection *con = container_of(work, struct connection, swork); - if (test_and_clear_bit(CF_CONNECT_PENDING, &con->flags)) { + if (test_and_clear_bit(CF_CONNECT_PENDING, &con->flags)) con->connect_action(con); - set_bit(CF_WRITE_PENDING, &con->flags); - } if (test_and_clear_bit(CF_WRITE_PENDING, &con->flags)) send_to_sock(con); } @@ -1735,7 +1592,7 @@ static void stop_conn(struct connection *con) static void free_conn(struct connection *con) { - close_connection(con, true); + close_connection(con, true, true, true); if (con->othercon) kmem_cache_free(con_cache, con->othercon); hlist_del(&con->list); @@ -1806,7 +1663,7 @@ int dlm_lowcomms_start(void) dlm_allow_conn = 0; con = nodeid2con(0,0); if (con) { - close_connection(con, false); + close_connection(con, false, true, true); kmem_cache_free(con_cache, con); } fail_destroy: diff --git a/fs/dlm/user.c b/fs/dlm/user.c index 75ecc0d3bc85ad..173b3873a4f4ee 100644 --- a/fs/dlm/user.c +++ b/fs/dlm/user.c @@ -782,6 +782,7 @@ static ssize_t device_read(struct file *file, char __user *buf, size_t count, DECLARE_WAITQUEUE(wait, current); struct dlm_callback cb; int rv, resid, copy_lvb = 0; + int old_mode, new_mode; if (count == sizeof(struct dlm_device_version)) { rv = copy_version_to_user(buf, count); @@ -838,6 +839,9 @@ static ssize_t device_read(struct file *file, char __user *buf, size_t count, lkb = list_entry(proc->asts.next, struct dlm_lkb, lkb_cb_list); + /* rem_lkb_callback sets a new lkb_last_cast */ + old_mode = lkb->lkb_last_cast.mode; + rv = dlm_rem_lkb_callback(lkb->lkb_resource->res_ls, lkb, &cb, &resid); if (rv < 0) { /* this shouldn't happen; lkb should have been removed from @@ -861,9 +865,6 @@ static ssize_t device_read(struct file *file, char __user *buf, size_t count, } if (cb.flags & DLM_CB_CAST) { - int old_mode, new_mode; - - old_mode = lkb->lkb_last_cast.mode; new_mode = cb.mode; if (!cb.sb_status && lkb->lkb_lksb->sb_lvbptr && diff --git a/fs/ext2/ialloc.c b/fs/ext2/ialloc.c index 5c04a0ddea804e..efe5fb21c53321 100644 --- a/fs/ext2/ialloc.c +++ b/fs/ext2/ialloc.c @@ -577,7 +577,10 @@ struct inode *ext2_new_inode(struct inode *dir, umode_t mode, goto fail; } - dquot_initialize(inode); + err = dquot_initialize(inode); + if (err) + goto fail_drop; + err = dquot_alloc_inode(inode); if (err) goto fail_drop; diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c index 5c09776d347fc3..a3a404c5df2edf 100644 --- a/fs/ext2/inode.c +++ b/fs/ext2/inode.c @@ -1552,8 +1552,11 @@ int ext2_setattr(struct dentry *dentry, struct iattr *iattr) if (error) return error; - if (is_quota_modification(inode, iattr)) - dquot_initialize(inode); + if (is_quota_modification(inode, iattr)) { + error = dquot_initialize(inode); + if (error) + return error; + } if ((iattr->ia_valid & ATTR_UID && !uid_eq(iattr->ia_uid, inode->i_uid)) || (iattr->ia_valid & ATTR_GID && !gid_eq(iattr->ia_gid, inode->i_gid))) { error = dquot_transfer(inode, iattr); diff --git a/fs/ext2/namei.c b/fs/ext2/namei.c index 13ec54a99c962a..b4841e3066a5bb 100644 --- a/fs/ext2/namei.c +++ b/fs/ext2/namei.c @@ -96,8 +96,11 @@ struct dentry *ext2_get_parent(struct dentry *child) static int ext2_create (struct inode * dir, struct dentry * dentry, umode_t mode, bool excl) { struct inode *inode; + int err; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + return err; inode = ext2_new_inode(dir, mode, &dentry->d_name); if (IS_ERR(inode)) @@ -143,7 +146,9 @@ static int ext2_mknod (struct inode * dir, struct dentry *dentry, umode_t mode, if (!new_valid_dev(rdev)) return -EINVAL; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + return err; inode = ext2_new_inode (dir, mode, &dentry->d_name); err = PTR_ERR(inode); @@ -169,7 +174,9 @@ static int ext2_symlink (struct inode * dir, struct dentry * dentry, if (l > sb->s_blocksize) goto out; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + goto out; inode = ext2_new_inode (dir, S_IFLNK | S_IRWXUGO, &dentry->d_name); err = PTR_ERR(inode); @@ -212,7 +219,9 @@ static int ext2_link (struct dentry * old_dentry, struct inode * dir, struct inode *inode = d_inode(old_dentry); int err; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + return err; inode->i_ctime = CURRENT_TIME_SEC; inode_inc_link_count(inode); @@ -233,7 +242,9 @@ static int ext2_mkdir(struct inode * dir, struct dentry * dentry, umode_t mode) struct inode * inode; int err; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + return err; inode_inc_link_count(dir); @@ -279,13 +290,17 @@ static int ext2_unlink(struct inode * dir, struct dentry *dentry) struct inode * inode = d_inode(dentry); struct ext2_dir_entry_2 * de; struct page * page; - int err = -ENOENT; + int err; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + goto out; de = ext2_find_entry (dir, &dentry->d_name, &page); - if (!de) + if (!de) { + err = -ENOENT; goto out; + } err = ext2_delete_entry (de, page); if (err) @@ -323,14 +338,21 @@ static int ext2_rename (struct inode * old_dir, struct dentry * old_dentry, struct ext2_dir_entry_2 * dir_de = NULL; struct page * old_page; struct ext2_dir_entry_2 * old_de; - int err = -ENOENT; + int err; + + err = dquot_initialize(old_dir); + if (err) + goto out; - dquot_initialize(old_dir); - dquot_initialize(new_dir); + err = dquot_initialize(new_dir); + if (err) + goto out; old_de = ext2_find_entry (old_dir, &old_dentry->d_name, &old_page); - if (!old_de) + if (!old_de) { + err = -ENOENT; goto out; + } if (S_ISDIR(old_inode->i_mode)) { err = -EIO; diff --git a/fs/ext3/Kconfig b/fs/ext3/Kconfig deleted file mode 100644 index e8c6ba0e4a3ea6..00000000000000 --- a/fs/ext3/Kconfig +++ /dev/null @@ -1,89 +0,0 @@ -config EXT3_FS - tristate "Ext3 journalling file system support" - select JBD - help - This is the journalling version of the Second extended file system - (often called ext3), the de facto standard Linux file system - (method to organize files on a storage device) for hard disks. - - The journalling code included in this driver means you do not have - to run e2fsck (file system checker) on your file systems after a - crash. The journal keeps track of any changes that were being made - at the time the system crashed, and can ensure that your file system - is consistent without the need for a lengthy check. - - Other than adding the journal to the file system, the on-disk format - of ext3 is identical to ext2. It is possible to freely switch - between using the ext3 driver and the ext2 driver, as long as the - file system has been cleanly unmounted, or e2fsck is run on the file - system. - - To add a journal on an existing ext2 file system or change the - behavior of ext3 file systems, you can use the tune2fs utility ("man - tune2fs"). To modify attributes of files and directories on ext3 - file systems, use chattr ("man chattr"). You need to be using - e2fsprogs version 1.20 or later in order to create ext3 journals - (available at ). - - To compile this file system support as a module, choose M here: the - module will be called ext3. - -config EXT3_DEFAULTS_TO_ORDERED - bool "Default to 'data=ordered' in ext3" - depends on EXT3_FS - default y - help - The journal mode options for ext3 have different tradeoffs - between when data is guaranteed to be on disk and - performance. The use of "data=writeback" can cause - unwritten data to appear in files after an system crash or - power failure, which can be a security issue. However, - "data=ordered" mode can also result in major performance - problems, including seconds-long delays before an fsync() - call returns. For details, see: - - http://ext4.wiki.kernel.org/index.php/Ext3_data_mode_tradeoffs - - If you have been historically happy with ext3's performance, - data=ordered mode will be a safe choice and you should - answer 'y' here. If you understand the reliability and data - privacy issues of data=writeback and are willing to make - that trade off, answer 'n'. - -config EXT3_FS_XATTR - bool "Ext3 extended attributes" - depends on EXT3_FS - default y - help - Extended attributes are name:value pairs associated with inodes by - the kernel or by users (see the attr(5) manual page, or visit - for details). - - If unsure, say N. - - You need this for POSIX ACL support on ext3. - -config EXT3_FS_POSIX_ACL - bool "Ext3 POSIX Access Control Lists" - depends on EXT3_FS_XATTR - select FS_POSIX_ACL - help - Posix Access Control Lists (ACLs) support permissions for users and - groups beyond the owner/group/world scheme. - - To learn more about Access Control Lists, visit the Posix ACLs for - Linux website . - - If you don't know what Access Control Lists are, say N - -config EXT3_FS_SECURITY - bool "Ext3 Security Labels" - depends on EXT3_FS_XATTR - help - Security labels support alternative access control models - implemented by security modules like SELinux. This option - enables an extended attribute handler for file security - labels in the ext3 filesystem. - - If you are not using a security module that requires using - extended attributes for file security labels, say N. diff --git a/fs/ext3/Makefile b/fs/ext3/Makefile deleted file mode 100644 index e77766a8b3f07a..00000000000000 --- a/fs/ext3/Makefile +++ /dev/null @@ -1,12 +0,0 @@ -# -# Makefile for the linux ext3-filesystem routines. -# - -obj-$(CONFIG_EXT3_FS) += ext3.o - -ext3-y := balloc.o bitmap.o dir.o file.o fsync.o ialloc.o inode.o \ - ioctl.o namei.o super.o symlink.o hash.o resize.o ext3_jbd.o - -ext3-$(CONFIG_EXT3_FS_XATTR) += xattr.o xattr_user.o xattr_trusted.o -ext3-$(CONFIG_EXT3_FS_POSIX_ACL) += acl.o -ext3-$(CONFIG_EXT3_FS_SECURITY) += xattr_security.o diff --git a/fs/ext3/acl.c b/fs/ext3/acl.c deleted file mode 100644 index 8bbaf5bcf9820c..00000000000000 --- a/fs/ext3/acl.c +++ /dev/null @@ -1,281 +0,0 @@ -/* - * linux/fs/ext3/acl.c - * - * Copyright (C) 2001-2003 Andreas Gruenbacher, - */ - -#include "ext3.h" -#include "xattr.h" -#include "acl.h" - -/* - * Convert from filesystem to in-memory representation. - */ -static struct posix_acl * -ext3_acl_from_disk(const void *value, size_t size) -{ - const char *end = (char *)value + size; - int n, count; - struct posix_acl *acl; - - if (!value) - return NULL; - if (size < sizeof(ext3_acl_header)) - return ERR_PTR(-EINVAL); - if (((ext3_acl_header *)value)->a_version != - cpu_to_le32(EXT3_ACL_VERSION)) - return ERR_PTR(-EINVAL); - value = (char *)value + sizeof(ext3_acl_header); - count = ext3_acl_count(size); - if (count < 0) - return ERR_PTR(-EINVAL); - if (count == 0) - return NULL; - acl = posix_acl_alloc(count, GFP_NOFS); - if (!acl) - return ERR_PTR(-ENOMEM); - for (n=0; n < count; n++) { - ext3_acl_entry *entry = - (ext3_acl_entry *)value; - if ((char *)value + sizeof(ext3_acl_entry_short) > end) - goto fail; - acl->a_entries[n].e_tag = le16_to_cpu(entry->e_tag); - acl->a_entries[n].e_perm = le16_to_cpu(entry->e_perm); - switch(acl->a_entries[n].e_tag) { - case ACL_USER_OBJ: - case ACL_GROUP_OBJ: - case ACL_MASK: - case ACL_OTHER: - value = (char *)value + - sizeof(ext3_acl_entry_short); - break; - - case ACL_USER: - value = (char *)value + sizeof(ext3_acl_entry); - if ((char *)value > end) - goto fail; - acl->a_entries[n].e_uid = - make_kuid(&init_user_ns, - le32_to_cpu(entry->e_id)); - break; - case ACL_GROUP: - value = (char *)value + sizeof(ext3_acl_entry); - if ((char *)value > end) - goto fail; - acl->a_entries[n].e_gid = - make_kgid(&init_user_ns, - le32_to_cpu(entry->e_id)); - break; - - default: - goto fail; - } - } - if (value != end) - goto fail; - return acl; - -fail: - posix_acl_release(acl); - return ERR_PTR(-EINVAL); -} - -/* - * Convert from in-memory to filesystem representation. - */ -static void * -ext3_acl_to_disk(const struct posix_acl *acl, size_t *size) -{ - ext3_acl_header *ext_acl; - char *e; - size_t n; - - *size = ext3_acl_size(acl->a_count); - ext_acl = kmalloc(sizeof(ext3_acl_header) + acl->a_count * - sizeof(ext3_acl_entry), GFP_NOFS); - if (!ext_acl) - return ERR_PTR(-ENOMEM); - ext_acl->a_version = cpu_to_le32(EXT3_ACL_VERSION); - e = (char *)ext_acl + sizeof(ext3_acl_header); - for (n=0; n < acl->a_count; n++) { - const struct posix_acl_entry *acl_e = &acl->a_entries[n]; - ext3_acl_entry *entry = (ext3_acl_entry *)e; - entry->e_tag = cpu_to_le16(acl_e->e_tag); - entry->e_perm = cpu_to_le16(acl_e->e_perm); - switch(acl_e->e_tag) { - case ACL_USER: - entry->e_id = cpu_to_le32( - from_kuid(&init_user_ns, acl_e->e_uid)); - e += sizeof(ext3_acl_entry); - break; - case ACL_GROUP: - entry->e_id = cpu_to_le32( - from_kgid(&init_user_ns, acl_e->e_gid)); - e += sizeof(ext3_acl_entry); - break; - - case ACL_USER_OBJ: - case ACL_GROUP_OBJ: - case ACL_MASK: - case ACL_OTHER: - e += sizeof(ext3_acl_entry_short); - break; - - default: - goto fail; - } - } - return (char *)ext_acl; - -fail: - kfree(ext_acl); - return ERR_PTR(-EINVAL); -} - -/* - * Inode operation get_posix_acl(). - * - * inode->i_mutex: don't care - */ -struct posix_acl * -ext3_get_acl(struct inode *inode, int type) -{ - int name_index; - char *value = NULL; - struct posix_acl *acl; - int retval; - - switch (type) { - case ACL_TYPE_ACCESS: - name_index = EXT3_XATTR_INDEX_POSIX_ACL_ACCESS; - break; - case ACL_TYPE_DEFAULT: - name_index = EXT3_XATTR_INDEX_POSIX_ACL_DEFAULT; - break; - default: - BUG(); - } - - retval = ext3_xattr_get(inode, name_index, "", NULL, 0); - if (retval > 0) { - value = kmalloc(retval, GFP_NOFS); - if (!value) - return ERR_PTR(-ENOMEM); - retval = ext3_xattr_get(inode, name_index, "", value, retval); - } - if (retval > 0) - acl = ext3_acl_from_disk(value, retval); - else if (retval == -ENODATA || retval == -ENOSYS) - acl = NULL; - else - acl = ERR_PTR(retval); - kfree(value); - - if (!IS_ERR(acl)) - set_cached_acl(inode, type, acl); - - return acl; -} - -/* - * Set the access or default ACL of an inode. - * - * inode->i_mutex: down unless called from ext3_new_inode - */ -static int -__ext3_set_acl(handle_t *handle, struct inode *inode, int type, - struct posix_acl *acl) -{ - int name_index; - void *value = NULL; - size_t size = 0; - int error; - - switch(type) { - case ACL_TYPE_ACCESS: - name_index = EXT3_XATTR_INDEX_POSIX_ACL_ACCESS; - if (acl) { - error = posix_acl_equiv_mode(acl, &inode->i_mode); - if (error < 0) - return error; - else { - inode->i_ctime = CURRENT_TIME_SEC; - ext3_mark_inode_dirty(handle, inode); - if (error == 0) - acl = NULL; - } - } - break; - - case ACL_TYPE_DEFAULT: - name_index = EXT3_XATTR_INDEX_POSIX_ACL_DEFAULT; - if (!S_ISDIR(inode->i_mode)) - return acl ? -EACCES : 0; - break; - - default: - return -EINVAL; - } - if (acl) { - value = ext3_acl_to_disk(acl, &size); - if (IS_ERR(value)) - return (int)PTR_ERR(value); - } - - error = ext3_xattr_set_handle(handle, inode, name_index, "", - value, size, 0); - - kfree(value); - - if (!error) - set_cached_acl(inode, type, acl); - - return error; -} - -int -ext3_set_acl(struct inode *inode, struct posix_acl *acl, int type) -{ - handle_t *handle; - int error, retries = 0; - -retry: - handle = ext3_journal_start(inode, EXT3_DATA_TRANS_BLOCKS(inode->i_sb)); - if (IS_ERR(handle)) - return PTR_ERR(handle); - error = __ext3_set_acl(handle, inode, type, acl); - ext3_journal_stop(handle); - if (error == -ENOSPC && ext3_should_retry_alloc(inode->i_sb, &retries)) - goto retry; - return error; -} - -/* - * Initialize the ACLs of a new inode. Called from ext3_new_inode. - * - * dir->i_mutex: down - * inode->i_mutex: up (access to inode is still exclusive) - */ -int -ext3_init_acl(handle_t *handle, struct inode *inode, struct inode *dir) -{ - struct posix_acl *default_acl, *acl; - int error; - - error = posix_acl_create(dir, &inode->i_mode, &default_acl, &acl); - if (error) - return error; - - if (default_acl) { - error = __ext3_set_acl(handle, inode, ACL_TYPE_DEFAULT, - default_acl); - posix_acl_release(default_acl); - } - if (acl) { - if (!error) - error = __ext3_set_acl(handle, inode, ACL_TYPE_ACCESS, - acl); - posix_acl_release(acl); - } - return error; -} diff --git a/fs/ext3/acl.h b/fs/ext3/acl.h deleted file mode 100644 index ea1c69edab9e15..00000000000000 --- a/fs/ext3/acl.h +++ /dev/null @@ -1,72 +0,0 @@ -/* - File: fs/ext3/acl.h - - (C) 2001 Andreas Gruenbacher, -*/ - -#include - -#define EXT3_ACL_VERSION 0x0001 - -typedef struct { - __le16 e_tag; - __le16 e_perm; - __le32 e_id; -} ext3_acl_entry; - -typedef struct { - __le16 e_tag; - __le16 e_perm; -} ext3_acl_entry_short; - -typedef struct { - __le32 a_version; -} ext3_acl_header; - -static inline size_t ext3_acl_size(int count) -{ - if (count <= 4) { - return sizeof(ext3_acl_header) + - count * sizeof(ext3_acl_entry_short); - } else { - return sizeof(ext3_acl_header) + - 4 * sizeof(ext3_acl_entry_short) + - (count - 4) * sizeof(ext3_acl_entry); - } -} - -static inline int ext3_acl_count(size_t size) -{ - ssize_t s; - size -= sizeof(ext3_acl_header); - s = size - 4 * sizeof(ext3_acl_entry_short); - if (s < 0) { - if (size % sizeof(ext3_acl_entry_short)) - return -1; - return size / sizeof(ext3_acl_entry_short); - } else { - if (s % sizeof(ext3_acl_entry)) - return -1; - return s / sizeof(ext3_acl_entry) + 4; - } -} - -#ifdef CONFIG_EXT3_FS_POSIX_ACL - -/* acl.c */ -extern struct posix_acl *ext3_get_acl(struct inode *inode, int type); -extern int ext3_set_acl(struct inode *inode, struct posix_acl *acl, int type); -extern int ext3_init_acl (handle_t *, struct inode *, struct inode *); - -#else /* CONFIG_EXT3_FS_POSIX_ACL */ -#include -#define ext3_get_acl NULL -#define ext3_set_acl NULL - -static inline int -ext3_init_acl(handle_t *handle, struct inode *inode, struct inode *dir) -{ - return 0; -} -#endif /* CONFIG_EXT3_FS_POSIX_ACL */ - diff --git a/fs/ext3/balloc.c b/fs/ext3/balloc.c deleted file mode 100644 index 158b5d4ce0670f..00000000000000 --- a/fs/ext3/balloc.c +++ /dev/null @@ -1,2158 +0,0 @@ -/* - * linux/fs/ext3/balloc.c - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * - * Enhanced block allocation by Stephen Tweedie (sct@redhat.com), 1993 - * Big-endian to little-endian byte-swapping/bitmaps by - * David S. Miller (davem@caip.rutgers.edu), 1995 - */ - -#include -#include -#include "ext3.h" - -/* - * balloc.c contains the blocks allocation and deallocation routines - */ - -/* - * The free blocks are managed by bitmaps. A file system contains several - * blocks groups. Each group contains 1 bitmap block for blocks, 1 bitmap - * block for inodes, N blocks for the inode table and data blocks. - * - * The file system contains group descriptors which are located after the - * super block. Each descriptor contains the number of the bitmap block and - * the free blocks count in the block. The descriptors are loaded in memory - * when a file system is mounted (see ext3_fill_super). - */ - - -#define in_range(b, first, len) ((b) >= (first) && (b) <= (first) + (len) - 1) - -/* - * Calculate the block group number and offset, given a block number - */ -static void ext3_get_group_no_and_offset(struct super_block *sb, - ext3_fsblk_t blocknr, unsigned long *blockgrpp, ext3_grpblk_t *offsetp) -{ - struct ext3_super_block *es = EXT3_SB(sb)->s_es; - - blocknr = blocknr - le32_to_cpu(es->s_first_data_block); - if (offsetp) - *offsetp = blocknr % EXT3_BLOCKS_PER_GROUP(sb); - if (blockgrpp) - *blockgrpp = blocknr / EXT3_BLOCKS_PER_GROUP(sb); -} - -/** - * ext3_get_group_desc() -- load group descriptor from disk - * @sb: super block - * @block_group: given block group - * @bh: pointer to the buffer head to store the block - * group descriptor - */ -struct ext3_group_desc * ext3_get_group_desc(struct super_block * sb, - unsigned int block_group, - struct buffer_head ** bh) -{ - unsigned long group_desc; - unsigned long offset; - struct ext3_group_desc * desc; - struct ext3_sb_info *sbi = EXT3_SB(sb); - - if (block_group >= sbi->s_groups_count) { - ext3_error (sb, "ext3_get_group_desc", - "block_group >= groups_count - " - "block_group = %d, groups_count = %lu", - block_group, sbi->s_groups_count); - - return NULL; - } - smp_rmb(); - - group_desc = block_group >> EXT3_DESC_PER_BLOCK_BITS(sb); - offset = block_group & (EXT3_DESC_PER_BLOCK(sb) - 1); - if (!sbi->s_group_desc[group_desc]) { - ext3_error (sb, "ext3_get_group_desc", - "Group descriptor not loaded - " - "block_group = %d, group_desc = %lu, desc = %lu", - block_group, group_desc, offset); - return NULL; - } - - desc = (struct ext3_group_desc *) sbi->s_group_desc[group_desc]->b_data; - if (bh) - *bh = sbi->s_group_desc[group_desc]; - return desc + offset; -} - -static int ext3_valid_block_bitmap(struct super_block *sb, - struct ext3_group_desc *desc, - unsigned int block_group, - struct buffer_head *bh) -{ - ext3_grpblk_t offset; - ext3_grpblk_t next_zero_bit; - ext3_fsblk_t bitmap_blk; - ext3_fsblk_t group_first_block; - - group_first_block = ext3_group_first_block_no(sb, block_group); - - /* check whether block bitmap block number is set */ - bitmap_blk = le32_to_cpu(desc->bg_block_bitmap); - offset = bitmap_blk - group_first_block; - if (!ext3_test_bit(offset, bh->b_data)) - /* bad block bitmap */ - goto err_out; - - /* check whether the inode bitmap block number is set */ - bitmap_blk = le32_to_cpu(desc->bg_inode_bitmap); - offset = bitmap_blk - group_first_block; - if (!ext3_test_bit(offset, bh->b_data)) - /* bad block bitmap */ - goto err_out; - - /* check whether the inode table block number is set */ - bitmap_blk = le32_to_cpu(desc->bg_inode_table); - offset = bitmap_blk - group_first_block; - next_zero_bit = ext3_find_next_zero_bit(bh->b_data, - offset + EXT3_SB(sb)->s_itb_per_group, - offset); - if (next_zero_bit >= offset + EXT3_SB(sb)->s_itb_per_group) - /* good bitmap for inode tables */ - return 1; - -err_out: - ext3_error(sb, __func__, - "Invalid block bitmap - " - "block_group = %d, block = %lu", - block_group, bitmap_blk); - return 0; -} - -/** - * read_block_bitmap() - * @sb: super block - * @block_group: given block group - * - * Read the bitmap for a given block_group,and validate the - * bits for block/inode/inode tables are set in the bitmaps - * - * Return buffer_head on success or NULL in case of failure. - */ -static struct buffer_head * -read_block_bitmap(struct super_block *sb, unsigned int block_group) -{ - struct ext3_group_desc * desc; - struct buffer_head * bh = NULL; - ext3_fsblk_t bitmap_blk; - - desc = ext3_get_group_desc(sb, block_group, NULL); - if (!desc) - return NULL; - trace_ext3_read_block_bitmap(sb, block_group); - bitmap_blk = le32_to_cpu(desc->bg_block_bitmap); - bh = sb_getblk(sb, bitmap_blk); - if (unlikely(!bh)) { - ext3_error(sb, __func__, - "Cannot read block bitmap - " - "block_group = %d, block_bitmap = %u", - block_group, le32_to_cpu(desc->bg_block_bitmap)); - return NULL; - } - if (likely(bh_uptodate_or_lock(bh))) - return bh; - - if (bh_submit_read(bh) < 0) { - brelse(bh); - ext3_error(sb, __func__, - "Cannot read block bitmap - " - "block_group = %d, block_bitmap = %u", - block_group, le32_to_cpu(desc->bg_block_bitmap)); - return NULL; - } - ext3_valid_block_bitmap(sb, desc, block_group, bh); - /* - * file system mounted not to panic on error, continue with corrupt - * bitmap - */ - return bh; -} -/* - * The reservation window structure operations - * -------------------------------------------- - * Operations include: - * dump, find, add, remove, is_empty, find_next_reservable_window, etc. - * - * We use a red-black tree to represent per-filesystem reservation - * windows. - * - */ - -/** - * __rsv_window_dump() -- Dump the filesystem block allocation reservation map - * @rb_root: root of per-filesystem reservation rb tree - * @verbose: verbose mode - * @fn: function which wishes to dump the reservation map - * - * If verbose is turned on, it will print the whole block reservation - * windows(start, end). Otherwise, it will only print out the "bad" windows, - * those windows that overlap with their immediate neighbors. - */ -#if 1 -static void __rsv_window_dump(struct rb_root *root, int verbose, - const char *fn) -{ - struct rb_node *n; - struct ext3_reserve_window_node *rsv, *prev; - int bad; - -restart: - n = rb_first(root); - bad = 0; - prev = NULL; - - printk("Block Allocation Reservation Windows Map (%s):\n", fn); - while (n) { - rsv = rb_entry(n, struct ext3_reserve_window_node, rsv_node); - if (verbose) - printk("reservation window 0x%p " - "start: %lu, end: %lu\n", - rsv, rsv->rsv_start, rsv->rsv_end); - if (rsv->rsv_start && rsv->rsv_start >= rsv->rsv_end) { - printk("Bad reservation %p (start >= end)\n", - rsv); - bad = 1; - } - if (prev && prev->rsv_end >= rsv->rsv_start) { - printk("Bad reservation %p (prev->end >= start)\n", - rsv); - bad = 1; - } - if (bad) { - if (!verbose) { - printk("Restarting reservation walk in verbose mode\n"); - verbose = 1; - goto restart; - } - } - n = rb_next(n); - prev = rsv; - } - printk("Window map complete.\n"); - BUG_ON(bad); -} -#define rsv_window_dump(root, verbose) \ - __rsv_window_dump((root), (verbose), __func__) -#else -#define rsv_window_dump(root, verbose) do {} while (0) -#endif - -/** - * goal_in_my_reservation() - * @rsv: inode's reservation window - * @grp_goal: given goal block relative to the allocation block group - * @group: the current allocation block group - * @sb: filesystem super block - * - * Test if the given goal block (group relative) is within the file's - * own block reservation window range. - * - * If the reservation window is outside the goal allocation group, return 0; - * grp_goal (given goal block) could be -1, which means no specific - * goal block. In this case, always return 1. - * If the goal block is within the reservation window, return 1; - * otherwise, return 0; - */ -static int -goal_in_my_reservation(struct ext3_reserve_window *rsv, ext3_grpblk_t grp_goal, - unsigned int group, struct super_block * sb) -{ - ext3_fsblk_t group_first_block, group_last_block; - - group_first_block = ext3_group_first_block_no(sb, group); - group_last_block = group_first_block + (EXT3_BLOCKS_PER_GROUP(sb) - 1); - - if ((rsv->_rsv_start > group_last_block) || - (rsv->_rsv_end < group_first_block)) - return 0; - if ((grp_goal >= 0) && ((grp_goal + group_first_block < rsv->_rsv_start) - || (grp_goal + group_first_block > rsv->_rsv_end))) - return 0; - return 1; -} - -/** - * search_reserve_window() - * @rb_root: root of reservation tree - * @goal: target allocation block - * - * Find the reserved window which includes the goal, or the previous one - * if the goal is not in any window. - * Returns NULL if there are no windows or if all windows start after the goal. - */ -static struct ext3_reserve_window_node * -search_reserve_window(struct rb_root *root, ext3_fsblk_t goal) -{ - struct rb_node *n = root->rb_node; - struct ext3_reserve_window_node *rsv; - - if (!n) - return NULL; - - do { - rsv = rb_entry(n, struct ext3_reserve_window_node, rsv_node); - - if (goal < rsv->rsv_start) - n = n->rb_left; - else if (goal > rsv->rsv_end) - n = n->rb_right; - else - return rsv; - } while (n); - /* - * We've fallen off the end of the tree: the goal wasn't inside - * any particular node. OK, the previous node must be to one - * side of the interval containing the goal. If it's the RHS, - * we need to back up one. - */ - if (rsv->rsv_start > goal) { - n = rb_prev(&rsv->rsv_node); - rsv = rb_entry(n, struct ext3_reserve_window_node, rsv_node); - } - return rsv; -} - -/** - * ext3_rsv_window_add() -- Insert a window to the block reservation rb tree. - * @sb: super block - * @rsv: reservation window to add - * - * Must be called with rsv_lock hold. - */ -void ext3_rsv_window_add(struct super_block *sb, - struct ext3_reserve_window_node *rsv) -{ - struct rb_root *root = &EXT3_SB(sb)->s_rsv_window_root; - struct rb_node *node = &rsv->rsv_node; - ext3_fsblk_t start = rsv->rsv_start; - - struct rb_node ** p = &root->rb_node; - struct rb_node * parent = NULL; - struct ext3_reserve_window_node *this; - - trace_ext3_rsv_window_add(sb, rsv); - while (*p) - { - parent = *p; - this = rb_entry(parent, struct ext3_reserve_window_node, rsv_node); - - if (start < this->rsv_start) - p = &(*p)->rb_left; - else if (start > this->rsv_end) - p = &(*p)->rb_right; - else { - rsv_window_dump(root, 1); - BUG(); - } - } - - rb_link_node(node, parent, p); - rb_insert_color(node, root); -} - -/** - * ext3_rsv_window_remove() -- unlink a window from the reservation rb tree - * @sb: super block - * @rsv: reservation window to remove - * - * Mark the block reservation window as not allocated, and unlink it - * from the filesystem reservation window rb tree. Must be called with - * rsv_lock hold. - */ -static void rsv_window_remove(struct super_block *sb, - struct ext3_reserve_window_node *rsv) -{ - rsv->rsv_start = EXT3_RESERVE_WINDOW_NOT_ALLOCATED; - rsv->rsv_end = EXT3_RESERVE_WINDOW_NOT_ALLOCATED; - rsv->rsv_alloc_hit = 0; - rb_erase(&rsv->rsv_node, &EXT3_SB(sb)->s_rsv_window_root); -} - -/* - * rsv_is_empty() -- Check if the reservation window is allocated. - * @rsv: given reservation window to check - * - * returns 1 if the end block is EXT3_RESERVE_WINDOW_NOT_ALLOCATED. - */ -static inline int rsv_is_empty(struct ext3_reserve_window *rsv) -{ - /* a valid reservation end block could not be 0 */ - return rsv->_rsv_end == EXT3_RESERVE_WINDOW_NOT_ALLOCATED; -} - -/** - * ext3_init_block_alloc_info() - * @inode: file inode structure - * - * Allocate and initialize the reservation window structure, and - * link the window to the ext3 inode structure at last - * - * The reservation window structure is only dynamically allocated - * and linked to ext3 inode the first time the open file - * needs a new block. So, before every ext3_new_block(s) call, for - * regular files, we should check whether the reservation window - * structure exists or not. In the latter case, this function is called. - * Fail to do so will result in block reservation being turned off for that - * open file. - * - * This function is called from ext3_get_blocks_handle(), also called - * when setting the reservation window size through ioctl before the file - * is open for write (needs block allocation). - * - * Needs truncate_mutex protection prior to call this function. - */ -void ext3_init_block_alloc_info(struct inode *inode) -{ - struct ext3_inode_info *ei = EXT3_I(inode); - struct ext3_block_alloc_info *block_i; - struct super_block *sb = inode->i_sb; - - block_i = kmalloc(sizeof(*block_i), GFP_NOFS); - if (block_i) { - struct ext3_reserve_window_node *rsv = &block_i->rsv_window_node; - - rsv->rsv_start = EXT3_RESERVE_WINDOW_NOT_ALLOCATED; - rsv->rsv_end = EXT3_RESERVE_WINDOW_NOT_ALLOCATED; - - /* - * if filesystem is mounted with NORESERVATION, the goal - * reservation window size is set to zero to indicate - * block reservation is off - */ - if (!test_opt(sb, RESERVATION)) - rsv->rsv_goal_size = 0; - else - rsv->rsv_goal_size = EXT3_DEFAULT_RESERVE_BLOCKS; - rsv->rsv_alloc_hit = 0; - block_i->last_alloc_logical_block = 0; - block_i->last_alloc_physical_block = 0; - } - ei->i_block_alloc_info = block_i; -} - -/** - * ext3_discard_reservation() - * @inode: inode - * - * Discard(free) block reservation window on last file close, or truncate - * or at last iput(). - * - * It is being called in three cases: - * ext3_release_file(): last writer close the file - * ext3_clear_inode(): last iput(), when nobody link to this file. - * ext3_truncate(): when the block indirect map is about to change. - * - */ -void ext3_discard_reservation(struct inode *inode) -{ - struct ext3_inode_info *ei = EXT3_I(inode); - struct ext3_block_alloc_info *block_i = ei->i_block_alloc_info; - struct ext3_reserve_window_node *rsv; - spinlock_t *rsv_lock = &EXT3_SB(inode->i_sb)->s_rsv_window_lock; - - if (!block_i) - return; - - rsv = &block_i->rsv_window_node; - if (!rsv_is_empty(&rsv->rsv_window)) { - spin_lock(rsv_lock); - if (!rsv_is_empty(&rsv->rsv_window)) { - trace_ext3_discard_reservation(inode, rsv); - rsv_window_remove(inode->i_sb, rsv); - } - spin_unlock(rsv_lock); - } -} - -/** - * ext3_free_blocks_sb() -- Free given blocks and update quota - * @handle: handle to this transaction - * @sb: super block - * @block: start physical block to free - * @count: number of blocks to free - * @pdquot_freed_blocks: pointer to quota - */ -void ext3_free_blocks_sb(handle_t *handle, struct super_block *sb, - ext3_fsblk_t block, unsigned long count, - unsigned long *pdquot_freed_blocks) -{ - struct buffer_head *bitmap_bh = NULL; - struct buffer_head *gd_bh; - unsigned long block_group; - ext3_grpblk_t bit; - unsigned long i; - unsigned long overflow; - struct ext3_group_desc * desc; - struct ext3_super_block * es; - struct ext3_sb_info *sbi; - int err = 0, ret; - ext3_grpblk_t group_freed; - - *pdquot_freed_blocks = 0; - sbi = EXT3_SB(sb); - es = sbi->s_es; - if (block < le32_to_cpu(es->s_first_data_block) || - block + count < block || - block + count > le32_to_cpu(es->s_blocks_count)) { - ext3_error (sb, "ext3_free_blocks", - "Freeing blocks not in datazone - " - "block = "E3FSBLK", count = %lu", block, count); - goto error_return; - } - - ext3_debug ("freeing block(s) %lu-%lu\n", block, block + count - 1); - -do_more: - overflow = 0; - block_group = (block - le32_to_cpu(es->s_first_data_block)) / - EXT3_BLOCKS_PER_GROUP(sb); - bit = (block - le32_to_cpu(es->s_first_data_block)) % - EXT3_BLOCKS_PER_GROUP(sb); - /* - * Check to see if we are freeing blocks across a group - * boundary. - */ - if (bit + count > EXT3_BLOCKS_PER_GROUP(sb)) { - overflow = bit + count - EXT3_BLOCKS_PER_GROUP(sb); - count -= overflow; - } - brelse(bitmap_bh); - bitmap_bh = read_block_bitmap(sb, block_group); - if (!bitmap_bh) - goto error_return; - desc = ext3_get_group_desc (sb, block_group, &gd_bh); - if (!desc) - goto error_return; - - if (in_range (le32_to_cpu(desc->bg_block_bitmap), block, count) || - in_range (le32_to_cpu(desc->bg_inode_bitmap), block, count) || - in_range (block, le32_to_cpu(desc->bg_inode_table), - sbi->s_itb_per_group) || - in_range (block + count - 1, le32_to_cpu(desc->bg_inode_table), - sbi->s_itb_per_group)) { - ext3_error (sb, "ext3_free_blocks", - "Freeing blocks in system zones - " - "Block = "E3FSBLK", count = %lu", - block, count); - goto error_return; - } - - /* - * We are about to start releasing blocks in the bitmap, - * so we need undo access. - */ - /* @@@ check errors */ - BUFFER_TRACE(bitmap_bh, "getting undo access"); - err = ext3_journal_get_undo_access(handle, bitmap_bh); - if (err) - goto error_return; - - /* - * We are about to modify some metadata. Call the journal APIs - * to unshare ->b_data if a currently-committing transaction is - * using it - */ - BUFFER_TRACE(gd_bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, gd_bh); - if (err) - goto error_return; - - jbd_lock_bh_state(bitmap_bh); - - for (i = 0, group_freed = 0; i < count; i++) { - /* - * An HJ special. This is expensive... - */ -#ifdef CONFIG_JBD_DEBUG - jbd_unlock_bh_state(bitmap_bh); - { - struct buffer_head *debug_bh; - debug_bh = sb_find_get_block(sb, block + i); - if (debug_bh) { - BUFFER_TRACE(debug_bh, "Deleted!"); - if (!bh2jh(bitmap_bh)->b_committed_data) - BUFFER_TRACE(debug_bh, - "No committed data in bitmap"); - BUFFER_TRACE2(debug_bh, bitmap_bh, "bitmap"); - __brelse(debug_bh); - } - } - jbd_lock_bh_state(bitmap_bh); -#endif - if (need_resched()) { - jbd_unlock_bh_state(bitmap_bh); - cond_resched(); - jbd_lock_bh_state(bitmap_bh); - } - /* @@@ This prevents newly-allocated data from being - * freed and then reallocated within the same - * transaction. - * - * Ideally we would want to allow that to happen, but to - * do so requires making journal_forget() capable of - * revoking the queued write of a data block, which - * implies blocking on the journal lock. *forget() - * cannot block due to truncate races. - * - * Eventually we can fix this by making journal_forget() - * return a status indicating whether or not it was able - * to revoke the buffer. On successful revoke, it is - * safe not to set the allocation bit in the committed - * bitmap, because we know that there is no outstanding - * activity on the buffer any more and so it is safe to - * reallocate it. - */ - BUFFER_TRACE(bitmap_bh, "set in b_committed_data"); - J_ASSERT_BH(bitmap_bh, - bh2jh(bitmap_bh)->b_committed_data != NULL); - ext3_set_bit_atomic(sb_bgl_lock(sbi, block_group), bit + i, - bh2jh(bitmap_bh)->b_committed_data); - - /* - * We clear the bit in the bitmap after setting the committed - * data bit, because this is the reverse order to that which - * the allocator uses. - */ - BUFFER_TRACE(bitmap_bh, "clear bit"); - if (!ext3_clear_bit_atomic(sb_bgl_lock(sbi, block_group), - bit + i, bitmap_bh->b_data)) { - jbd_unlock_bh_state(bitmap_bh); - ext3_error(sb, __func__, - "bit already cleared for block "E3FSBLK, - block + i); - jbd_lock_bh_state(bitmap_bh); - BUFFER_TRACE(bitmap_bh, "bit already cleared"); - } else { - group_freed++; - } - } - jbd_unlock_bh_state(bitmap_bh); - - spin_lock(sb_bgl_lock(sbi, block_group)); - le16_add_cpu(&desc->bg_free_blocks_count, group_freed); - spin_unlock(sb_bgl_lock(sbi, block_group)); - percpu_counter_add(&sbi->s_freeblocks_counter, count); - - /* We dirtied the bitmap block */ - BUFFER_TRACE(bitmap_bh, "dirtied bitmap block"); - err = ext3_journal_dirty_metadata(handle, bitmap_bh); - - /* And the group descriptor block */ - BUFFER_TRACE(gd_bh, "dirtied group descriptor block"); - ret = ext3_journal_dirty_metadata(handle, gd_bh); - if (!err) err = ret; - *pdquot_freed_blocks += group_freed; - - if (overflow && !err) { - block += count; - count = overflow; - goto do_more; - } - -error_return: - brelse(bitmap_bh); - ext3_std_error(sb, err); - return; -} - -/** - * ext3_free_blocks() -- Free given blocks and update quota - * @handle: handle for this transaction - * @inode: inode - * @block: start physical block to free - * @count: number of blocks to count - */ -void ext3_free_blocks(handle_t *handle, struct inode *inode, - ext3_fsblk_t block, unsigned long count) -{ - struct super_block *sb = inode->i_sb; - unsigned long dquot_freed_blocks; - - trace_ext3_free_blocks(inode, block, count); - ext3_free_blocks_sb(handle, sb, block, count, &dquot_freed_blocks); - if (dquot_freed_blocks) - dquot_free_block(inode, dquot_freed_blocks); - return; -} - -/** - * ext3_test_allocatable() - * @nr: given allocation block group - * @bh: bufferhead contains the bitmap of the given block group - * - * For ext3 allocations, we must not reuse any blocks which are - * allocated in the bitmap buffer's "last committed data" copy. This - * prevents deletes from freeing up the page for reuse until we have - * committed the delete transaction. - * - * If we didn't do this, then deleting something and reallocating it as - * data would allow the old block to be overwritten before the - * transaction committed (because we force data to disk before commit). - * This would lead to corruption if we crashed between overwriting the - * data and committing the delete. - * - * @@@ We may want to make this allocation behaviour conditional on - * data-writes at some point, and disable it for metadata allocations or - * sync-data inodes. - */ -static int ext3_test_allocatable(ext3_grpblk_t nr, struct buffer_head *bh) -{ - int ret; - struct journal_head *jh = bh2jh(bh); - - if (ext3_test_bit(nr, bh->b_data)) - return 0; - - jbd_lock_bh_state(bh); - if (!jh->b_committed_data) - ret = 1; - else - ret = !ext3_test_bit(nr, jh->b_committed_data); - jbd_unlock_bh_state(bh); - return ret; -} - -/** - * bitmap_search_next_usable_block() - * @start: the starting block (group relative) of the search - * @bh: bufferhead contains the block group bitmap - * @maxblocks: the ending block (group relative) of the reservation - * - * The bitmap search --- search forward alternately through the actual - * bitmap on disk and the last-committed copy in journal, until we find a - * bit free in both bitmaps. - */ -static ext3_grpblk_t -bitmap_search_next_usable_block(ext3_grpblk_t start, struct buffer_head *bh, - ext3_grpblk_t maxblocks) -{ - ext3_grpblk_t next; - struct journal_head *jh = bh2jh(bh); - - while (start < maxblocks) { - next = ext3_find_next_zero_bit(bh->b_data, maxblocks, start); - if (next >= maxblocks) - return -1; - if (ext3_test_allocatable(next, bh)) - return next; - jbd_lock_bh_state(bh); - if (jh->b_committed_data) - start = ext3_find_next_zero_bit(jh->b_committed_data, - maxblocks, next); - jbd_unlock_bh_state(bh); - } - return -1; -} - -/** - * find_next_usable_block() - * @start: the starting block (group relative) to find next - * allocatable block in bitmap. - * @bh: bufferhead contains the block group bitmap - * @maxblocks: the ending block (group relative) for the search - * - * Find an allocatable block in a bitmap. We honor both the bitmap and - * its last-committed copy (if that exists), and perform the "most - * appropriate allocation" algorithm of looking for a free block near - * the initial goal; then for a free byte somewhere in the bitmap; then - * for any free bit in the bitmap. - */ -static ext3_grpblk_t -find_next_usable_block(ext3_grpblk_t start, struct buffer_head *bh, - ext3_grpblk_t maxblocks) -{ - ext3_grpblk_t here, next; - char *p, *r; - - if (start > 0) { - /* - * The goal was occupied; search forward for a free - * block within the next XX blocks. - * - * end_goal is more or less random, but it has to be - * less than EXT3_BLOCKS_PER_GROUP. Aligning up to the - * next 64-bit boundary is simple.. - */ - ext3_grpblk_t end_goal = (start + 63) & ~63; - if (end_goal > maxblocks) - end_goal = maxblocks; - here = ext3_find_next_zero_bit(bh->b_data, end_goal, start); - if (here < end_goal && ext3_test_allocatable(here, bh)) - return here; - ext3_debug("Bit not found near goal\n"); - } - - here = start; - if (here < 0) - here = 0; - - p = bh->b_data + (here >> 3); - r = memscan(p, 0, ((maxblocks + 7) >> 3) - (here >> 3)); - next = (r - bh->b_data) << 3; - - if (next < maxblocks && next >= start && ext3_test_allocatable(next, bh)) - return next; - - /* - * The bitmap search --- search forward alternately through the actual - * bitmap and the last-committed copy until we find a bit free in - * both - */ - here = bitmap_search_next_usable_block(here, bh, maxblocks); - return here; -} - -/** - * claim_block() - * @lock: the spin lock for this block group - * @block: the free block (group relative) to allocate - * @bh: the buffer_head contains the block group bitmap - * - * We think we can allocate this block in this bitmap. Try to set the bit. - * If that succeeds then check that nobody has allocated and then freed the - * block since we saw that is was not marked in b_committed_data. If it _was_ - * allocated and freed then clear the bit in the bitmap again and return - * zero (failure). - */ -static inline int -claim_block(spinlock_t *lock, ext3_grpblk_t block, struct buffer_head *bh) -{ - struct journal_head *jh = bh2jh(bh); - int ret; - - if (ext3_set_bit_atomic(lock, block, bh->b_data)) - return 0; - jbd_lock_bh_state(bh); - if (jh->b_committed_data && ext3_test_bit(block,jh->b_committed_data)) { - ext3_clear_bit_atomic(lock, block, bh->b_data); - ret = 0; - } else { - ret = 1; - } - jbd_unlock_bh_state(bh); - return ret; -} - -/** - * ext3_try_to_allocate() - * @sb: superblock - * @handle: handle to this transaction - * @group: given allocation block group - * @bitmap_bh: bufferhead holds the block bitmap - * @grp_goal: given target block within the group - * @count: target number of blocks to allocate - * @my_rsv: reservation window - * - * Attempt to allocate blocks within a give range. Set the range of allocation - * first, then find the first free bit(s) from the bitmap (within the range), - * and at last, allocate the blocks by claiming the found free bit as allocated. - * - * To set the range of this allocation: - * if there is a reservation window, only try to allocate block(s) from the - * file's own reservation window; - * Otherwise, the allocation range starts from the give goal block, ends at - * the block group's last block. - * - * If we failed to allocate the desired block then we may end up crossing to a - * new bitmap. In that case we must release write access to the old one via - * ext3_journal_release_buffer(), else we'll run out of credits. - */ -static ext3_grpblk_t -ext3_try_to_allocate(struct super_block *sb, handle_t *handle, int group, - struct buffer_head *bitmap_bh, ext3_grpblk_t grp_goal, - unsigned long *count, struct ext3_reserve_window *my_rsv) -{ - ext3_fsblk_t group_first_block; - ext3_grpblk_t start, end; - unsigned long num = 0; - - /* we do allocation within the reservation window if we have a window */ - if (my_rsv) { - group_first_block = ext3_group_first_block_no(sb, group); - if (my_rsv->_rsv_start >= group_first_block) - start = my_rsv->_rsv_start - group_first_block; - else - /* reservation window cross group boundary */ - start = 0; - end = my_rsv->_rsv_end - group_first_block + 1; - if (end > EXT3_BLOCKS_PER_GROUP(sb)) - /* reservation window crosses group boundary */ - end = EXT3_BLOCKS_PER_GROUP(sb); - if ((start <= grp_goal) && (grp_goal < end)) - start = grp_goal; - else - grp_goal = -1; - } else { - if (grp_goal > 0) - start = grp_goal; - else - start = 0; - end = EXT3_BLOCKS_PER_GROUP(sb); - } - - BUG_ON(start > EXT3_BLOCKS_PER_GROUP(sb)); - -repeat: - if (grp_goal < 0 || !ext3_test_allocatable(grp_goal, bitmap_bh)) { - grp_goal = find_next_usable_block(start, bitmap_bh, end); - if (grp_goal < 0) - goto fail_access; - if (!my_rsv) { - int i; - - for (i = 0; i < 7 && grp_goal > start && - ext3_test_allocatable(grp_goal - 1, - bitmap_bh); - i++, grp_goal--) - ; - } - } - start = grp_goal; - - if (!claim_block(sb_bgl_lock(EXT3_SB(sb), group), - grp_goal, bitmap_bh)) { - /* - * The block was allocated by another thread, or it was - * allocated and then freed by another thread - */ - start++; - grp_goal++; - if (start >= end) - goto fail_access; - goto repeat; - } - num++; - grp_goal++; - while (num < *count && grp_goal < end - && ext3_test_allocatable(grp_goal, bitmap_bh) - && claim_block(sb_bgl_lock(EXT3_SB(sb), group), - grp_goal, bitmap_bh)) { - num++; - grp_goal++; - } - *count = num; - return grp_goal - num; -fail_access: - *count = num; - return -1; -} - -/** - * find_next_reservable_window(): - * find a reservable space within the given range. - * It does not allocate the reservation window for now: - * alloc_new_reservation() will do the work later. - * - * @search_head: the head of the searching list; - * This is not necessarily the list head of the whole filesystem - * - * We have both head and start_block to assist the search - * for the reservable space. The list starts from head, - * but we will shift to the place where start_block is, - * then start from there, when looking for a reservable space. - * - * @my_rsv: the reservation window - * - * @sb: the super block - * - * @start_block: the first block we consider to start - * the real search from - * - * @last_block: - * the maximum block number that our goal reservable space - * could start from. This is normally the last block in this - * group. The search will end when we found the start of next - * possible reservable space is out of this boundary. - * This could handle the cross boundary reservation window - * request. - * - * basically we search from the given range, rather than the whole - * reservation double linked list, (start_block, last_block) - * to find a free region that is of my size and has not - * been reserved. - * - */ -static int find_next_reservable_window( - struct ext3_reserve_window_node *search_head, - struct ext3_reserve_window_node *my_rsv, - struct super_block * sb, - ext3_fsblk_t start_block, - ext3_fsblk_t last_block) -{ - struct rb_node *next; - struct ext3_reserve_window_node *rsv, *prev; - ext3_fsblk_t cur; - int size = my_rsv->rsv_goal_size; - - /* TODO: make the start of the reservation window byte-aligned */ - /* cur = *start_block & ~7;*/ - cur = start_block; - rsv = search_head; - if (!rsv) - return -1; - - while (1) { - if (cur <= rsv->rsv_end) - cur = rsv->rsv_end + 1; - - /* TODO? - * in the case we could not find a reservable space - * that is what is expected, during the re-search, we could - * remember what's the largest reservable space we could have - * and return that one. - * - * For now it will fail if we could not find the reservable - * space with expected-size (or more)... - */ - if (cur > last_block) - return -1; /* fail */ - - prev = rsv; - next = rb_next(&rsv->rsv_node); - rsv = rb_entry(next,struct ext3_reserve_window_node,rsv_node); - - /* - * Reached the last reservation, we can just append to the - * previous one. - */ - if (!next) - break; - - if (cur + size <= rsv->rsv_start) { - /* - * Found a reserveable space big enough. We could - * have a reservation across the group boundary here - */ - break; - } - } - /* - * we come here either : - * when we reach the end of the whole list, - * and there is empty reservable space after last entry in the list. - * append it to the end of the list. - * - * or we found one reservable space in the middle of the list, - * return the reservation window that we could append to. - * succeed. - */ - - if ((prev != my_rsv) && (!rsv_is_empty(&my_rsv->rsv_window))) - rsv_window_remove(sb, my_rsv); - - /* - * Let's book the whole available window for now. We will check the - * disk bitmap later and then, if there are free blocks then we adjust - * the window size if it's larger than requested. - * Otherwise, we will remove this node from the tree next time - * call find_next_reservable_window. - */ - my_rsv->rsv_start = cur; - my_rsv->rsv_end = cur + size - 1; - my_rsv->rsv_alloc_hit = 0; - - if (prev != my_rsv) - ext3_rsv_window_add(sb, my_rsv); - - return 0; -} - -/** - * alloc_new_reservation()--allocate a new reservation window - * - * To make a new reservation, we search part of the filesystem - * reservation list (the list that inside the group). We try to - * allocate a new reservation window near the allocation goal, - * or the beginning of the group, if there is no goal. - * - * We first find a reservable space after the goal, then from - * there, we check the bitmap for the first free block after - * it. If there is no free block until the end of group, then the - * whole group is full, we failed. Otherwise, check if the free - * block is inside the expected reservable space, if so, we - * succeed. - * If the first free block is outside the reservable space, then - * start from the first free block, we search for next available - * space, and go on. - * - * on succeed, a new reservation will be found and inserted into the list - * It contains at least one free block, and it does not overlap with other - * reservation windows. - * - * failed: we failed to find a reservation window in this group - * - * @my_rsv: the reservation window - * - * @grp_goal: The goal (group-relative). It is where the search for a - * free reservable space should start from. - * if we have a grp_goal(grp_goal >0 ), then start from there, - * no grp_goal(grp_goal = -1), we start from the first block - * of the group. - * - * @sb: the super block - * @group: the group we are trying to allocate in - * @bitmap_bh: the block group block bitmap - * - */ -static int alloc_new_reservation(struct ext3_reserve_window_node *my_rsv, - ext3_grpblk_t grp_goal, struct super_block *sb, - unsigned int group, struct buffer_head *bitmap_bh) -{ - struct ext3_reserve_window_node *search_head; - ext3_fsblk_t group_first_block, group_end_block, start_block; - ext3_grpblk_t first_free_block; - struct rb_root *fs_rsv_root = &EXT3_SB(sb)->s_rsv_window_root; - unsigned long size; - int ret; - spinlock_t *rsv_lock = &EXT3_SB(sb)->s_rsv_window_lock; - - group_first_block = ext3_group_first_block_no(sb, group); - group_end_block = group_first_block + (EXT3_BLOCKS_PER_GROUP(sb) - 1); - - if (grp_goal < 0) - start_block = group_first_block; - else - start_block = grp_goal + group_first_block; - - trace_ext3_alloc_new_reservation(sb, start_block); - size = my_rsv->rsv_goal_size; - - if (!rsv_is_empty(&my_rsv->rsv_window)) { - /* - * if the old reservation is cross group boundary - * and if the goal is inside the old reservation window, - * we will come here when we just failed to allocate from - * the first part of the window. We still have another part - * that belongs to the next group. In this case, there is no - * point to discard our window and try to allocate a new one - * in this group(which will fail). we should - * keep the reservation window, just simply move on. - * - * Maybe we could shift the start block of the reservation - * window to the first block of next group. - */ - - if ((my_rsv->rsv_start <= group_end_block) && - (my_rsv->rsv_end > group_end_block) && - (start_block >= my_rsv->rsv_start)) - return -1; - - if ((my_rsv->rsv_alloc_hit > - (my_rsv->rsv_end - my_rsv->rsv_start + 1) / 2)) { - /* - * if the previously allocation hit ratio is - * greater than 1/2, then we double the size of - * the reservation window the next time, - * otherwise we keep the same size window - */ - size = size * 2; - if (size > EXT3_MAX_RESERVE_BLOCKS) - size = EXT3_MAX_RESERVE_BLOCKS; - my_rsv->rsv_goal_size= size; - } - } - - spin_lock(rsv_lock); - /* - * shift the search start to the window near the goal block - */ - search_head = search_reserve_window(fs_rsv_root, start_block); - - /* - * find_next_reservable_window() simply finds a reservable window - * inside the given range(start_block, group_end_block). - * - * To make sure the reservation window has a free bit inside it, we - * need to check the bitmap after we found a reservable window. - */ -retry: - ret = find_next_reservable_window(search_head, my_rsv, sb, - start_block, group_end_block); - - if (ret == -1) { - if (!rsv_is_empty(&my_rsv->rsv_window)) - rsv_window_remove(sb, my_rsv); - spin_unlock(rsv_lock); - return -1; - } - - /* - * On success, find_next_reservable_window() returns the - * reservation window where there is a reservable space after it. - * Before we reserve this reservable space, we need - * to make sure there is at least a free block inside this region. - * - * searching the first free bit on the block bitmap and copy of - * last committed bitmap alternatively, until we found a allocatable - * block. Search start from the start block of the reservable space - * we just found. - */ - spin_unlock(rsv_lock); - first_free_block = bitmap_search_next_usable_block( - my_rsv->rsv_start - group_first_block, - bitmap_bh, group_end_block - group_first_block + 1); - - if (first_free_block < 0) { - /* - * no free block left on the bitmap, no point - * to reserve the space. return failed. - */ - spin_lock(rsv_lock); - if (!rsv_is_empty(&my_rsv->rsv_window)) - rsv_window_remove(sb, my_rsv); - spin_unlock(rsv_lock); - return -1; /* failed */ - } - - start_block = first_free_block + group_first_block; - /* - * check if the first free block is within the - * free space we just reserved - */ - if (start_block >= my_rsv->rsv_start && - start_block <= my_rsv->rsv_end) { - trace_ext3_reserved(sb, start_block, my_rsv); - return 0; /* success */ - } - /* - * if the first free bit we found is out of the reservable space - * continue search for next reservable space, - * start from where the free block is, - * we also shift the list head to where we stopped last time - */ - search_head = my_rsv; - spin_lock(rsv_lock); - goto retry; -} - -/** - * try_to_extend_reservation() - * @my_rsv: given reservation window - * @sb: super block - * @size: the delta to extend - * - * Attempt to expand the reservation window large enough to have - * required number of free blocks - * - * Since ext3_try_to_allocate() will always allocate blocks within - * the reservation window range, if the window size is too small, - * multiple blocks allocation has to stop at the end of the reservation - * window. To make this more efficient, given the total number of - * blocks needed and the current size of the window, we try to - * expand the reservation window size if necessary on a best-effort - * basis before ext3_new_blocks() tries to allocate blocks, - */ -static void try_to_extend_reservation(struct ext3_reserve_window_node *my_rsv, - struct super_block *sb, int size) -{ - struct ext3_reserve_window_node *next_rsv; - struct rb_node *next; - spinlock_t *rsv_lock = &EXT3_SB(sb)->s_rsv_window_lock; - - if (!spin_trylock(rsv_lock)) - return; - - next = rb_next(&my_rsv->rsv_node); - - if (!next) - my_rsv->rsv_end += size; - else { - next_rsv = rb_entry(next, struct ext3_reserve_window_node, rsv_node); - - if ((next_rsv->rsv_start - my_rsv->rsv_end - 1) >= size) - my_rsv->rsv_end += size; - else - my_rsv->rsv_end = next_rsv->rsv_start - 1; - } - spin_unlock(rsv_lock); -} - -/** - * ext3_try_to_allocate_with_rsv() - * @sb: superblock - * @handle: handle to this transaction - * @group: given allocation block group - * @bitmap_bh: bufferhead holds the block bitmap - * @grp_goal: given target block within the group - * @my_rsv: reservation window - * @count: target number of blocks to allocate - * @errp: pointer to store the error code - * - * This is the main function used to allocate a new block and its reservation - * window. - * - * Each time when a new block allocation is need, first try to allocate from - * its own reservation. If it does not have a reservation window, instead of - * looking for a free bit on bitmap first, then look up the reservation list to - * see if it is inside somebody else's reservation window, we try to allocate a - * reservation window for it starting from the goal first. Then do the block - * allocation within the reservation window. - * - * This will avoid keeping on searching the reservation list again and - * again when somebody is looking for a free block (without - * reservation), and there are lots of free blocks, but they are all - * being reserved. - * - * We use a red-black tree for the per-filesystem reservation list. - * - */ -static ext3_grpblk_t -ext3_try_to_allocate_with_rsv(struct super_block *sb, handle_t *handle, - unsigned int group, struct buffer_head *bitmap_bh, - ext3_grpblk_t grp_goal, - struct ext3_reserve_window_node * my_rsv, - unsigned long *count, int *errp) -{ - ext3_fsblk_t group_first_block, group_last_block; - ext3_grpblk_t ret = 0; - int fatal; - unsigned long num = *count; - - *errp = 0; - - /* - * Make sure we use undo access for the bitmap, because it is critical - * that we do the frozen_data COW on bitmap buffers in all cases even - * if the buffer is in BJ_Forget state in the committing transaction. - */ - BUFFER_TRACE(bitmap_bh, "get undo access for new block"); - fatal = ext3_journal_get_undo_access(handle, bitmap_bh); - if (fatal) { - *errp = fatal; - return -1; - } - - /* - * we don't deal with reservation when - * filesystem is mounted without reservation - * or the file is not a regular file - * or last attempt to allocate a block with reservation turned on failed - */ - if (my_rsv == NULL ) { - ret = ext3_try_to_allocate(sb, handle, group, bitmap_bh, - grp_goal, count, NULL); - goto out; - } - /* - * grp_goal is a group relative block number (if there is a goal) - * 0 <= grp_goal < EXT3_BLOCKS_PER_GROUP(sb) - * first block is a filesystem wide block number - * first block is the block number of the first block in this group - */ - group_first_block = ext3_group_first_block_no(sb, group); - group_last_block = group_first_block + (EXT3_BLOCKS_PER_GROUP(sb) - 1); - - /* - * Basically we will allocate a new block from inode's reservation - * window. - * - * We need to allocate a new reservation window, if: - * a) inode does not have a reservation window; or - * b) last attempt to allocate a block from existing reservation - * failed; or - * c) we come here with a goal and with a reservation window - * - * We do not need to allocate a new reservation window if we come here - * at the beginning with a goal and the goal is inside the window, or - * we don't have a goal but already have a reservation window. - * then we could go to allocate from the reservation window directly. - */ - while (1) { - if (rsv_is_empty(&my_rsv->rsv_window) || (ret < 0) || - !goal_in_my_reservation(&my_rsv->rsv_window, - grp_goal, group, sb)) { - if (my_rsv->rsv_goal_size < *count) - my_rsv->rsv_goal_size = *count; - ret = alloc_new_reservation(my_rsv, grp_goal, sb, - group, bitmap_bh); - if (ret < 0) - break; /* failed */ - - if (!goal_in_my_reservation(&my_rsv->rsv_window, - grp_goal, group, sb)) - grp_goal = -1; - } else if (grp_goal >= 0) { - int curr = my_rsv->rsv_end - - (grp_goal + group_first_block) + 1; - - if (curr < *count) - try_to_extend_reservation(my_rsv, sb, - *count - curr); - } - - if ((my_rsv->rsv_start > group_last_block) || - (my_rsv->rsv_end < group_first_block)) { - rsv_window_dump(&EXT3_SB(sb)->s_rsv_window_root, 1); - BUG(); - } - ret = ext3_try_to_allocate(sb, handle, group, bitmap_bh, - grp_goal, &num, &my_rsv->rsv_window); - if (ret >= 0) { - my_rsv->rsv_alloc_hit += num; - *count = num; - break; /* succeed */ - } - num = *count; - } -out: - if (ret >= 0) { - BUFFER_TRACE(bitmap_bh, "journal_dirty_metadata for " - "bitmap block"); - fatal = ext3_journal_dirty_metadata(handle, bitmap_bh); - if (fatal) { - *errp = fatal; - return -1; - } - return ret; - } - - BUFFER_TRACE(bitmap_bh, "journal_release_buffer"); - ext3_journal_release_buffer(handle, bitmap_bh); - return ret; -} - -/** - * ext3_has_free_blocks() - * @sbi: in-core super block structure. - * - * Check if filesystem has at least 1 free block available for allocation. - */ -static int ext3_has_free_blocks(struct ext3_sb_info *sbi, int use_reservation) -{ - ext3_fsblk_t free_blocks, root_blocks; - - free_blocks = percpu_counter_read_positive(&sbi->s_freeblocks_counter); - root_blocks = le32_to_cpu(sbi->s_es->s_r_blocks_count); - if (free_blocks < root_blocks + 1 && !capable(CAP_SYS_RESOURCE) && - !use_reservation && !uid_eq(sbi->s_resuid, current_fsuid()) && - (gid_eq(sbi->s_resgid, GLOBAL_ROOT_GID) || - !in_group_p (sbi->s_resgid))) { - return 0; - } - return 1; -} - -/** - * ext3_should_retry_alloc() - * @sb: super block - * @retries number of attemps has been made - * - * ext3_should_retry_alloc() is called when ENOSPC is returned, and if - * it is profitable to retry the operation, this function will wait - * for the current or committing transaction to complete, and then - * return TRUE. - * - * if the total number of retries exceed three times, return FALSE. - */ -int ext3_should_retry_alloc(struct super_block *sb, int *retries) -{ - if (!ext3_has_free_blocks(EXT3_SB(sb), 0) || (*retries)++ > 3) - return 0; - - jbd_debug(1, "%s: retrying operation after ENOSPC\n", sb->s_id); - - return journal_force_commit_nested(EXT3_SB(sb)->s_journal); -} - -/** - * ext3_new_blocks() -- core block(s) allocation function - * @handle: handle to this transaction - * @inode: file inode - * @goal: given target block(filesystem wide) - * @count: target number of blocks to allocate - * @errp: error code - * - * ext3_new_blocks uses a goal block to assist allocation. It tries to - * allocate block(s) from the block group contains the goal block first. If that - * fails, it will try to allocate block(s) from other block groups without - * any specific goal block. - * - */ -ext3_fsblk_t ext3_new_blocks(handle_t *handle, struct inode *inode, - ext3_fsblk_t goal, unsigned long *count, int *errp) -{ - struct buffer_head *bitmap_bh = NULL; - struct buffer_head *gdp_bh; - int group_no; - int goal_group; - ext3_grpblk_t grp_target_blk; /* blockgroup relative goal block */ - ext3_grpblk_t grp_alloc_blk; /* blockgroup-relative allocated block*/ - ext3_fsblk_t ret_block; /* filesyetem-wide allocated block */ - int bgi; /* blockgroup iteration index */ - int fatal = 0, err; - int performed_allocation = 0; - ext3_grpblk_t free_blocks; /* number of free blocks in a group */ - struct super_block *sb; - struct ext3_group_desc *gdp; - struct ext3_super_block *es; - struct ext3_sb_info *sbi; - struct ext3_reserve_window_node *my_rsv = NULL; - struct ext3_block_alloc_info *block_i; - unsigned short windowsz = 0; -#ifdef EXT3FS_DEBUG - static int goal_hits, goal_attempts; -#endif - unsigned long ngroups; - unsigned long num = *count; - - *errp = -ENOSPC; - sb = inode->i_sb; - - /* - * Check quota for allocation of this block. - */ - err = dquot_alloc_block(inode, num); - if (err) { - *errp = err; - return 0; - } - - trace_ext3_request_blocks(inode, goal, num); - - sbi = EXT3_SB(sb); - es = sbi->s_es; - ext3_debug("goal=%lu.\n", goal); - /* - * Allocate a block from reservation only when - * filesystem is mounted with reservation(default,-o reservation), and - * it's a regular file, and - * the desired window size is greater than 0 (One could use ioctl - * command EXT3_IOC_SETRSVSZ to set the window size to 0 to turn off - * reservation on that particular file) - */ - block_i = EXT3_I(inode)->i_block_alloc_info; - if (block_i && ((windowsz = block_i->rsv_window_node.rsv_goal_size) > 0)) - my_rsv = &block_i->rsv_window_node; - - if (!ext3_has_free_blocks(sbi, IS_NOQUOTA(inode))) { - *errp = -ENOSPC; - goto out; - } - - /* - * First, test whether the goal block is free. - */ - if (goal < le32_to_cpu(es->s_first_data_block) || - goal >= le32_to_cpu(es->s_blocks_count)) - goal = le32_to_cpu(es->s_first_data_block); - group_no = (goal - le32_to_cpu(es->s_first_data_block)) / - EXT3_BLOCKS_PER_GROUP(sb); - goal_group = group_no; -retry_alloc: - gdp = ext3_get_group_desc(sb, group_no, &gdp_bh); - if (!gdp) - goto io_error; - - free_blocks = le16_to_cpu(gdp->bg_free_blocks_count); - /* - * if there is not enough free blocks to make a new resevation - * turn off reservation for this allocation - */ - if (my_rsv && (free_blocks < windowsz) - && (free_blocks > 0) - && (rsv_is_empty(&my_rsv->rsv_window))) - my_rsv = NULL; - - if (free_blocks > 0) { - grp_target_blk = ((goal - le32_to_cpu(es->s_first_data_block)) % - EXT3_BLOCKS_PER_GROUP(sb)); - bitmap_bh = read_block_bitmap(sb, group_no); - if (!bitmap_bh) - goto io_error; - grp_alloc_blk = ext3_try_to_allocate_with_rsv(sb, handle, - group_no, bitmap_bh, grp_target_blk, - my_rsv, &num, &fatal); - if (fatal) - goto out; - if (grp_alloc_blk >= 0) - goto allocated; - } - - ngroups = EXT3_SB(sb)->s_groups_count; - smp_rmb(); - - /* - * Now search the rest of the groups. We assume that - * group_no and gdp correctly point to the last group visited. - */ - for (bgi = 0; bgi < ngroups; bgi++) { - group_no++; - if (group_no >= ngroups) - group_no = 0; - gdp = ext3_get_group_desc(sb, group_no, &gdp_bh); - if (!gdp) - goto io_error; - free_blocks = le16_to_cpu(gdp->bg_free_blocks_count); - /* - * skip this group (and avoid loading bitmap) if there - * are no free blocks - */ - if (!free_blocks) - continue; - /* - * skip this group if the number of - * free blocks is less than half of the reservation - * window size. - */ - if (my_rsv && (free_blocks <= (windowsz/2))) - continue; - - brelse(bitmap_bh); - bitmap_bh = read_block_bitmap(sb, group_no); - if (!bitmap_bh) - goto io_error; - /* - * try to allocate block(s) from this group, without a goal(-1). - */ - grp_alloc_blk = ext3_try_to_allocate_with_rsv(sb, handle, - group_no, bitmap_bh, -1, my_rsv, - &num, &fatal); - if (fatal) - goto out; - if (grp_alloc_blk >= 0) - goto allocated; - } - /* - * We may end up a bogus earlier ENOSPC error due to - * filesystem is "full" of reservations, but - * there maybe indeed free blocks available on disk - * In this case, we just forget about the reservations - * just do block allocation as without reservations. - */ - if (my_rsv) { - my_rsv = NULL; - windowsz = 0; - group_no = goal_group; - goto retry_alloc; - } - /* No space left on the device */ - *errp = -ENOSPC; - goto out; - -allocated: - - ext3_debug("using block group %d(%d)\n", - group_no, gdp->bg_free_blocks_count); - - BUFFER_TRACE(gdp_bh, "get_write_access"); - fatal = ext3_journal_get_write_access(handle, gdp_bh); - if (fatal) - goto out; - - ret_block = grp_alloc_blk + ext3_group_first_block_no(sb, group_no); - - if (in_range(le32_to_cpu(gdp->bg_block_bitmap), ret_block, num) || - in_range(le32_to_cpu(gdp->bg_inode_bitmap), ret_block, num) || - in_range(ret_block, le32_to_cpu(gdp->bg_inode_table), - EXT3_SB(sb)->s_itb_per_group) || - in_range(ret_block + num - 1, le32_to_cpu(gdp->bg_inode_table), - EXT3_SB(sb)->s_itb_per_group)) { - ext3_error(sb, "ext3_new_block", - "Allocating block in system zone - " - "blocks from "E3FSBLK", length %lu", - ret_block, num); - /* - * claim_block() marked the blocks we allocated as in use. So we - * may want to selectively mark some of the blocks as free. - */ - goto retry_alloc; - } - - performed_allocation = 1; - -#ifdef CONFIG_JBD_DEBUG - { - struct buffer_head *debug_bh; - - /* Record bitmap buffer state in the newly allocated block */ - debug_bh = sb_find_get_block(sb, ret_block); - if (debug_bh) { - BUFFER_TRACE(debug_bh, "state when allocated"); - BUFFER_TRACE2(debug_bh, bitmap_bh, "bitmap state"); - brelse(debug_bh); - } - } - jbd_lock_bh_state(bitmap_bh); - spin_lock(sb_bgl_lock(sbi, group_no)); - if (buffer_jbd(bitmap_bh) && bh2jh(bitmap_bh)->b_committed_data) { - int i; - - for (i = 0; i < num; i++) { - if (ext3_test_bit(grp_alloc_blk+i, - bh2jh(bitmap_bh)->b_committed_data)) { - printk("%s: block was unexpectedly set in " - "b_committed_data\n", __func__); - } - } - } - ext3_debug("found bit %d\n", grp_alloc_blk); - spin_unlock(sb_bgl_lock(sbi, group_no)); - jbd_unlock_bh_state(bitmap_bh); -#endif - - if (ret_block + num - 1 >= le32_to_cpu(es->s_blocks_count)) { - ext3_error(sb, "ext3_new_block", - "block("E3FSBLK") >= blocks count(%d) - " - "block_group = %d, es == %p ", ret_block, - le32_to_cpu(es->s_blocks_count), group_no, es); - goto out; - } - - /* - * It is up to the caller to add the new buffer to a journal - * list of some description. We don't know in advance whether - * the caller wants to use it as metadata or data. - */ - ext3_debug("allocating block %lu. Goal hits %d of %d.\n", - ret_block, goal_hits, goal_attempts); - - spin_lock(sb_bgl_lock(sbi, group_no)); - le16_add_cpu(&gdp->bg_free_blocks_count, -num); - spin_unlock(sb_bgl_lock(sbi, group_no)); - percpu_counter_sub(&sbi->s_freeblocks_counter, num); - - BUFFER_TRACE(gdp_bh, "journal_dirty_metadata for group descriptor"); - fatal = ext3_journal_dirty_metadata(handle, gdp_bh); - if (fatal) - goto out; - - *errp = 0; - brelse(bitmap_bh); - - if (num < *count) { - dquot_free_block(inode, *count-num); - *count = num; - } - - trace_ext3_allocate_blocks(inode, goal, num, - (unsigned long long)ret_block); - - return ret_block; - -io_error: - *errp = -EIO; -out: - if (fatal) { - *errp = fatal; - ext3_std_error(sb, fatal); - } - /* - * Undo the block allocation - */ - if (!performed_allocation) - dquot_free_block(inode, *count); - brelse(bitmap_bh); - return 0; -} - -ext3_fsblk_t ext3_new_block(handle_t *handle, struct inode *inode, - ext3_fsblk_t goal, int *errp) -{ - unsigned long count = 1; - - return ext3_new_blocks(handle, inode, goal, &count, errp); -} - -/** - * ext3_count_free_blocks() -- count filesystem free blocks - * @sb: superblock - * - * Adds up the number of free blocks from each block group. - */ -ext3_fsblk_t ext3_count_free_blocks(struct super_block *sb) -{ - ext3_fsblk_t desc_count; - struct ext3_group_desc *gdp; - int i; - unsigned long ngroups = EXT3_SB(sb)->s_groups_count; -#ifdef EXT3FS_DEBUG - struct ext3_super_block *es; - ext3_fsblk_t bitmap_count; - unsigned long x; - struct buffer_head *bitmap_bh = NULL; - - es = EXT3_SB(sb)->s_es; - desc_count = 0; - bitmap_count = 0; - gdp = NULL; - - smp_rmb(); - for (i = 0; i < ngroups; i++) { - gdp = ext3_get_group_desc(sb, i, NULL); - if (!gdp) - continue; - desc_count += le16_to_cpu(gdp->bg_free_blocks_count); - brelse(bitmap_bh); - bitmap_bh = read_block_bitmap(sb, i); - if (bitmap_bh == NULL) - continue; - - x = ext3_count_free(bitmap_bh, sb->s_blocksize); - printk("group %d: stored = %d, counted = %lu\n", - i, le16_to_cpu(gdp->bg_free_blocks_count), x); - bitmap_count += x; - } - brelse(bitmap_bh); - printk("ext3_count_free_blocks: stored = "E3FSBLK - ", computed = "E3FSBLK", "E3FSBLK"\n", - (ext3_fsblk_t)le32_to_cpu(es->s_free_blocks_count), - desc_count, bitmap_count); - return bitmap_count; -#else - desc_count = 0; - smp_rmb(); - for (i = 0; i < ngroups; i++) { - gdp = ext3_get_group_desc(sb, i, NULL); - if (!gdp) - continue; - desc_count += le16_to_cpu(gdp->bg_free_blocks_count); - } - - return desc_count; -#endif -} - -static inline int test_root(int a, int b) -{ - int num = b; - - while (a > num) - num *= b; - return num == a; -} - -static int ext3_group_sparse(int group) -{ - if (group <= 1) - return 1; - if (!(group & 1)) - return 0; - return (test_root(group, 7) || test_root(group, 5) || - test_root(group, 3)); -} - -/** - * ext3_bg_has_super - number of blocks used by the superblock in group - * @sb: superblock for filesystem - * @group: group number to check - * - * Return the number of blocks used by the superblock (primary or backup) - * in this group. Currently this will be only 0 or 1. - */ -int ext3_bg_has_super(struct super_block *sb, int group) -{ - if (EXT3_HAS_RO_COMPAT_FEATURE(sb, - EXT3_FEATURE_RO_COMPAT_SPARSE_SUPER) && - !ext3_group_sparse(group)) - return 0; - return 1; -} - -static unsigned long ext3_bg_num_gdb_meta(struct super_block *sb, int group) -{ - unsigned long metagroup = group / EXT3_DESC_PER_BLOCK(sb); - unsigned long first = metagroup * EXT3_DESC_PER_BLOCK(sb); - unsigned long last = first + EXT3_DESC_PER_BLOCK(sb) - 1; - - if (group == first || group == first + 1 || group == last) - return 1; - return 0; -} - -static unsigned long ext3_bg_num_gdb_nometa(struct super_block *sb, int group) -{ - return ext3_bg_has_super(sb, group) ? EXT3_SB(sb)->s_gdb_count : 0; -} - -/** - * ext3_bg_num_gdb - number of blocks used by the group table in group - * @sb: superblock for filesystem - * @group: group number to check - * - * Return the number of blocks used by the group descriptor table - * (primary or backup) in this group. In the future there may be a - * different number of descriptor blocks in each group. - */ -unsigned long ext3_bg_num_gdb(struct super_block *sb, int group) -{ - unsigned long first_meta_bg = - le32_to_cpu(EXT3_SB(sb)->s_es->s_first_meta_bg); - unsigned long metagroup = group / EXT3_DESC_PER_BLOCK(sb); - - if (!EXT3_HAS_INCOMPAT_FEATURE(sb,EXT3_FEATURE_INCOMPAT_META_BG) || - metagroup < first_meta_bg) - return ext3_bg_num_gdb_nometa(sb,group); - - return ext3_bg_num_gdb_meta(sb,group); - -} - -/** - * ext3_trim_all_free -- function to trim all free space in alloc. group - * @sb: super block for file system - * @group: allocation group to trim - * @start: first group block to examine - * @max: last group block to examine - * @gdp: allocation group description structure - * @minblocks: minimum extent block count - * - * ext3_trim_all_free walks through group's block bitmap searching for free - * blocks. When the free block is found, it tries to allocate this block and - * consequent free block to get the biggest free extent possible, until it - * reaches any used block. Then issue a TRIM command on this extent and free - * the extent in the block bitmap. This is done until whole group is scanned. - */ -static ext3_grpblk_t ext3_trim_all_free(struct super_block *sb, - unsigned int group, - ext3_grpblk_t start, ext3_grpblk_t max, - ext3_grpblk_t minblocks) -{ - handle_t *handle; - ext3_grpblk_t next, free_blocks, bit, freed, count = 0; - ext3_fsblk_t discard_block; - struct ext3_sb_info *sbi; - struct buffer_head *gdp_bh, *bitmap_bh = NULL; - struct ext3_group_desc *gdp; - int err = 0, ret = 0; - - /* - * We will update one block bitmap, and one group descriptor - */ - handle = ext3_journal_start_sb(sb, 2); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - bitmap_bh = read_block_bitmap(sb, group); - if (!bitmap_bh) { - err = -EIO; - goto err_out; - } - - BUFFER_TRACE(bitmap_bh, "getting undo access"); - err = ext3_journal_get_undo_access(handle, bitmap_bh); - if (err) - goto err_out; - - gdp = ext3_get_group_desc(sb, group, &gdp_bh); - if (!gdp) { - err = -EIO; - goto err_out; - } - - BUFFER_TRACE(gdp_bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, gdp_bh); - if (err) - goto err_out; - - free_blocks = le16_to_cpu(gdp->bg_free_blocks_count); - sbi = EXT3_SB(sb); - - /* Walk through the whole group */ - while (start <= max) { - start = bitmap_search_next_usable_block(start, bitmap_bh, max); - if (start < 0) - break; - next = start; - - /* - * Allocate contiguous free extents by setting bits in the - * block bitmap - */ - while (next <= max - && claim_block(sb_bgl_lock(sbi, group), - next, bitmap_bh)) { - next++; - } - - /* We did not claim any blocks */ - if (next == start) - continue; - - discard_block = (ext3_fsblk_t)start + - ext3_group_first_block_no(sb, group); - - /* Update counters */ - spin_lock(sb_bgl_lock(sbi, group)); - le16_add_cpu(&gdp->bg_free_blocks_count, start - next); - spin_unlock(sb_bgl_lock(sbi, group)); - percpu_counter_sub(&sbi->s_freeblocks_counter, next - start); - - free_blocks -= next - start; - /* Do not issue a TRIM on extents smaller than minblocks */ - if ((next - start) < minblocks) - goto free_extent; - - trace_ext3_discard_blocks(sb, discard_block, next - start); - /* Send the TRIM command down to the device */ - err = sb_issue_discard(sb, discard_block, next - start, - GFP_NOFS, 0); - count += (next - start); -free_extent: - freed = 0; - - /* - * Clear bits in the bitmap - */ - for (bit = start; bit < next; bit++) { - BUFFER_TRACE(bitmap_bh, "clear bit"); - if (!ext3_clear_bit_atomic(sb_bgl_lock(sbi, group), - bit, bitmap_bh->b_data)) { - ext3_error(sb, __func__, - "bit already cleared for block "E3FSBLK, - (unsigned long)bit); - BUFFER_TRACE(bitmap_bh, "bit already cleared"); - } else { - freed++; - } - } - - /* Update couters */ - spin_lock(sb_bgl_lock(sbi, group)); - le16_add_cpu(&gdp->bg_free_blocks_count, freed); - spin_unlock(sb_bgl_lock(sbi, group)); - percpu_counter_add(&sbi->s_freeblocks_counter, freed); - - start = next; - if (err < 0) { - if (err != -EOPNOTSUPP) - ext3_warning(sb, __func__, "Discard command " - "returned error %d\n", err); - break; - } - - if (fatal_signal_pending(current)) { - err = -ERESTARTSYS; - break; - } - - cond_resched(); - - /* No more suitable extents */ - if (free_blocks < minblocks) - break; - } - - /* We dirtied the bitmap block */ - BUFFER_TRACE(bitmap_bh, "dirtied bitmap block"); - ret = ext3_journal_dirty_metadata(handle, bitmap_bh); - if (!err) - err = ret; - - /* And the group descriptor block */ - BUFFER_TRACE(gdp_bh, "dirtied group descriptor block"); - ret = ext3_journal_dirty_metadata(handle, gdp_bh); - if (!err) - err = ret; - - ext3_debug("trimmed %d blocks in the group %d\n", - count, group); - -err_out: - if (err) - count = err; - ext3_journal_stop(handle); - brelse(bitmap_bh); - - return count; -} - -/** - * ext3_trim_fs() -- trim ioctl handle function - * @sb: superblock for filesystem - * @start: First Byte to trim - * @len: number of Bytes to trim from start - * @minlen: minimum extent length in Bytes - * - * ext3_trim_fs goes through all allocation groups containing Bytes from - * start to start+len. For each such a group ext3_trim_all_free function - * is invoked to trim all free space. - */ -int ext3_trim_fs(struct super_block *sb, struct fstrim_range *range) -{ - ext3_grpblk_t last_block, first_block; - unsigned long group, first_group, last_group; - struct ext3_group_desc *gdp; - struct ext3_super_block *es = EXT3_SB(sb)->s_es; - uint64_t start, minlen, end, trimmed = 0; - ext3_fsblk_t first_data_blk = - le32_to_cpu(EXT3_SB(sb)->s_es->s_first_data_block); - ext3_fsblk_t max_blks = le32_to_cpu(es->s_blocks_count); - int ret = 0; - - start = range->start >> sb->s_blocksize_bits; - end = start + (range->len >> sb->s_blocksize_bits) - 1; - minlen = range->minlen >> sb->s_blocksize_bits; - - if (minlen > EXT3_BLOCKS_PER_GROUP(sb) || - start >= max_blks || - range->len < sb->s_blocksize) - return -EINVAL; - if (end >= max_blks) - end = max_blks - 1; - if (end <= first_data_blk) - goto out; - if (start < first_data_blk) - start = first_data_blk; - - smp_rmb(); - - /* Determine first and last group to examine based on start and len */ - ext3_get_group_no_and_offset(sb, (ext3_fsblk_t) start, - &first_group, &first_block); - ext3_get_group_no_and_offset(sb, (ext3_fsblk_t) end, - &last_group, &last_block); - - /* end now represents the last block to discard in this group */ - end = EXT3_BLOCKS_PER_GROUP(sb) - 1; - - for (group = first_group; group <= last_group; group++) { - gdp = ext3_get_group_desc(sb, group, NULL); - if (!gdp) - break; - - /* - * For all the groups except the last one, last block will - * always be EXT3_BLOCKS_PER_GROUP(sb)-1, so we only need to - * change it for the last group, note that last_block is - * already computed earlier by ext3_get_group_no_and_offset() - */ - if (group == last_group) - end = last_block; - - if (le16_to_cpu(gdp->bg_free_blocks_count) >= minlen) { - ret = ext3_trim_all_free(sb, group, first_block, - end, minlen); - if (ret < 0) - break; - trimmed += ret; - } - - /* - * For every group except the first one, we are sure - * that the first block to discard will be block #0. - */ - first_block = 0; - } - - if (ret > 0) - ret = 0; - -out: - range->len = trimmed * sb->s_blocksize; - return ret; -} diff --git a/fs/ext3/bitmap.c b/fs/ext3/bitmap.c deleted file mode 100644 index ef9c643e8e9d5f..00000000000000 --- a/fs/ext3/bitmap.c +++ /dev/null @@ -1,20 +0,0 @@ -/* - * linux/fs/ext3/bitmap.c - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - */ - -#include "ext3.h" - -#ifdef EXT3FS_DEBUG - -unsigned long ext3_count_free (struct buffer_head * map, unsigned int numchars) -{ - return numchars * BITS_PER_BYTE - memweight(map->b_data, numchars); -} - -#endif /* EXT3FS_DEBUG */ - diff --git a/fs/ext3/dir.c b/fs/ext3/dir.c deleted file mode 100644 index 17742eed2c16f2..00000000000000 --- a/fs/ext3/dir.c +++ /dev/null @@ -1,537 +0,0 @@ -/* - * linux/fs/ext3/dir.c - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * - * from - * - * linux/fs/minix/dir.c - * - * Copyright (C) 1991, 1992 Linus Torvalds - * - * ext3 directory handling functions - * - * Big-endian to little-endian byte-swapping/bitmaps by - * David S. Miller (davem@caip.rutgers.edu), 1995 - * - * Hash Tree Directory indexing (c) 2001 Daniel Phillips - * - */ - -#include -#include "ext3.h" - -static unsigned char ext3_filetype_table[] = { - DT_UNKNOWN, DT_REG, DT_DIR, DT_CHR, DT_BLK, DT_FIFO, DT_SOCK, DT_LNK -}; - -static int ext3_dx_readdir(struct file *, struct dir_context *); - -static unsigned char get_dtype(struct super_block *sb, int filetype) -{ - if (!EXT3_HAS_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_FILETYPE) || - (filetype >= EXT3_FT_MAX)) - return DT_UNKNOWN; - - return (ext3_filetype_table[filetype]); -} - -/** - * Check if the given dir-inode refers to an htree-indexed directory - * (or a directory which could potentially get converted to use htree - * indexing). - * - * Return 1 if it is a dx dir, 0 if not - */ -static int is_dx_dir(struct inode *inode) -{ - struct super_block *sb = inode->i_sb; - - if (EXT3_HAS_COMPAT_FEATURE(inode->i_sb, - EXT3_FEATURE_COMPAT_DIR_INDEX) && - ((EXT3_I(inode)->i_flags & EXT3_INDEX_FL) || - ((inode->i_size >> sb->s_blocksize_bits) == 1))) - return 1; - - return 0; -} - -int ext3_check_dir_entry (const char * function, struct inode * dir, - struct ext3_dir_entry_2 * de, - struct buffer_head * bh, - unsigned long offset) -{ - const char * error_msg = NULL; - const int rlen = ext3_rec_len_from_disk(de->rec_len); - - if (unlikely(rlen < EXT3_DIR_REC_LEN(1))) - error_msg = "rec_len is smaller than minimal"; - else if (unlikely(rlen % 4 != 0)) - error_msg = "rec_len % 4 != 0"; - else if (unlikely(rlen < EXT3_DIR_REC_LEN(de->name_len))) - error_msg = "rec_len is too small for name_len"; - else if (unlikely((((char *) de - bh->b_data) + rlen > dir->i_sb->s_blocksize))) - error_msg = "directory entry across blocks"; - else if (unlikely(le32_to_cpu(de->inode) > - le32_to_cpu(EXT3_SB(dir->i_sb)->s_es->s_inodes_count))) - error_msg = "inode out of bounds"; - - if (unlikely(error_msg != NULL)) - ext3_error (dir->i_sb, function, - "bad entry in directory #%lu: %s - " - "offset=%lu, inode=%lu, rec_len=%d, name_len=%d", - dir->i_ino, error_msg, offset, - (unsigned long) le32_to_cpu(de->inode), - rlen, de->name_len); - - return error_msg == NULL ? 1 : 0; -} - -static int ext3_readdir(struct file *file, struct dir_context *ctx) -{ - unsigned long offset; - int i; - struct ext3_dir_entry_2 *de; - int err; - struct inode *inode = file_inode(file); - struct super_block *sb = inode->i_sb; - int dir_has_error = 0; - - if (is_dx_dir(inode)) { - err = ext3_dx_readdir(file, ctx); - if (err != ERR_BAD_DX_DIR) - return err; - /* - * We don't set the inode dirty flag since it's not - * critical that it get flushed back to the disk. - */ - EXT3_I(inode)->i_flags &= ~EXT3_INDEX_FL; - } - offset = ctx->pos & (sb->s_blocksize - 1); - - while (ctx->pos < inode->i_size) { - unsigned long blk = ctx->pos >> EXT3_BLOCK_SIZE_BITS(sb); - struct buffer_head map_bh; - struct buffer_head *bh = NULL; - - map_bh.b_state = 0; - err = ext3_get_blocks_handle(NULL, inode, blk, 1, &map_bh, 0); - if (err > 0) { - pgoff_t index = map_bh.b_blocknr >> - (PAGE_CACHE_SHIFT - inode->i_blkbits); - if (!ra_has_index(&file->f_ra, index)) - page_cache_sync_readahead( - sb->s_bdev->bd_inode->i_mapping, - &file->f_ra, file, - index, 1); - file->f_ra.prev_pos = (loff_t)index << PAGE_CACHE_SHIFT; - bh = ext3_bread(NULL, inode, blk, 0, &err); - } - - /* - * We ignore I/O errors on directories so users have a chance - * of recovering data when there's a bad sector - */ - if (!bh) { - if (!dir_has_error) { - ext3_error(sb, __func__, "directory #%lu " - "contains a hole at offset %lld", - inode->i_ino, ctx->pos); - dir_has_error = 1; - } - /* corrupt size? Maybe no more blocks to read */ - if (ctx->pos > inode->i_blocks << 9) - break; - ctx->pos += sb->s_blocksize - offset; - continue; - } - - /* If the dir block has changed since the last call to - * readdir(2), then we might be pointing to an invalid - * dirent right now. Scan from the start of the block - * to make sure. */ - if (offset && file->f_version != inode->i_version) { - for (i = 0; i < sb->s_blocksize && i < offset; ) { - de = (struct ext3_dir_entry_2 *) - (bh->b_data + i); - /* It's too expensive to do a full - * dirent test each time round this - * loop, but we do have to test at - * least that it is non-zero. A - * failure will be detected in the - * dirent test below. */ - if (ext3_rec_len_from_disk(de->rec_len) < - EXT3_DIR_REC_LEN(1)) - break; - i += ext3_rec_len_from_disk(de->rec_len); - } - offset = i; - ctx->pos = (ctx->pos & ~(sb->s_blocksize - 1)) - | offset; - file->f_version = inode->i_version; - } - - while (ctx->pos < inode->i_size - && offset < sb->s_blocksize) { - de = (struct ext3_dir_entry_2 *) (bh->b_data + offset); - if (!ext3_check_dir_entry ("ext3_readdir", inode, de, - bh, offset)) { - /* On error, skip the to the - next block. */ - ctx->pos = (ctx->pos | - (sb->s_blocksize - 1)) + 1; - break; - } - offset += ext3_rec_len_from_disk(de->rec_len); - if (le32_to_cpu(de->inode)) { - if (!dir_emit(ctx, de->name, de->name_len, - le32_to_cpu(de->inode), - get_dtype(sb, de->file_type))) { - brelse(bh); - return 0; - } - } - ctx->pos += ext3_rec_len_from_disk(de->rec_len); - } - offset = 0; - brelse (bh); - if (ctx->pos < inode->i_size) - if (!dir_relax(inode)) - return 0; - } - return 0; -} - -static inline int is_32bit_api(void) -{ -#ifdef CONFIG_COMPAT - return is_compat_task(); -#else - return (BITS_PER_LONG == 32); -#endif -} - -/* - * These functions convert from the major/minor hash to an f_pos - * value for dx directories - * - * Upper layer (for example NFS) should specify FMODE_32BITHASH or - * FMODE_64BITHASH explicitly. On the other hand, we allow ext3 to be mounted - * directly on both 32-bit and 64-bit nodes, under such case, neither - * FMODE_32BITHASH nor FMODE_64BITHASH is specified. - */ -static inline loff_t hash2pos(struct file *filp, __u32 major, __u32 minor) -{ - if ((filp->f_mode & FMODE_32BITHASH) || - (!(filp->f_mode & FMODE_64BITHASH) && is_32bit_api())) - return major >> 1; - else - return ((__u64)(major >> 1) << 32) | (__u64)minor; -} - -static inline __u32 pos2maj_hash(struct file *filp, loff_t pos) -{ - if ((filp->f_mode & FMODE_32BITHASH) || - (!(filp->f_mode & FMODE_64BITHASH) && is_32bit_api())) - return (pos << 1) & 0xffffffff; - else - return ((pos >> 32) << 1) & 0xffffffff; -} - -static inline __u32 pos2min_hash(struct file *filp, loff_t pos) -{ - if ((filp->f_mode & FMODE_32BITHASH) || - (!(filp->f_mode & FMODE_64BITHASH) && is_32bit_api())) - return 0; - else - return pos & 0xffffffff; -} - -/* - * Return 32- or 64-bit end-of-file for dx directories - */ -static inline loff_t ext3_get_htree_eof(struct file *filp) -{ - if ((filp->f_mode & FMODE_32BITHASH) || - (!(filp->f_mode & FMODE_64BITHASH) && is_32bit_api())) - return EXT3_HTREE_EOF_32BIT; - else - return EXT3_HTREE_EOF_64BIT; -} - - -/* - * ext3_dir_llseek() calls generic_file_llseek[_size]() to handle both - * non-htree and htree directories, where the "offset" is in terms - * of the filename hash value instead of the byte offset. - * - * Because we may return a 64-bit hash that is well beyond s_maxbytes, - * we need to pass the max hash as the maximum allowable offset in - * the htree directory case. - * - * NOTE: offsets obtained *before* ext3_set_inode_flag(dir, EXT3_INODE_INDEX) - * will be invalid once the directory was converted into a dx directory - */ -static loff_t ext3_dir_llseek(struct file *file, loff_t offset, int whence) -{ - struct inode *inode = file->f_mapping->host; - int dx_dir = is_dx_dir(inode); - loff_t htree_max = ext3_get_htree_eof(file); - - if (likely(dx_dir)) - return generic_file_llseek_size(file, offset, whence, - htree_max, htree_max); - else - return generic_file_llseek(file, offset, whence); -} - -/* - * This structure holds the nodes of the red-black tree used to store - * the directory entry in hash order. - */ -struct fname { - __u32 hash; - __u32 minor_hash; - struct rb_node rb_hash; - struct fname *next; - __u32 inode; - __u8 name_len; - __u8 file_type; - char name[0]; -}; - -/* - * This functoin implements a non-recursive way of freeing all of the - * nodes in the red-black tree. - */ -static void free_rb_tree_fname(struct rb_root *root) -{ - struct fname *fname, *next; - - rbtree_postorder_for_each_entry_safe(fname, next, root, rb_hash) - do { - struct fname *old = fname; - fname = fname->next; - kfree(old); - } while (fname); - - *root = RB_ROOT; -} - -static struct dir_private_info *ext3_htree_create_dir_info(struct file *filp, - loff_t pos) -{ - struct dir_private_info *p; - - p = kzalloc(sizeof(struct dir_private_info), GFP_KERNEL); - if (!p) - return NULL; - p->curr_hash = pos2maj_hash(filp, pos); - p->curr_minor_hash = pos2min_hash(filp, pos); - return p; -} - -void ext3_htree_free_dir_info(struct dir_private_info *p) -{ - free_rb_tree_fname(&p->root); - kfree(p); -} - -/* - * Given a directory entry, enter it into the fname rb tree. - */ -int ext3_htree_store_dirent(struct file *dir_file, __u32 hash, - __u32 minor_hash, - struct ext3_dir_entry_2 *dirent) -{ - struct rb_node **p, *parent = NULL; - struct fname * fname, *new_fn; - struct dir_private_info *info; - int len; - - info = (struct dir_private_info *) dir_file->private_data; - p = &info->root.rb_node; - - /* Create and allocate the fname structure */ - len = sizeof(struct fname) + dirent->name_len + 1; - new_fn = kzalloc(len, GFP_KERNEL); - if (!new_fn) - return -ENOMEM; - new_fn->hash = hash; - new_fn->minor_hash = minor_hash; - new_fn->inode = le32_to_cpu(dirent->inode); - new_fn->name_len = dirent->name_len; - new_fn->file_type = dirent->file_type; - memcpy(new_fn->name, dirent->name, dirent->name_len); - new_fn->name[dirent->name_len] = 0; - - while (*p) { - parent = *p; - fname = rb_entry(parent, struct fname, rb_hash); - - /* - * If the hash and minor hash match up, then we put - * them on a linked list. This rarely happens... - */ - if ((new_fn->hash == fname->hash) && - (new_fn->minor_hash == fname->minor_hash)) { - new_fn->next = fname->next; - fname->next = new_fn; - return 0; - } - - if (new_fn->hash < fname->hash) - p = &(*p)->rb_left; - else if (new_fn->hash > fname->hash) - p = &(*p)->rb_right; - else if (new_fn->minor_hash < fname->minor_hash) - p = &(*p)->rb_left; - else /* if (new_fn->minor_hash > fname->minor_hash) */ - p = &(*p)->rb_right; - } - - rb_link_node(&new_fn->rb_hash, parent, p); - rb_insert_color(&new_fn->rb_hash, &info->root); - return 0; -} - - - -/* - * This is a helper function for ext3_dx_readdir. It calls filldir - * for all entres on the fname linked list. (Normally there is only - * one entry on the linked list, unless there are 62 bit hash collisions.) - */ -static bool call_filldir(struct file *file, struct dir_context *ctx, - struct fname *fname) -{ - struct dir_private_info *info = file->private_data; - struct inode *inode = file_inode(file); - struct super_block *sb = inode->i_sb; - - if (!fname) { - printk("call_filldir: called with null fname?!?\n"); - return true; - } - ctx->pos = hash2pos(file, fname->hash, fname->minor_hash); - while (fname) { - if (!dir_emit(ctx, fname->name, fname->name_len, - fname->inode, - get_dtype(sb, fname->file_type))) { - info->extra_fname = fname; - return false; - } - fname = fname->next; - } - return true; -} - -static int ext3_dx_readdir(struct file *file, struct dir_context *ctx) -{ - struct dir_private_info *info = file->private_data; - struct inode *inode = file_inode(file); - struct fname *fname; - int ret; - - if (!info) { - info = ext3_htree_create_dir_info(file, ctx->pos); - if (!info) - return -ENOMEM; - file->private_data = info; - } - - if (ctx->pos == ext3_get_htree_eof(file)) - return 0; /* EOF */ - - /* Some one has messed with f_pos; reset the world */ - if (info->last_pos != ctx->pos) { - free_rb_tree_fname(&info->root); - info->curr_node = NULL; - info->extra_fname = NULL; - info->curr_hash = pos2maj_hash(file, ctx->pos); - info->curr_minor_hash = pos2min_hash(file, ctx->pos); - } - - /* - * If there are any leftover names on the hash collision - * chain, return them first. - */ - if (info->extra_fname) { - if (!call_filldir(file, ctx, info->extra_fname)) - goto finished; - info->extra_fname = NULL; - goto next_node; - } else if (!info->curr_node) - info->curr_node = rb_first(&info->root); - - while (1) { - /* - * Fill the rbtree if we have no more entries, - * or the inode has changed since we last read in the - * cached entries. - */ - if ((!info->curr_node) || - (file->f_version != inode->i_version)) { - info->curr_node = NULL; - free_rb_tree_fname(&info->root); - file->f_version = inode->i_version; - ret = ext3_htree_fill_tree(file, info->curr_hash, - info->curr_minor_hash, - &info->next_hash); - if (ret < 0) - return ret; - if (ret == 0) { - ctx->pos = ext3_get_htree_eof(file); - break; - } - info->curr_node = rb_first(&info->root); - } - - fname = rb_entry(info->curr_node, struct fname, rb_hash); - info->curr_hash = fname->hash; - info->curr_minor_hash = fname->minor_hash; - if (!call_filldir(file, ctx, fname)) - break; - next_node: - info->curr_node = rb_next(info->curr_node); - if (info->curr_node) { - fname = rb_entry(info->curr_node, struct fname, - rb_hash); - info->curr_hash = fname->hash; - info->curr_minor_hash = fname->minor_hash; - } else { - if (info->next_hash == ~0) { - ctx->pos = ext3_get_htree_eof(file); - break; - } - info->curr_hash = info->next_hash; - info->curr_minor_hash = 0; - } - } -finished: - info->last_pos = ctx->pos; - return 0; -} - -static int ext3_release_dir (struct inode * inode, struct file * filp) -{ - if (filp->private_data) - ext3_htree_free_dir_info(filp->private_data); - - return 0; -} - -const struct file_operations ext3_dir_operations = { - .llseek = ext3_dir_llseek, - .read = generic_read_dir, - .iterate = ext3_readdir, - .unlocked_ioctl = ext3_ioctl, -#ifdef CONFIG_COMPAT - .compat_ioctl = ext3_compat_ioctl, -#endif - .fsync = ext3_sync_file, - .release = ext3_release_dir, -}; diff --git a/fs/ext3/ext3.h b/fs/ext3/ext3.h deleted file mode 100644 index f483a80b3fe783..00000000000000 --- a/fs/ext3/ext3.h +++ /dev/null @@ -1,1332 +0,0 @@ -/* - * Written by Stephen C. Tweedie , 1999 - * - * Copyright 1998--1999 Red Hat corp --- All Rights Reserved - * - * This file is part of the Linux kernel and is made available under - * the terms of the GNU General Public License, version 2, or at your - * option, any later version, incorporated herein by reference. - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * - * from - * - * linux/include/linux/minix_fs.h - * - * Copyright (C) 1991, 1992 Linus Torvalds - */ - -#include -#include -#include -#include -#include - -/* - * The second extended filesystem constants/structures - */ - -/* - * Define EXT3FS_DEBUG to produce debug messages - */ -#undef EXT3FS_DEBUG - -/* - * Define EXT3_RESERVATION to reserve data blocks for expanding files - */ -#define EXT3_DEFAULT_RESERVE_BLOCKS 8 -/*max window size: 1024(direct blocks) + 3([t,d]indirect blocks) */ -#define EXT3_MAX_RESERVE_BLOCKS 1027 -#define EXT3_RESERVE_WINDOW_NOT_ALLOCATED 0 - -/* - * Debug code - */ -#ifdef EXT3FS_DEBUG -#define ext3_debug(f, a...) \ - do { \ - printk (KERN_DEBUG "EXT3-fs DEBUG (%s, %d): %s:", \ - __FILE__, __LINE__, __func__); \ - printk (KERN_DEBUG f, ## a); \ - } while (0) -#else -#define ext3_debug(f, a...) do {} while (0) -#endif - -/* - * Special inodes numbers - */ -#define EXT3_BAD_INO 1 /* Bad blocks inode */ -#define EXT3_ROOT_INO 2 /* Root inode */ -#define EXT3_BOOT_LOADER_INO 5 /* Boot loader inode */ -#define EXT3_UNDEL_DIR_INO 6 /* Undelete directory inode */ -#define EXT3_RESIZE_INO 7 /* Reserved group descriptors inode */ -#define EXT3_JOURNAL_INO 8 /* Journal inode */ - -/* First non-reserved inode for old ext3 filesystems */ -#define EXT3_GOOD_OLD_FIRST_INO 11 - -/* - * Maximal count of links to a file - */ -#define EXT3_LINK_MAX 32000 - -/* - * Macro-instructions used to manage several block sizes - */ -#define EXT3_MIN_BLOCK_SIZE 1024 -#define EXT3_MAX_BLOCK_SIZE 65536 -#define EXT3_MIN_BLOCK_LOG_SIZE 10 -#define EXT3_BLOCK_SIZE(s) ((s)->s_blocksize) -#define EXT3_ADDR_PER_BLOCK(s) (EXT3_BLOCK_SIZE(s) / sizeof (__u32)) -#define EXT3_BLOCK_SIZE_BITS(s) ((s)->s_blocksize_bits) -#define EXT3_ADDR_PER_BLOCK_BITS(s) (EXT3_SB(s)->s_addr_per_block_bits) -#define EXT3_INODE_SIZE(s) (EXT3_SB(s)->s_inode_size) -#define EXT3_FIRST_INO(s) (EXT3_SB(s)->s_first_ino) - -/* - * Macro-instructions used to manage fragments - */ -#define EXT3_MIN_FRAG_SIZE 1024 -#define EXT3_MAX_FRAG_SIZE 4096 -#define EXT3_MIN_FRAG_LOG_SIZE 10 -#define EXT3_FRAG_SIZE(s) (EXT3_SB(s)->s_frag_size) -#define EXT3_FRAGS_PER_BLOCK(s) (EXT3_SB(s)->s_frags_per_block) - -/* - * Structure of a blocks group descriptor - */ -struct ext3_group_desc -{ - __le32 bg_block_bitmap; /* Blocks bitmap block */ - __le32 bg_inode_bitmap; /* Inodes bitmap block */ - __le32 bg_inode_table; /* Inodes table block */ - __le16 bg_free_blocks_count; /* Free blocks count */ - __le16 bg_free_inodes_count; /* Free inodes count */ - __le16 bg_used_dirs_count; /* Directories count */ - __u16 bg_pad; - __le32 bg_reserved[3]; -}; - -/* - * Macro-instructions used to manage group descriptors - */ -#define EXT3_BLOCKS_PER_GROUP(s) (EXT3_SB(s)->s_blocks_per_group) -#define EXT3_DESC_PER_BLOCK(s) (EXT3_SB(s)->s_desc_per_block) -#define EXT3_INODES_PER_GROUP(s) (EXT3_SB(s)->s_inodes_per_group) -#define EXT3_DESC_PER_BLOCK_BITS(s) (EXT3_SB(s)->s_desc_per_block_bits) - -/* - * Constants relative to the data blocks - */ -#define EXT3_NDIR_BLOCKS 12 -#define EXT3_IND_BLOCK EXT3_NDIR_BLOCKS -#define EXT3_DIND_BLOCK (EXT3_IND_BLOCK + 1) -#define EXT3_TIND_BLOCK (EXT3_DIND_BLOCK + 1) -#define EXT3_N_BLOCKS (EXT3_TIND_BLOCK + 1) - -/* - * Inode flags - */ -#define EXT3_SECRM_FL 0x00000001 /* Secure deletion */ -#define EXT3_UNRM_FL 0x00000002 /* Undelete */ -#define EXT3_COMPR_FL 0x00000004 /* Compress file */ -#define EXT3_SYNC_FL 0x00000008 /* Synchronous updates */ -#define EXT3_IMMUTABLE_FL 0x00000010 /* Immutable file */ -#define EXT3_APPEND_FL 0x00000020 /* writes to file may only append */ -#define EXT3_NODUMP_FL 0x00000040 /* do not dump file */ -#define EXT3_NOATIME_FL 0x00000080 /* do not update atime */ -/* Reserved for compression usage... */ -#define EXT3_DIRTY_FL 0x00000100 -#define EXT3_COMPRBLK_FL 0x00000200 /* One or more compressed clusters */ -#define EXT3_NOCOMPR_FL 0x00000400 /* Don't compress */ -#define EXT3_ECOMPR_FL 0x00000800 /* Compression error */ -/* End compression flags --- maybe not all used */ -#define EXT3_INDEX_FL 0x00001000 /* hash-indexed directory */ -#define EXT3_IMAGIC_FL 0x00002000 /* AFS directory */ -#define EXT3_JOURNAL_DATA_FL 0x00004000 /* file data should be journaled */ -#define EXT3_NOTAIL_FL 0x00008000 /* file tail should not be merged */ -#define EXT3_DIRSYNC_FL 0x00010000 /* dirsync behaviour (directories only) */ -#define EXT3_TOPDIR_FL 0x00020000 /* Top of directory hierarchies*/ -#define EXT3_RESERVED_FL 0x80000000 /* reserved for ext3 lib */ - -#define EXT3_FL_USER_VISIBLE 0x0003DFFF /* User visible flags */ -#define EXT3_FL_USER_MODIFIABLE 0x000380FF /* User modifiable flags */ - -/* Flags that should be inherited by new inodes from their parent. */ -#define EXT3_FL_INHERITED (EXT3_SECRM_FL | EXT3_UNRM_FL | EXT3_COMPR_FL |\ - EXT3_SYNC_FL | EXT3_NODUMP_FL |\ - EXT3_NOATIME_FL | EXT3_COMPRBLK_FL |\ - EXT3_NOCOMPR_FL | EXT3_JOURNAL_DATA_FL |\ - EXT3_NOTAIL_FL | EXT3_DIRSYNC_FL) - -/* Flags that are appropriate for regular files (all but dir-specific ones). */ -#define EXT3_REG_FLMASK (~(EXT3_DIRSYNC_FL | EXT3_TOPDIR_FL)) - -/* Flags that are appropriate for non-directories/regular files. */ -#define EXT3_OTHER_FLMASK (EXT3_NODUMP_FL | EXT3_NOATIME_FL) - -/* Mask out flags that are inappropriate for the given type of inode. */ -static inline __u32 ext3_mask_flags(umode_t mode, __u32 flags) -{ - if (S_ISDIR(mode)) - return flags; - else if (S_ISREG(mode)) - return flags & EXT3_REG_FLMASK; - else - return flags & EXT3_OTHER_FLMASK; -} - -/* Used to pass group descriptor data when online resize is done */ -struct ext3_new_group_input { - __u32 group; /* Group number for this data */ - __u32 block_bitmap; /* Absolute block number of block bitmap */ - __u32 inode_bitmap; /* Absolute block number of inode bitmap */ - __u32 inode_table; /* Absolute block number of inode table start */ - __u32 blocks_count; /* Total number of blocks in this group */ - __u16 reserved_blocks; /* Number of reserved blocks in this group */ - __u16 unused; -}; - -/* The struct ext3_new_group_input in kernel space, with free_blocks_count */ -struct ext3_new_group_data { - __u32 group; - __u32 block_bitmap; - __u32 inode_bitmap; - __u32 inode_table; - __u32 blocks_count; - __u16 reserved_blocks; - __u16 unused; - __u32 free_blocks_count; -}; - - -/* - * ioctl commands - */ -#define EXT3_IOC_GETFLAGS FS_IOC_GETFLAGS -#define EXT3_IOC_SETFLAGS FS_IOC_SETFLAGS -#define EXT3_IOC_GETVERSION _IOR('f', 3, long) -#define EXT3_IOC_SETVERSION _IOW('f', 4, long) -#define EXT3_IOC_GROUP_EXTEND _IOW('f', 7, unsigned long) -#define EXT3_IOC_GROUP_ADD _IOW('f', 8,struct ext3_new_group_input) -#define EXT3_IOC_GETVERSION_OLD FS_IOC_GETVERSION -#define EXT3_IOC_SETVERSION_OLD FS_IOC_SETVERSION -#ifdef CONFIG_JBD_DEBUG -#define EXT3_IOC_WAIT_FOR_READONLY _IOR('f', 99, long) -#endif -#define EXT3_IOC_GETRSVSZ _IOR('f', 5, long) -#define EXT3_IOC_SETRSVSZ _IOW('f', 6, long) - -/* - * ioctl commands in 32 bit emulation - */ -#define EXT3_IOC32_GETFLAGS FS_IOC32_GETFLAGS -#define EXT3_IOC32_SETFLAGS FS_IOC32_SETFLAGS -#define EXT3_IOC32_GETVERSION _IOR('f', 3, int) -#define EXT3_IOC32_SETVERSION _IOW('f', 4, int) -#define EXT3_IOC32_GETRSVSZ _IOR('f', 5, int) -#define EXT3_IOC32_SETRSVSZ _IOW('f', 6, int) -#define EXT3_IOC32_GROUP_EXTEND _IOW('f', 7, unsigned int) -#ifdef CONFIG_JBD_DEBUG -#define EXT3_IOC32_WAIT_FOR_READONLY _IOR('f', 99, int) -#endif -#define EXT3_IOC32_GETVERSION_OLD FS_IOC32_GETVERSION -#define EXT3_IOC32_SETVERSION_OLD FS_IOC32_SETVERSION - -/* Number of supported quota types */ -#define EXT3_MAXQUOTAS 2 - -/* - * Mount options - */ -struct ext3_mount_options { - unsigned long s_mount_opt; - kuid_t s_resuid; - kgid_t s_resgid; - unsigned long s_commit_interval; -#ifdef CONFIG_QUOTA - int s_jquota_fmt; - char *s_qf_names[EXT3_MAXQUOTAS]; -#endif -}; - -/* - * Structure of an inode on the disk - */ -struct ext3_inode { - __le16 i_mode; /* File mode */ - __le16 i_uid; /* Low 16 bits of Owner Uid */ - __le32 i_size; /* Size in bytes */ - __le32 i_atime; /* Access time */ - __le32 i_ctime; /* Creation time */ - __le32 i_mtime; /* Modification time */ - __le32 i_dtime; /* Deletion Time */ - __le16 i_gid; /* Low 16 bits of Group Id */ - __le16 i_links_count; /* Links count */ - __le32 i_blocks; /* Blocks count */ - __le32 i_flags; /* File flags */ - union { - struct { - __u32 l_i_reserved1; - } linux1; - struct { - __u32 h_i_translator; - } hurd1; - struct { - __u32 m_i_reserved1; - } masix1; - } osd1; /* OS dependent 1 */ - __le32 i_block[EXT3_N_BLOCKS];/* Pointers to blocks */ - __le32 i_generation; /* File version (for NFS) */ - __le32 i_file_acl; /* File ACL */ - __le32 i_dir_acl; /* Directory ACL */ - __le32 i_faddr; /* Fragment address */ - union { - struct { - __u8 l_i_frag; /* Fragment number */ - __u8 l_i_fsize; /* Fragment size */ - __u16 i_pad1; - __le16 l_i_uid_high; /* these 2 fields */ - __le16 l_i_gid_high; /* were reserved2[0] */ - __u32 l_i_reserved2; - } linux2; - struct { - __u8 h_i_frag; /* Fragment number */ - __u8 h_i_fsize; /* Fragment size */ - __u16 h_i_mode_high; - __u16 h_i_uid_high; - __u16 h_i_gid_high; - __u32 h_i_author; - } hurd2; - struct { - __u8 m_i_frag; /* Fragment number */ - __u8 m_i_fsize; /* Fragment size */ - __u16 m_pad1; - __u32 m_i_reserved2[2]; - } masix2; - } osd2; /* OS dependent 2 */ - __le16 i_extra_isize; - __le16 i_pad1; -}; - -#define i_size_high i_dir_acl - -#define i_reserved1 osd1.linux1.l_i_reserved1 -#define i_frag osd2.linux2.l_i_frag -#define i_fsize osd2.linux2.l_i_fsize -#define i_uid_low i_uid -#define i_gid_low i_gid -#define i_uid_high osd2.linux2.l_i_uid_high -#define i_gid_high osd2.linux2.l_i_gid_high -#define i_reserved2 osd2.linux2.l_i_reserved2 - -/* - * File system states - */ -#define EXT3_VALID_FS 0x0001 /* Unmounted cleanly */ -#define EXT3_ERROR_FS 0x0002 /* Errors detected */ -#define EXT3_ORPHAN_FS 0x0004 /* Orphans being recovered */ - -/* - * Misc. filesystem flags - */ -#define EXT2_FLAGS_SIGNED_HASH 0x0001 /* Signed dirhash in use */ -#define EXT2_FLAGS_UNSIGNED_HASH 0x0002 /* Unsigned dirhash in use */ -#define EXT2_FLAGS_TEST_FILESYS 0x0004 /* to test development code */ - -/* - * Mount flags - */ -#define EXT3_MOUNT_CHECK 0x00001 /* Do mount-time checks */ -/* EXT3_MOUNT_OLDALLOC was there */ -#define EXT3_MOUNT_GRPID 0x00004 /* Create files with directory's group */ -#define EXT3_MOUNT_DEBUG 0x00008 /* Some debugging messages */ -#define EXT3_MOUNT_ERRORS_CONT 0x00010 /* Continue on errors */ -#define EXT3_MOUNT_ERRORS_RO 0x00020 /* Remount fs ro on errors */ -#define EXT3_MOUNT_ERRORS_PANIC 0x00040 /* Panic on errors */ -#define EXT3_MOUNT_MINIX_DF 0x00080 /* Mimics the Minix statfs */ -#define EXT3_MOUNT_NOLOAD 0x00100 /* Don't use existing journal*/ -#define EXT3_MOUNT_ABORT 0x00200 /* Fatal error detected */ -#define EXT3_MOUNT_DATA_FLAGS 0x00C00 /* Mode for data writes: */ -#define EXT3_MOUNT_JOURNAL_DATA 0x00400 /* Write data to journal */ -#define EXT3_MOUNT_ORDERED_DATA 0x00800 /* Flush data before commit */ -#define EXT3_MOUNT_WRITEBACK_DATA 0x00C00 /* No data ordering */ -#define EXT3_MOUNT_UPDATE_JOURNAL 0x01000 /* Update the journal format */ -#define EXT3_MOUNT_NO_UID32 0x02000 /* Disable 32-bit UIDs */ -#define EXT3_MOUNT_XATTR_USER 0x04000 /* Extended user attributes */ -#define EXT3_MOUNT_POSIX_ACL 0x08000 /* POSIX Access Control Lists */ -#define EXT3_MOUNT_RESERVATION 0x10000 /* Preallocation */ -#define EXT3_MOUNT_BARRIER 0x20000 /* Use block barriers */ -#define EXT3_MOUNT_QUOTA 0x80000 /* Some quota option set */ -#define EXT3_MOUNT_USRQUOTA 0x100000 /* "old" user quota */ -#define EXT3_MOUNT_GRPQUOTA 0x200000 /* "old" group quota */ -#define EXT3_MOUNT_DATA_ERR_ABORT 0x400000 /* Abort on file data write - * error in ordered mode */ - -/* Compatibility, for having both ext2_fs.h and ext3_fs.h included at once */ -#ifndef _LINUX_EXT2_FS_H -#define clear_opt(o, opt) o &= ~EXT3_MOUNT_##opt -#define set_opt(o, opt) o |= EXT3_MOUNT_##opt -#define test_opt(sb, opt) (EXT3_SB(sb)->s_mount_opt & \ - EXT3_MOUNT_##opt) -#else -#define EXT2_MOUNT_NOLOAD EXT3_MOUNT_NOLOAD -#define EXT2_MOUNT_ABORT EXT3_MOUNT_ABORT -#define EXT2_MOUNT_DATA_FLAGS EXT3_MOUNT_DATA_FLAGS -#endif - -#define ext3_set_bit __set_bit_le -#define ext3_set_bit_atomic ext2_set_bit_atomic -#define ext3_clear_bit __clear_bit_le -#define ext3_clear_bit_atomic ext2_clear_bit_atomic -#define ext3_test_bit test_bit_le -#define ext3_find_next_zero_bit find_next_zero_bit_le - -/* - * Maximal mount counts between two filesystem checks - */ -#define EXT3_DFL_MAX_MNT_COUNT 20 /* Allow 20 mounts */ -#define EXT3_DFL_CHECKINTERVAL 0 /* Don't use interval check */ - -/* - * Behaviour when detecting errors - */ -#define EXT3_ERRORS_CONTINUE 1 /* Continue execution */ -#define EXT3_ERRORS_RO 2 /* Remount fs read-only */ -#define EXT3_ERRORS_PANIC 3 /* Panic */ -#define EXT3_ERRORS_DEFAULT EXT3_ERRORS_CONTINUE - -/* - * Structure of the super block - */ -struct ext3_super_block { -/*00*/ __le32 s_inodes_count; /* Inodes count */ - __le32 s_blocks_count; /* Blocks count */ - __le32 s_r_blocks_count; /* Reserved blocks count */ - __le32 s_free_blocks_count; /* Free blocks count */ -/*10*/ __le32 s_free_inodes_count; /* Free inodes count */ - __le32 s_first_data_block; /* First Data Block */ - __le32 s_log_block_size; /* Block size */ - __le32 s_log_frag_size; /* Fragment size */ -/*20*/ __le32 s_blocks_per_group; /* # Blocks per group */ - __le32 s_frags_per_group; /* # Fragments per group */ - __le32 s_inodes_per_group; /* # Inodes per group */ - __le32 s_mtime; /* Mount time */ -/*30*/ __le32 s_wtime; /* Write time */ - __le16 s_mnt_count; /* Mount count */ - __le16 s_max_mnt_count; /* Maximal mount count */ - __le16 s_magic; /* Magic signature */ - __le16 s_state; /* File system state */ - __le16 s_errors; /* Behaviour when detecting errors */ - __le16 s_minor_rev_level; /* minor revision level */ -/*40*/ __le32 s_lastcheck; /* time of last check */ - __le32 s_checkinterval; /* max. time between checks */ - __le32 s_creator_os; /* OS */ - __le32 s_rev_level; /* Revision level */ -/*50*/ __le16 s_def_resuid; /* Default uid for reserved blocks */ - __le16 s_def_resgid; /* Default gid for reserved blocks */ - /* - * These fields are for EXT3_DYNAMIC_REV superblocks only. - * - * Note: the difference between the compatible feature set and - * the incompatible feature set is that if there is a bit set - * in the incompatible feature set that the kernel doesn't - * know about, it should refuse to mount the filesystem. - * - * e2fsck's requirements are more strict; if it doesn't know - * about a feature in either the compatible or incompatible - * feature set, it must abort and not try to meddle with - * things it doesn't understand... - */ - __le32 s_first_ino; /* First non-reserved inode */ - __le16 s_inode_size; /* size of inode structure */ - __le16 s_block_group_nr; /* block group # of this superblock */ - __le32 s_feature_compat; /* compatible feature set */ -/*60*/ __le32 s_feature_incompat; /* incompatible feature set */ - __le32 s_feature_ro_compat; /* readonly-compatible feature set */ -/*68*/ __u8 s_uuid[16]; /* 128-bit uuid for volume */ -/*78*/ char s_volume_name[16]; /* volume name */ -/*88*/ char s_last_mounted[64]; /* directory where last mounted */ -/*C8*/ __le32 s_algorithm_usage_bitmap; /* For compression */ - /* - * Performance hints. Directory preallocation should only - * happen if the EXT3_FEATURE_COMPAT_DIR_PREALLOC flag is on. - */ - __u8 s_prealloc_blocks; /* Nr of blocks to try to preallocate*/ - __u8 s_prealloc_dir_blocks; /* Nr to preallocate for dirs */ - __le16 s_reserved_gdt_blocks; /* Per group desc for online growth */ - /* - * Journaling support valid if EXT3_FEATURE_COMPAT_HAS_JOURNAL set. - */ -/*D0*/ __u8 s_journal_uuid[16]; /* uuid of journal superblock */ -/*E0*/ __le32 s_journal_inum; /* inode number of journal file */ - __le32 s_journal_dev; /* device number of journal file */ - __le32 s_last_orphan; /* start of list of inodes to delete */ - __le32 s_hash_seed[4]; /* HTREE hash seed */ - __u8 s_def_hash_version; /* Default hash version to use */ - __u8 s_reserved_char_pad; - __u16 s_reserved_word_pad; - __le32 s_default_mount_opts; - __le32 s_first_meta_bg; /* First metablock block group */ - __le32 s_mkfs_time; /* When the filesystem was created */ - __le32 s_jnl_blocks[17]; /* Backup of the journal inode */ - /* 64bit support valid if EXT4_FEATURE_COMPAT_64BIT */ -/*150*/ __le32 s_blocks_count_hi; /* Blocks count */ - __le32 s_r_blocks_count_hi; /* Reserved blocks count */ - __le32 s_free_blocks_count_hi; /* Free blocks count */ - __le16 s_min_extra_isize; /* All inodes have at least # bytes */ - __le16 s_want_extra_isize; /* New inodes should reserve # bytes */ - __le32 s_flags; /* Miscellaneous flags */ - __le16 s_raid_stride; /* RAID stride */ - __le16 s_mmp_interval; /* # seconds to wait in MMP checking */ - __le64 s_mmp_block; /* Block for multi-mount protection */ - __le32 s_raid_stripe_width; /* blocks on all data disks (N*stride)*/ - __u8 s_log_groups_per_flex; /* FLEX_BG group size */ - __u8 s_reserved_char_pad2; - __le16 s_reserved_pad; - __u32 s_reserved[162]; /* Padding to the end of the block */ -}; - -/* data type for block offset of block group */ -typedef int ext3_grpblk_t; - -/* data type for filesystem-wide blocks number */ -typedef unsigned long ext3_fsblk_t; - -#define E3FSBLK "%lu" - -struct ext3_reserve_window { - ext3_fsblk_t _rsv_start; /* First byte reserved */ - ext3_fsblk_t _rsv_end; /* Last byte reserved or 0 */ -}; - -struct ext3_reserve_window_node { - struct rb_node rsv_node; - __u32 rsv_goal_size; - __u32 rsv_alloc_hit; - struct ext3_reserve_window rsv_window; -}; - -struct ext3_block_alloc_info { - /* information about reservation window */ - struct ext3_reserve_window_node rsv_window_node; - /* - * was i_next_alloc_block in ext3_inode_info - * is the logical (file-relative) number of the - * most-recently-allocated block in this file. - * We use this for detecting linearly ascending allocation requests. - */ - __u32 last_alloc_logical_block; - /* - * Was i_next_alloc_goal in ext3_inode_info - * is the *physical* companion to i_next_alloc_block. - * it the physical block number of the block which was most-recentl - * allocated to this file. This give us the goal (target) for the next - * allocation when we detect linearly ascending requests. - */ - ext3_fsblk_t last_alloc_physical_block; -}; - -#define rsv_start rsv_window._rsv_start -#define rsv_end rsv_window._rsv_end - -/* - * third extended file system inode data in memory - */ -struct ext3_inode_info { - __le32 i_data[15]; /* unconverted */ - __u32 i_flags; -#ifdef EXT3_FRAGMENTS - __u32 i_faddr; - __u8 i_frag_no; - __u8 i_frag_size; -#endif - ext3_fsblk_t i_file_acl; - __u32 i_dir_acl; - __u32 i_dtime; - - /* - * i_block_group is the number of the block group which contains - * this file's inode. Constant across the lifetime of the inode, - * it is ued for making block allocation decisions - we try to - * place a file's data blocks near its inode block, and new inodes - * near to their parent directory's inode. - */ - __u32 i_block_group; - unsigned long i_state_flags; /* Dynamic state flags for ext3 */ - - /* block reservation info */ - struct ext3_block_alloc_info *i_block_alloc_info; - - __u32 i_dir_start_lookup; -#ifdef CONFIG_EXT3_FS_XATTR - /* - * Extended attributes can be read independently of the main file - * data. Taking i_mutex even when reading would cause contention - * between readers of EAs and writers of regular file data, so - * instead we synchronize on xattr_sem when reading or changing - * EAs. - */ - struct rw_semaphore xattr_sem; -#endif - - struct list_head i_orphan; /* unlinked but open inodes */ - - /* - * i_disksize keeps track of what the inode size is ON DISK, not - * in memory. During truncate, i_size is set to the new size by - * the VFS prior to calling ext3_truncate(), but the filesystem won't - * set i_disksize to 0 until the truncate is actually under way. - * - * The intent is that i_disksize always represents the blocks which - * are used by this file. This allows recovery to restart truncate - * on orphans if we crash during truncate. We actually write i_disksize - * into the on-disk inode when writing inodes out, instead of i_size. - * - * The only time when i_disksize and i_size may be different is when - * a truncate is in progress. The only things which change i_disksize - * are ext3_get_block (growth) and ext3_truncate (shrinkth). - */ - loff_t i_disksize; - - /* on-disk additional length */ - __u16 i_extra_isize; - - /* - * truncate_mutex is for serialising ext3_truncate() against - * ext3_getblock(). In the 2.4 ext2 design, great chunks of inode's - * data tree are chopped off during truncate. We can't do that in - * ext3 because whenever we perform intermediate commits during - * truncate, the inode and all the metadata blocks *must* be in a - * consistent state which allows truncation of the orphans to restart - * during recovery. Hence we must fix the get_block-vs-truncate race - * by other means, so we have truncate_mutex. - */ - struct mutex truncate_mutex; - - /* - * Transactions that contain inode's metadata needed to complete - * fsync and fdatasync, respectively. - */ - atomic_t i_sync_tid; - atomic_t i_datasync_tid; - -#ifdef CONFIG_QUOTA - struct dquot *i_dquot[MAXQUOTAS]; -#endif - - struct inode vfs_inode; -}; - -/* - * third extended-fs super-block data in memory - */ -struct ext3_sb_info { - unsigned long s_frag_size; /* Size of a fragment in bytes */ - unsigned long s_frags_per_block;/* Number of fragments per block */ - unsigned long s_inodes_per_block;/* Number of inodes per block */ - unsigned long s_frags_per_group;/* Number of fragments in a group */ - unsigned long s_blocks_per_group;/* Number of blocks in a group */ - unsigned long s_inodes_per_group;/* Number of inodes in a group */ - unsigned long s_itb_per_group; /* Number of inode table blocks per group */ - unsigned long s_gdb_count; /* Number of group descriptor blocks */ - unsigned long s_desc_per_block; /* Number of group descriptors per block */ - unsigned long s_groups_count; /* Number of groups in the fs */ - unsigned long s_overhead_last; /* Last calculated overhead */ - unsigned long s_blocks_last; /* Last seen block count */ - struct buffer_head * s_sbh; /* Buffer containing the super block */ - struct ext3_super_block * s_es; /* Pointer to the super block in the buffer */ - struct buffer_head ** s_group_desc; - unsigned long s_mount_opt; - ext3_fsblk_t s_sb_block; - kuid_t s_resuid; - kgid_t s_resgid; - unsigned short s_mount_state; - unsigned short s_pad; - int s_addr_per_block_bits; - int s_desc_per_block_bits; - int s_inode_size; - int s_first_ino; - spinlock_t s_next_gen_lock; - u32 s_next_generation; - u32 s_hash_seed[4]; - int s_def_hash_version; - int s_hash_unsigned; /* 3 if hash should be signed, 0 if not */ - struct percpu_counter s_freeblocks_counter; - struct percpu_counter s_freeinodes_counter; - struct percpu_counter s_dirs_counter; - struct blockgroup_lock *s_blockgroup_lock; - - /* root of the per fs reservation window tree */ - spinlock_t s_rsv_window_lock; - struct rb_root s_rsv_window_root; - struct ext3_reserve_window_node s_rsv_window_head; - - /* Journaling */ - struct inode * s_journal_inode; - struct journal_s * s_journal; - struct list_head s_orphan; - struct mutex s_orphan_lock; - struct mutex s_resize_lock; - unsigned long s_commit_interval; - struct block_device *journal_bdev; -#ifdef CONFIG_QUOTA - char *s_qf_names[EXT3_MAXQUOTAS]; /* Names of quota files with journalled quota */ - int s_jquota_fmt; /* Format of quota to use */ -#endif -}; - -static inline spinlock_t * -sb_bgl_lock(struct ext3_sb_info *sbi, unsigned int block_group) -{ - return bgl_lock_ptr(sbi->s_blockgroup_lock, block_group); -} - -static inline struct ext3_sb_info * EXT3_SB(struct super_block *sb) -{ - return sb->s_fs_info; -} -static inline struct ext3_inode_info *EXT3_I(struct inode *inode) -{ - return container_of(inode, struct ext3_inode_info, vfs_inode); -} - -static inline int ext3_valid_inum(struct super_block *sb, unsigned long ino) -{ - return ino == EXT3_ROOT_INO || - ino == EXT3_JOURNAL_INO || - ino == EXT3_RESIZE_INO || - (ino >= EXT3_FIRST_INO(sb) && - ino <= le32_to_cpu(EXT3_SB(sb)->s_es->s_inodes_count)); -} - -/* - * Inode dynamic state flags - */ -enum { - EXT3_STATE_JDATA, /* journaled data exists */ - EXT3_STATE_NEW, /* inode is newly created */ - EXT3_STATE_XATTR, /* has in-inode xattrs */ - EXT3_STATE_FLUSH_ON_CLOSE, /* flush dirty pages on close */ -}; - -static inline int ext3_test_inode_state(struct inode *inode, int bit) -{ - return test_bit(bit, &EXT3_I(inode)->i_state_flags); -} - -static inline void ext3_set_inode_state(struct inode *inode, int bit) -{ - set_bit(bit, &EXT3_I(inode)->i_state_flags); -} - -static inline void ext3_clear_inode_state(struct inode *inode, int bit) -{ - clear_bit(bit, &EXT3_I(inode)->i_state_flags); -} - -#define NEXT_ORPHAN(inode) EXT3_I(inode)->i_dtime - -/* - * Codes for operating systems - */ -#define EXT3_OS_LINUX 0 -#define EXT3_OS_HURD 1 -#define EXT3_OS_MASIX 2 -#define EXT3_OS_FREEBSD 3 -#define EXT3_OS_LITES 4 - -/* - * Revision levels - */ -#define EXT3_GOOD_OLD_REV 0 /* The good old (original) format */ -#define EXT3_DYNAMIC_REV 1 /* V2 format w/ dynamic inode sizes */ - -#define EXT3_CURRENT_REV EXT3_GOOD_OLD_REV -#define EXT3_MAX_SUPP_REV EXT3_DYNAMIC_REV - -#define EXT3_GOOD_OLD_INODE_SIZE 128 - -/* - * Feature set definitions - */ - -#define EXT3_HAS_COMPAT_FEATURE(sb,mask) \ - ( EXT3_SB(sb)->s_es->s_feature_compat & cpu_to_le32(mask) ) -#define EXT3_HAS_RO_COMPAT_FEATURE(sb,mask) \ - ( EXT3_SB(sb)->s_es->s_feature_ro_compat & cpu_to_le32(mask) ) -#define EXT3_HAS_INCOMPAT_FEATURE(sb,mask) \ - ( EXT3_SB(sb)->s_es->s_feature_incompat & cpu_to_le32(mask) ) -#define EXT3_SET_COMPAT_FEATURE(sb,mask) \ - EXT3_SB(sb)->s_es->s_feature_compat |= cpu_to_le32(mask) -#define EXT3_SET_RO_COMPAT_FEATURE(sb,mask) \ - EXT3_SB(sb)->s_es->s_feature_ro_compat |= cpu_to_le32(mask) -#define EXT3_SET_INCOMPAT_FEATURE(sb,mask) \ - EXT3_SB(sb)->s_es->s_feature_incompat |= cpu_to_le32(mask) -#define EXT3_CLEAR_COMPAT_FEATURE(sb,mask) \ - EXT3_SB(sb)->s_es->s_feature_compat &= ~cpu_to_le32(mask) -#define EXT3_CLEAR_RO_COMPAT_FEATURE(sb,mask) \ - EXT3_SB(sb)->s_es->s_feature_ro_compat &= ~cpu_to_le32(mask) -#define EXT3_CLEAR_INCOMPAT_FEATURE(sb,mask) \ - EXT3_SB(sb)->s_es->s_feature_incompat &= ~cpu_to_le32(mask) - -#define EXT3_FEATURE_COMPAT_DIR_PREALLOC 0x0001 -#define EXT3_FEATURE_COMPAT_IMAGIC_INODES 0x0002 -#define EXT3_FEATURE_COMPAT_HAS_JOURNAL 0x0004 -#define EXT3_FEATURE_COMPAT_EXT_ATTR 0x0008 -#define EXT3_FEATURE_COMPAT_RESIZE_INODE 0x0010 -#define EXT3_FEATURE_COMPAT_DIR_INDEX 0x0020 - -#define EXT3_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001 -#define EXT3_FEATURE_RO_COMPAT_LARGE_FILE 0x0002 -#define EXT3_FEATURE_RO_COMPAT_BTREE_DIR 0x0004 - -#define EXT3_FEATURE_INCOMPAT_COMPRESSION 0x0001 -#define EXT3_FEATURE_INCOMPAT_FILETYPE 0x0002 -#define EXT3_FEATURE_INCOMPAT_RECOVER 0x0004 /* Needs recovery */ -#define EXT3_FEATURE_INCOMPAT_JOURNAL_DEV 0x0008 /* Journal device */ -#define EXT3_FEATURE_INCOMPAT_META_BG 0x0010 - -#define EXT3_FEATURE_COMPAT_SUPP EXT2_FEATURE_COMPAT_EXT_ATTR -#define EXT3_FEATURE_INCOMPAT_SUPP (EXT3_FEATURE_INCOMPAT_FILETYPE| \ - EXT3_FEATURE_INCOMPAT_RECOVER| \ - EXT3_FEATURE_INCOMPAT_META_BG) -#define EXT3_FEATURE_RO_COMPAT_SUPP (EXT3_FEATURE_RO_COMPAT_SPARSE_SUPER| \ - EXT3_FEATURE_RO_COMPAT_LARGE_FILE| \ - EXT3_FEATURE_RO_COMPAT_BTREE_DIR) - -/* - * Default values for user and/or group using reserved blocks - */ -#define EXT3_DEF_RESUID 0 -#define EXT3_DEF_RESGID 0 - -/* - * Default mount options - */ -#define EXT3_DEFM_DEBUG 0x0001 -#define EXT3_DEFM_BSDGROUPS 0x0002 -#define EXT3_DEFM_XATTR_USER 0x0004 -#define EXT3_DEFM_ACL 0x0008 -#define EXT3_DEFM_UID16 0x0010 -#define EXT3_DEFM_JMODE 0x0060 -#define EXT3_DEFM_JMODE_DATA 0x0020 -#define EXT3_DEFM_JMODE_ORDERED 0x0040 -#define EXT3_DEFM_JMODE_WBACK 0x0060 - -/* - * Structure of a directory entry - */ -#define EXT3_NAME_LEN 255 - -struct ext3_dir_entry { - __le32 inode; /* Inode number */ - __le16 rec_len; /* Directory entry length */ - __le16 name_len; /* Name length */ - char name[EXT3_NAME_LEN]; /* File name */ -}; - -/* - * The new version of the directory entry. Since EXT3 structures are - * stored in intel byte order, and the name_len field could never be - * bigger than 255 chars, it's safe to reclaim the extra byte for the - * file_type field. - */ -struct ext3_dir_entry_2 { - __le32 inode; /* Inode number */ - __le16 rec_len; /* Directory entry length */ - __u8 name_len; /* Name length */ - __u8 file_type; - char name[EXT3_NAME_LEN]; /* File name */ -}; - -/* - * Ext3 directory file types. Only the low 3 bits are used. The - * other bits are reserved for now. - */ -#define EXT3_FT_UNKNOWN 0 -#define EXT3_FT_REG_FILE 1 -#define EXT3_FT_DIR 2 -#define EXT3_FT_CHRDEV 3 -#define EXT3_FT_BLKDEV 4 -#define EXT3_FT_FIFO 5 -#define EXT3_FT_SOCK 6 -#define EXT3_FT_SYMLINK 7 - -#define EXT3_FT_MAX 8 - -/* - * EXT3_DIR_PAD defines the directory entries boundaries - * - * NOTE: It must be a multiple of 4 - */ -#define EXT3_DIR_PAD 4 -#define EXT3_DIR_ROUND (EXT3_DIR_PAD - 1) -#define EXT3_DIR_REC_LEN(name_len) (((name_len) + 8 + EXT3_DIR_ROUND) & \ - ~EXT3_DIR_ROUND) -#define EXT3_MAX_REC_LEN ((1<<16)-1) - -/* - * Tests against MAX_REC_LEN etc were put in place for 64k block - * sizes; if that is not possible on this arch, we can skip - * those tests and speed things up. - */ -static inline unsigned ext3_rec_len_from_disk(__le16 dlen) -{ - unsigned len = le16_to_cpu(dlen); - -#if (PAGE_CACHE_SIZE >= 65536) - if (len == EXT3_MAX_REC_LEN) - return 1 << 16; -#endif - return len; -} - -static inline __le16 ext3_rec_len_to_disk(unsigned len) -{ -#if (PAGE_CACHE_SIZE >= 65536) - if (len == (1 << 16)) - return cpu_to_le16(EXT3_MAX_REC_LEN); - else if (len > (1 << 16)) - BUG(); -#endif - return cpu_to_le16(len); -} - -/* - * Hash Tree Directory indexing - * (c) Daniel Phillips, 2001 - */ - -#define is_dx(dir) (EXT3_HAS_COMPAT_FEATURE(dir->i_sb, \ - EXT3_FEATURE_COMPAT_DIR_INDEX) && \ - (EXT3_I(dir)->i_flags & EXT3_INDEX_FL)) -#define EXT3_DIR_LINK_MAX(dir) (!is_dx(dir) && (dir)->i_nlink >= EXT3_LINK_MAX) -#define EXT3_DIR_LINK_EMPTY(dir) ((dir)->i_nlink == 2 || (dir)->i_nlink == 1) - -/* Legal values for the dx_root hash_version field: */ - -#define DX_HASH_LEGACY 0 -#define DX_HASH_HALF_MD4 1 -#define DX_HASH_TEA 2 -#define DX_HASH_LEGACY_UNSIGNED 3 -#define DX_HASH_HALF_MD4_UNSIGNED 4 -#define DX_HASH_TEA_UNSIGNED 5 - -/* hash info structure used by the directory hash */ -struct dx_hash_info -{ - u32 hash; - u32 minor_hash; - int hash_version; - u32 *seed; -}; - - -/* 32 and 64 bit signed EOF for dx directories */ -#define EXT3_HTREE_EOF_32BIT ((1UL << (32 - 1)) - 1) -#define EXT3_HTREE_EOF_64BIT ((1ULL << (64 - 1)) - 1) - - -/* - * Control parameters used by ext3_htree_next_block - */ -#define HASH_NB_ALWAYS 1 - - -/* - * Describe an inode's exact location on disk and in memory - */ -struct ext3_iloc -{ - struct buffer_head *bh; - unsigned long offset; - unsigned long block_group; -}; - -static inline struct ext3_inode *ext3_raw_inode(struct ext3_iloc *iloc) -{ - return (struct ext3_inode *) (iloc->bh->b_data + iloc->offset); -} - -/* - * This structure is stuffed into the struct file's private_data field - * for directories. It is where we put information so that we can do - * readdir operations in hash tree order. - */ -struct dir_private_info { - struct rb_root root; - struct rb_node *curr_node; - struct fname *extra_fname; - loff_t last_pos; - __u32 curr_hash; - __u32 curr_minor_hash; - __u32 next_hash; -}; - -/* calculate the first block number of the group */ -static inline ext3_fsblk_t -ext3_group_first_block_no(struct super_block *sb, unsigned long group_no) -{ - return group_no * (ext3_fsblk_t)EXT3_BLOCKS_PER_GROUP(sb) + - le32_to_cpu(EXT3_SB(sb)->s_es->s_first_data_block); -} - -/* - * Special error return code only used by dx_probe() and its callers. - */ -#define ERR_BAD_DX_DIR -75000 - -/* - * Function prototypes - */ - -/* - * Ok, these declarations are also in but none of the - * ext3 source programs needs to include it so they are duplicated here. - */ -# define NORET_TYPE /**/ -# define ATTRIB_NORET __attribute__((noreturn)) -# define NORET_AND noreturn, - -/* balloc.c */ -extern int ext3_bg_has_super(struct super_block *sb, int group); -extern unsigned long ext3_bg_num_gdb(struct super_block *sb, int group); -extern ext3_fsblk_t ext3_new_block (handle_t *handle, struct inode *inode, - ext3_fsblk_t goal, int *errp); -extern ext3_fsblk_t ext3_new_blocks (handle_t *handle, struct inode *inode, - ext3_fsblk_t goal, unsigned long *count, int *errp); -extern void ext3_free_blocks (handle_t *handle, struct inode *inode, - ext3_fsblk_t block, unsigned long count); -extern void ext3_free_blocks_sb (handle_t *handle, struct super_block *sb, - ext3_fsblk_t block, unsigned long count, - unsigned long *pdquot_freed_blocks); -extern ext3_fsblk_t ext3_count_free_blocks (struct super_block *); -extern void ext3_check_blocks_bitmap (struct super_block *); -extern struct ext3_group_desc * ext3_get_group_desc(struct super_block * sb, - unsigned int block_group, - struct buffer_head ** bh); -extern int ext3_should_retry_alloc(struct super_block *sb, int *retries); -extern void ext3_init_block_alloc_info(struct inode *); -extern void ext3_rsv_window_add(struct super_block *sb, struct ext3_reserve_window_node *rsv); -extern int ext3_trim_fs(struct super_block *sb, struct fstrim_range *range); - -/* dir.c */ -extern int ext3_check_dir_entry(const char *, struct inode *, - struct ext3_dir_entry_2 *, - struct buffer_head *, unsigned long); -extern int ext3_htree_store_dirent(struct file *dir_file, __u32 hash, - __u32 minor_hash, - struct ext3_dir_entry_2 *dirent); -extern void ext3_htree_free_dir_info(struct dir_private_info *p); - -/* fsync.c */ -extern int ext3_sync_file(struct file *, loff_t, loff_t, int); - -/* hash.c */ -extern int ext3fs_dirhash(const char *name, int len, struct - dx_hash_info *hinfo); - -/* ialloc.c */ -extern struct inode * ext3_new_inode (handle_t *, struct inode *, - const struct qstr *, umode_t); -extern void ext3_free_inode (handle_t *, struct inode *); -extern struct inode * ext3_orphan_get (struct super_block *, unsigned long); -extern unsigned long ext3_count_free_inodes (struct super_block *); -extern unsigned long ext3_count_dirs (struct super_block *); -extern void ext3_check_inodes_bitmap (struct super_block *); -extern unsigned long ext3_count_free (struct buffer_head *, unsigned); - - -/* inode.c */ -int ext3_forget(handle_t *handle, int is_metadata, struct inode *inode, - struct buffer_head *bh, ext3_fsblk_t blocknr); -struct buffer_head * ext3_getblk (handle_t *, struct inode *, long, int, int *); -struct buffer_head * ext3_bread (handle_t *, struct inode *, int, int, int *); -int ext3_get_blocks_handle(handle_t *handle, struct inode *inode, - sector_t iblock, unsigned long maxblocks, struct buffer_head *bh_result, - int create); - -extern struct inode *ext3_iget(struct super_block *, unsigned long); -extern int ext3_write_inode (struct inode *, struct writeback_control *); -extern int ext3_setattr (struct dentry *, struct iattr *); -extern void ext3_evict_inode (struct inode *); -extern int ext3_sync_inode (handle_t *, struct inode *); -extern void ext3_discard_reservation (struct inode *); -extern void ext3_dirty_inode(struct inode *, int); -extern int ext3_change_inode_journal_flag(struct inode *, int); -extern int ext3_get_inode_loc(struct inode *, struct ext3_iloc *); -extern int ext3_can_truncate(struct inode *inode); -extern void ext3_truncate(struct inode *inode); -extern void ext3_set_inode_flags(struct inode *); -extern void ext3_get_inode_flags(struct ext3_inode_info *); -extern void ext3_set_aops(struct inode *inode); -extern int ext3_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, - u64 start, u64 len); - -/* ioctl.c */ -extern long ext3_ioctl(struct file *, unsigned int, unsigned long); -extern long ext3_compat_ioctl(struct file *, unsigned int, unsigned long); - -/* namei.c */ -extern int ext3_orphan_add(handle_t *, struct inode *); -extern int ext3_orphan_del(handle_t *, struct inode *); -extern int ext3_htree_fill_tree(struct file *dir_file, __u32 start_hash, - __u32 start_minor_hash, __u32 *next_hash); - -/* resize.c */ -extern int ext3_group_add(struct super_block *sb, - struct ext3_new_group_data *input); -extern int ext3_group_extend(struct super_block *sb, - struct ext3_super_block *es, - ext3_fsblk_t n_blocks_count); - -/* super.c */ -extern __printf(3, 4) -void ext3_error(struct super_block *, const char *, const char *, ...); -extern void __ext3_std_error (struct super_block *, const char *, int); -extern __printf(3, 4) -void ext3_abort(struct super_block *, const char *, const char *, ...); -extern __printf(3, 4) -void ext3_warning(struct super_block *, const char *, const char *, ...); -extern __printf(3, 4) -void ext3_msg(struct super_block *, const char *, const char *, ...); -extern void ext3_update_dynamic_rev (struct super_block *sb); - -#define ext3_std_error(sb, errno) \ -do { \ - if ((errno)) \ - __ext3_std_error((sb), __func__, (errno)); \ -} while (0) - -/* - * Inodes and files operations - */ - -/* dir.c */ -extern const struct file_operations ext3_dir_operations; - -/* file.c */ -extern const struct inode_operations ext3_file_inode_operations; -extern const struct file_operations ext3_file_operations; - -/* namei.c */ -extern const struct inode_operations ext3_dir_inode_operations; -extern const struct inode_operations ext3_special_inode_operations; - -/* symlink.c */ -extern const struct inode_operations ext3_symlink_inode_operations; -extern const struct inode_operations ext3_fast_symlink_inode_operations; - -#define EXT3_JOURNAL(inode) (EXT3_SB((inode)->i_sb)->s_journal) - -/* Define the number of blocks we need to account to a transaction to - * modify one block of data. - * - * We may have to touch one inode, one bitmap buffer, up to three - * indirection blocks, the group and superblock summaries, and the data - * block to complete the transaction. */ - -#define EXT3_SINGLEDATA_TRANS_BLOCKS 8U - -/* Extended attribute operations touch at most two data buffers, - * two bitmap buffers, and two group summaries, in addition to the inode - * and the superblock, which are already accounted for. */ - -#define EXT3_XATTR_TRANS_BLOCKS 6U - -/* Define the minimum size for a transaction which modifies data. This - * needs to take into account the fact that we may end up modifying two - * quota files too (one for the group, one for the user quota). The - * superblock only gets updated once, of course, so don't bother - * counting that again for the quota updates. */ - -#define EXT3_DATA_TRANS_BLOCKS(sb) (EXT3_SINGLEDATA_TRANS_BLOCKS + \ - EXT3_XATTR_TRANS_BLOCKS - 2 + \ - EXT3_MAXQUOTAS_TRANS_BLOCKS(sb)) - -/* Delete operations potentially hit one directory's namespace plus an - * entire inode, plus arbitrary amounts of bitmap/indirection data. Be - * generous. We can grow the delete transaction later if necessary. */ - -#define EXT3_DELETE_TRANS_BLOCKS(sb) (EXT3_MAXQUOTAS_TRANS_BLOCKS(sb) + 64) - -/* Define an arbitrary limit for the amount of data we will anticipate - * writing to any given transaction. For unbounded transactions such as - * write(2) and truncate(2) we can write more than this, but we always - * start off at the maximum transaction size and grow the transaction - * optimistically as we go. */ - -#define EXT3_MAX_TRANS_DATA 64U - -/* We break up a large truncate or write transaction once the handle's - * buffer credits gets this low, we need either to extend the - * transaction or to start a new one. Reserve enough space here for - * inode, bitmap, superblock, group and indirection updates for at least - * one block, plus two quota updates. Quota allocations are not - * needed. */ - -#define EXT3_RESERVE_TRANS_BLOCKS 12U - -#define EXT3_INDEX_EXTRA_TRANS_BLOCKS 8 - -#ifdef CONFIG_QUOTA -/* Amount of blocks needed for quota update - we know that the structure was - * allocated so we need to update only inode+data */ -#define EXT3_QUOTA_TRANS_BLOCKS(sb) (test_opt(sb, QUOTA) ? 2 : 0) -/* Amount of blocks needed for quota insert/delete - we do some block writes - * but inode, sb and group updates are done only once */ -#define EXT3_QUOTA_INIT_BLOCKS(sb) (test_opt(sb, QUOTA) ? (DQUOT_INIT_ALLOC*\ - (EXT3_SINGLEDATA_TRANS_BLOCKS-3)+3+DQUOT_INIT_REWRITE) : 0) -#define EXT3_QUOTA_DEL_BLOCKS(sb) (test_opt(sb, QUOTA) ? (DQUOT_DEL_ALLOC*\ - (EXT3_SINGLEDATA_TRANS_BLOCKS-3)+3+DQUOT_DEL_REWRITE) : 0) -#else -#define EXT3_QUOTA_TRANS_BLOCKS(sb) 0 -#define EXT3_QUOTA_INIT_BLOCKS(sb) 0 -#define EXT3_QUOTA_DEL_BLOCKS(sb) 0 -#endif -#define EXT3_MAXQUOTAS_TRANS_BLOCKS(sb) (EXT3_MAXQUOTAS*EXT3_QUOTA_TRANS_BLOCKS(sb)) -#define EXT3_MAXQUOTAS_INIT_BLOCKS(sb) (EXT3_MAXQUOTAS*EXT3_QUOTA_INIT_BLOCKS(sb)) -#define EXT3_MAXQUOTAS_DEL_BLOCKS(sb) (EXT3_MAXQUOTAS*EXT3_QUOTA_DEL_BLOCKS(sb)) - -int -ext3_mark_iloc_dirty(handle_t *handle, - struct inode *inode, - struct ext3_iloc *iloc); - -/* - * On success, We end up with an outstanding reference count against - * iloc->bh. This _must_ be cleaned up later. - */ - -int ext3_reserve_inode_write(handle_t *handle, struct inode *inode, - struct ext3_iloc *iloc); - -int ext3_mark_inode_dirty(handle_t *handle, struct inode *inode); - -/* - * Wrapper functions with which ext3 calls into JBD. The intent here is - * to allow these to be turned into appropriate stubs so ext3 can control - * ext2 filesystems, so ext2+ext3 systems only nee one fs. This work hasn't - * been done yet. - */ - -static inline void ext3_journal_release_buffer(handle_t *handle, - struct buffer_head *bh) -{ - journal_release_buffer(handle, bh); -} - -void ext3_journal_abort_handle(const char *caller, const char *err_fn, - struct buffer_head *bh, handle_t *handle, int err); - -int __ext3_journal_get_undo_access(const char *where, handle_t *handle, - struct buffer_head *bh); - -int __ext3_journal_get_write_access(const char *where, handle_t *handle, - struct buffer_head *bh); - -int __ext3_journal_forget(const char *where, handle_t *handle, - struct buffer_head *bh); - -int __ext3_journal_revoke(const char *where, handle_t *handle, - unsigned long blocknr, struct buffer_head *bh); - -int __ext3_journal_get_create_access(const char *where, - handle_t *handle, struct buffer_head *bh); - -int __ext3_journal_dirty_metadata(const char *where, - handle_t *handle, struct buffer_head *bh); - -#define ext3_journal_get_undo_access(handle, bh) \ - __ext3_journal_get_undo_access(__func__, (handle), (bh)) -#define ext3_journal_get_write_access(handle, bh) \ - __ext3_journal_get_write_access(__func__, (handle), (bh)) -#define ext3_journal_revoke(handle, blocknr, bh) \ - __ext3_journal_revoke(__func__, (handle), (blocknr), (bh)) -#define ext3_journal_get_create_access(handle, bh) \ - __ext3_journal_get_create_access(__func__, (handle), (bh)) -#define ext3_journal_dirty_metadata(handle, bh) \ - __ext3_journal_dirty_metadata(__func__, (handle), (bh)) -#define ext3_journal_forget(handle, bh) \ - __ext3_journal_forget(__func__, (handle), (bh)) - -int ext3_journal_dirty_data(handle_t *handle, struct buffer_head *bh); - -handle_t *ext3_journal_start_sb(struct super_block *sb, int nblocks); -int __ext3_journal_stop(const char *where, handle_t *handle); - -static inline handle_t *ext3_journal_start(struct inode *inode, int nblocks) -{ - return ext3_journal_start_sb(inode->i_sb, nblocks); -} - -#define ext3_journal_stop(handle) \ - __ext3_journal_stop(__func__, (handle)) - -static inline handle_t *ext3_journal_current_handle(void) -{ - return journal_current_handle(); -} - -static inline int ext3_journal_extend(handle_t *handle, int nblocks) -{ - return journal_extend(handle, nblocks); -} - -static inline int ext3_journal_restart(handle_t *handle, int nblocks) -{ - return journal_restart(handle, nblocks); -} - -static inline int ext3_journal_blocks_per_page(struct inode *inode) -{ - return journal_blocks_per_page(inode); -} - -static inline int ext3_journal_force_commit(journal_t *journal) -{ - return journal_force_commit(journal); -} - -/* super.c */ -int ext3_force_commit(struct super_block *sb); - -static inline int ext3_should_journal_data(struct inode *inode) -{ - if (!S_ISREG(inode->i_mode)) - return 1; - if (test_opt(inode->i_sb, DATA_FLAGS) == EXT3_MOUNT_JOURNAL_DATA) - return 1; - if (EXT3_I(inode)->i_flags & EXT3_JOURNAL_DATA_FL) - return 1; - return 0; -} - -static inline int ext3_should_order_data(struct inode *inode) -{ - if (!S_ISREG(inode->i_mode)) - return 0; - if (EXT3_I(inode)->i_flags & EXT3_JOURNAL_DATA_FL) - return 0; - if (test_opt(inode->i_sb, DATA_FLAGS) == EXT3_MOUNT_ORDERED_DATA) - return 1; - return 0; -} - -static inline int ext3_should_writeback_data(struct inode *inode) -{ - if (!S_ISREG(inode->i_mode)) - return 0; - if (EXT3_I(inode)->i_flags & EXT3_JOURNAL_DATA_FL) - return 0; - if (test_opt(inode->i_sb, DATA_FLAGS) == EXT3_MOUNT_WRITEBACK_DATA) - return 1; - return 0; -} - -#include diff --git a/fs/ext3/ext3_jbd.c b/fs/ext3/ext3_jbd.c deleted file mode 100644 index 785a3261a26c6a..00000000000000 --- a/fs/ext3/ext3_jbd.c +++ /dev/null @@ -1,59 +0,0 @@ -/* - * Interface between ext3 and JBD - */ - -#include "ext3.h" - -int __ext3_journal_get_undo_access(const char *where, handle_t *handle, - struct buffer_head *bh) -{ - int err = journal_get_undo_access(handle, bh); - if (err) - ext3_journal_abort_handle(where, __func__, bh, handle,err); - return err; -} - -int __ext3_journal_get_write_access(const char *where, handle_t *handle, - struct buffer_head *bh) -{ - int err = journal_get_write_access(handle, bh); - if (err) - ext3_journal_abort_handle(where, __func__, bh, handle,err); - return err; -} - -int __ext3_journal_forget(const char *where, handle_t *handle, - struct buffer_head *bh) -{ - int err = journal_forget(handle, bh); - if (err) - ext3_journal_abort_handle(where, __func__, bh, handle,err); - return err; -} - -int __ext3_journal_revoke(const char *where, handle_t *handle, - unsigned long blocknr, struct buffer_head *bh) -{ - int err = journal_revoke(handle, blocknr, bh); - if (err) - ext3_journal_abort_handle(where, __func__, bh, handle,err); - return err; -} - -int __ext3_journal_get_create_access(const char *where, - handle_t *handle, struct buffer_head *bh) -{ - int err = journal_get_create_access(handle, bh); - if (err) - ext3_journal_abort_handle(where, __func__, bh, handle,err); - return err; -} - -int __ext3_journal_dirty_metadata(const char *where, - handle_t *handle, struct buffer_head *bh) -{ - int err = journal_dirty_metadata(handle, bh); - if (err) - ext3_journal_abort_handle(where, __func__, bh, handle,err); - return err; -} diff --git a/fs/ext3/file.c b/fs/ext3/file.c deleted file mode 100644 index 3b8f650de22ce2..00000000000000 --- a/fs/ext3/file.c +++ /dev/null @@ -1,79 +0,0 @@ -/* - * linux/fs/ext3/file.c - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * - * from - * - * linux/fs/minix/file.c - * - * Copyright (C) 1991, 1992 Linus Torvalds - * - * ext3 fs regular file handling primitives - * - * 64-bit file support on 64-bit platforms by Jakub Jelinek - * (jj@sunsite.ms.mff.cuni.cz) - */ - -#include -#include "ext3.h" -#include "xattr.h" -#include "acl.h" - -/* - * Called when an inode is released. Note that this is different - * from ext3_file_open: open gets called at every open, but release - * gets called only when /all/ the files are closed. - */ -static int ext3_release_file (struct inode * inode, struct file * filp) -{ - if (ext3_test_inode_state(inode, EXT3_STATE_FLUSH_ON_CLOSE)) { - filemap_flush(inode->i_mapping); - ext3_clear_inode_state(inode, EXT3_STATE_FLUSH_ON_CLOSE); - } - /* if we are the last writer on the inode, drop the block reservation */ - if ((filp->f_mode & FMODE_WRITE) && - (atomic_read(&inode->i_writecount) == 1)) - { - mutex_lock(&EXT3_I(inode)->truncate_mutex); - ext3_discard_reservation(inode); - mutex_unlock(&EXT3_I(inode)->truncate_mutex); - } - if (is_dx(inode) && filp->private_data) - ext3_htree_free_dir_info(filp->private_data); - - return 0; -} - -const struct file_operations ext3_file_operations = { - .llseek = generic_file_llseek, - .read_iter = generic_file_read_iter, - .write_iter = generic_file_write_iter, - .unlocked_ioctl = ext3_ioctl, -#ifdef CONFIG_COMPAT - .compat_ioctl = ext3_compat_ioctl, -#endif - .mmap = generic_file_mmap, - .open = dquot_file_open, - .release = ext3_release_file, - .fsync = ext3_sync_file, - .splice_read = generic_file_splice_read, - .splice_write = iter_file_splice_write, -}; - -const struct inode_operations ext3_file_inode_operations = { - .setattr = ext3_setattr, -#ifdef CONFIG_EXT3_FS_XATTR - .setxattr = generic_setxattr, - .getxattr = generic_getxattr, - .listxattr = ext3_listxattr, - .removexattr = generic_removexattr, -#endif - .get_acl = ext3_get_acl, - .set_acl = ext3_set_acl, - .fiemap = ext3_fiemap, -}; - diff --git a/fs/ext3/fsync.c b/fs/ext3/fsync.c deleted file mode 100644 index 1cb9c7e10c6f22..00000000000000 --- a/fs/ext3/fsync.c +++ /dev/null @@ -1,109 +0,0 @@ -/* - * linux/fs/ext3/fsync.c - * - * Copyright (C) 1993 Stephen Tweedie (sct@redhat.com) - * from - * Copyright (C) 1992 Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * from - * linux/fs/minix/truncate.c Copyright (C) 1991, 1992 Linus Torvalds - * - * ext3fs fsync primitive - * - * Big-endian to little-endian byte-swapping/bitmaps by - * David S. Miller (davem@caip.rutgers.edu), 1995 - * - * Removed unnecessary code duplication for little endian machines - * and excessive __inline__s. - * Andi Kleen, 1997 - * - * Major simplications and cleanup - we only need to do the metadata, because - * we can depend on generic_block_fdatasync() to sync the data blocks. - */ - -#include -#include -#include "ext3.h" - -/* - * akpm: A new design for ext3_sync_file(). - * - * This is only called from sys_fsync(), sys_fdatasync() and sys_msync(). - * There cannot be a transaction open by this task. - * Another task could have dirtied this inode. Its data can be in any - * state in the journalling system. - * - * What we do is just kick off a commit and wait on it. This will snapshot the - * inode to disk. - */ - -int ext3_sync_file(struct file *file, loff_t start, loff_t end, int datasync) -{ - struct inode *inode = file->f_mapping->host; - struct ext3_inode_info *ei = EXT3_I(inode); - journal_t *journal = EXT3_SB(inode->i_sb)->s_journal; - int ret, needs_barrier = 0; - tid_t commit_tid; - - trace_ext3_sync_file_enter(file, datasync); - - if (inode->i_sb->s_flags & MS_RDONLY) { - /* Make sure that we read updated state */ - smp_rmb(); - if (EXT3_SB(inode->i_sb)->s_mount_state & EXT3_ERROR_FS) - return -EROFS; - return 0; - } - ret = filemap_write_and_wait_range(inode->i_mapping, start, end); - if (ret) - goto out; - - J_ASSERT(ext3_journal_current_handle() == NULL); - - /* - * data=writeback,ordered: - * The caller's filemap_fdatawrite()/wait will sync the data. - * Metadata is in the journal, we wait for a proper transaction - * to commit here. - * - * data=journal: - * filemap_fdatawrite won't do anything (the buffers are clean). - * ext3_force_commit will write the file data into the journal and - * will wait on that. - * filemap_fdatawait() will encounter a ton of newly-dirtied pages - * (they were dirtied by commit). But that's OK - the blocks are - * safe in-journal, which is all fsync() needs to ensure. - */ - if (ext3_should_journal_data(inode)) { - ret = ext3_force_commit(inode->i_sb); - goto out; - } - - if (datasync) - commit_tid = atomic_read(&ei->i_datasync_tid); - else - commit_tid = atomic_read(&ei->i_sync_tid); - - if (test_opt(inode->i_sb, BARRIER) && - !journal_trans_will_send_data_barrier(journal, commit_tid)) - needs_barrier = 1; - log_start_commit(journal, commit_tid); - ret = log_wait_commit(journal, commit_tid); - - /* - * In case we didn't commit a transaction, we have to flush - * disk caches manually so that data really is on persistent - * storage - */ - if (needs_barrier) { - int err; - - err = blkdev_issue_flush(inode->i_sb->s_bdev, GFP_KERNEL, NULL); - if (!ret) - ret = err; - } -out: - trace_ext3_sync_file_exit(inode, ret); - return ret; -} diff --git a/fs/ext3/hash.c b/fs/ext3/hash.c deleted file mode 100644 index ede315cdf12614..00000000000000 --- a/fs/ext3/hash.c +++ /dev/null @@ -1,206 +0,0 @@ -/* - * linux/fs/ext3/hash.c - * - * Copyright (C) 2002 by Theodore Ts'o - * - * This file is released under the GPL v2. - * - * This file may be redistributed under the terms of the GNU Public - * License. - */ - -#include "ext3.h" -#include - -#define DELTA 0x9E3779B9 - -static void TEA_transform(__u32 buf[4], __u32 const in[]) -{ - __u32 sum = 0; - __u32 b0 = buf[0], b1 = buf[1]; - __u32 a = in[0], b = in[1], c = in[2], d = in[3]; - int n = 16; - - do { - sum += DELTA; - b0 += ((b1 << 4)+a) ^ (b1+sum) ^ ((b1 >> 5)+b); - b1 += ((b0 << 4)+c) ^ (b0+sum) ^ ((b0 >> 5)+d); - } while(--n); - - buf[0] += b0; - buf[1] += b1; -} - - -/* The old legacy hash */ -static __u32 dx_hack_hash_unsigned(const char *name, int len) -{ - __u32 hash, hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9; - const unsigned char *ucp = (const unsigned char *) name; - - while (len--) { - hash = hash1 + (hash0 ^ (((int) *ucp++) * 7152373)); - - if (hash & 0x80000000) - hash -= 0x7fffffff; - hash1 = hash0; - hash0 = hash; - } - return hash0 << 1; -} - -static __u32 dx_hack_hash_signed(const char *name, int len) -{ - __u32 hash, hash0 = 0x12a3fe2d, hash1 = 0x37abe8f9; - const signed char *scp = (const signed char *) name; - - while (len--) { - hash = hash1 + (hash0 ^ (((int) *scp++) * 7152373)); - - if (hash & 0x80000000) - hash -= 0x7fffffff; - hash1 = hash0; - hash0 = hash; - } - return hash0 << 1; -} - -static void str2hashbuf_signed(const char *msg, int len, __u32 *buf, int num) -{ - __u32 pad, val; - int i; - const signed char *scp = (const signed char *) msg; - - pad = (__u32)len | ((__u32)len << 8); - pad |= pad << 16; - - val = pad; - if (len > num*4) - len = num * 4; - for (i = 0; i < len; i++) { - if ((i % 4) == 0) - val = pad; - val = ((int) scp[i]) + (val << 8); - if ((i % 4) == 3) { - *buf++ = val; - val = pad; - num--; - } - } - if (--num >= 0) - *buf++ = val; - while (--num >= 0) - *buf++ = pad; -} - -static void str2hashbuf_unsigned(const char *msg, int len, __u32 *buf, int num) -{ - __u32 pad, val; - int i; - const unsigned char *ucp = (const unsigned char *) msg; - - pad = (__u32)len | ((__u32)len << 8); - pad |= pad << 16; - - val = pad; - if (len > num*4) - len = num * 4; - for (i=0; i < len; i++) { - if ((i % 4) == 0) - val = pad; - val = ((int) ucp[i]) + (val << 8); - if ((i % 4) == 3) { - *buf++ = val; - val = pad; - num--; - } - } - if (--num >= 0) - *buf++ = val; - while (--num >= 0) - *buf++ = pad; -} - -/* - * Returns the hash of a filename. If len is 0 and name is NULL, then - * this function can be used to test whether or not a hash version is - * supported. - * - * The seed is an 4 longword (32 bits) "secret" which can be used to - * uniquify a hash. If the seed is all zero's, then some default seed - * may be used. - * - * A particular hash version specifies whether or not the seed is - * represented, and whether or not the returned hash is 32 bits or 64 - * bits. 32 bit hashes will return 0 for the minor hash. - */ -int ext3fs_dirhash(const char *name, int len, struct dx_hash_info *hinfo) -{ - __u32 hash; - __u32 minor_hash = 0; - const char *p; - int i; - __u32 in[8], buf[4]; - void (*str2hashbuf)(const char *, int, __u32 *, int) = - str2hashbuf_signed; - - /* Initialize the default seed for the hash checksum functions */ - buf[0] = 0x67452301; - buf[1] = 0xefcdab89; - buf[2] = 0x98badcfe; - buf[3] = 0x10325476; - - /* Check to see if the seed is all zero's */ - if (hinfo->seed) { - for (i=0; i < 4; i++) { - if (hinfo->seed[i]) - break; - } - if (i < 4) - memcpy(buf, hinfo->seed, sizeof(buf)); - } - - switch (hinfo->hash_version) { - case DX_HASH_LEGACY_UNSIGNED: - hash = dx_hack_hash_unsigned(name, len); - break; - case DX_HASH_LEGACY: - hash = dx_hack_hash_signed(name, len); - break; - case DX_HASH_HALF_MD4_UNSIGNED: - str2hashbuf = str2hashbuf_unsigned; - case DX_HASH_HALF_MD4: - p = name; - while (len > 0) { - (*str2hashbuf)(p, len, in, 8); - half_md4_transform(buf, in); - len -= 32; - p += 32; - } - minor_hash = buf[2]; - hash = buf[1]; - break; - case DX_HASH_TEA_UNSIGNED: - str2hashbuf = str2hashbuf_unsigned; - case DX_HASH_TEA: - p = name; - while (len > 0) { - (*str2hashbuf)(p, len, in, 4); - TEA_transform(buf, in); - len -= 16; - p += 16; - } - hash = buf[0]; - minor_hash = buf[1]; - break; - default: - hinfo->hash = 0; - return -1; - } - hash = hash & ~1; - if (hash == (EXT3_HTREE_EOF_32BIT << 1)) - hash = (EXT3_HTREE_EOF_32BIT - 1) << 1; - hinfo->hash = hash; - hinfo->minor_hash = minor_hash; - return 0; -} diff --git a/fs/ext3/ialloc.c b/fs/ext3/ialloc.c deleted file mode 100644 index 3ad242e5840e0b..00000000000000 --- a/fs/ext3/ialloc.c +++ /dev/null @@ -1,706 +0,0 @@ -/* - * linux/fs/ext3/ialloc.c - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * - * BSD ufs-inspired inode and directory allocation by - * Stephen Tweedie (sct@redhat.com), 1993 - * Big-endian to little-endian byte-swapping/bitmaps by - * David S. Miller (davem@caip.rutgers.edu), 1995 - */ - -#include -#include - -#include "ext3.h" -#include "xattr.h" -#include "acl.h" - -/* - * ialloc.c contains the inodes allocation and deallocation routines - */ - -/* - * The free inodes are managed by bitmaps. A file system contains several - * blocks groups. Each group contains 1 bitmap block for blocks, 1 bitmap - * block for inodes, N blocks for the inode table and data blocks. - * - * The file system contains group descriptors which are located after the - * super block. Each descriptor contains the number of the bitmap block and - * the free blocks count in the block. - */ - - -/* - * Read the inode allocation bitmap for a given block_group, reading - * into the specified slot in the superblock's bitmap cache. - * - * Return buffer_head of bitmap on success or NULL. - */ -static struct buffer_head * -read_inode_bitmap(struct super_block * sb, unsigned long block_group) -{ - struct ext3_group_desc *desc; - struct buffer_head *bh = NULL; - - desc = ext3_get_group_desc(sb, block_group, NULL); - if (!desc) - goto error_out; - - bh = sb_bread(sb, le32_to_cpu(desc->bg_inode_bitmap)); - if (!bh) - ext3_error(sb, "read_inode_bitmap", - "Cannot read inode bitmap - " - "block_group = %lu, inode_bitmap = %u", - block_group, le32_to_cpu(desc->bg_inode_bitmap)); -error_out: - return bh; -} - -/* - * NOTE! When we get the inode, we're the only people - * that have access to it, and as such there are no - * race conditions we have to worry about. The inode - * is not on the hash-lists, and it cannot be reached - * through the filesystem because the directory entry - * has been deleted earlier. - * - * HOWEVER: we must make sure that we get no aliases, - * which means that we have to call "clear_inode()" - * _before_ we mark the inode not in use in the inode - * bitmaps. Otherwise a newly created file might use - * the same inode number (not actually the same pointer - * though), and then we'd have two inodes sharing the - * same inode number and space on the harddisk. - */ -void ext3_free_inode (handle_t *handle, struct inode * inode) -{ - struct super_block * sb = inode->i_sb; - int is_directory; - unsigned long ino; - struct buffer_head *bitmap_bh = NULL; - struct buffer_head *bh2; - unsigned long block_group; - unsigned long bit; - struct ext3_group_desc * gdp; - struct ext3_super_block * es; - struct ext3_sb_info *sbi; - int fatal = 0, err; - - if (atomic_read(&inode->i_count) > 1) { - printk ("ext3_free_inode: inode has count=%d\n", - atomic_read(&inode->i_count)); - return; - } - if (inode->i_nlink) { - printk ("ext3_free_inode: inode has nlink=%d\n", - inode->i_nlink); - return; - } - if (!sb) { - printk("ext3_free_inode: inode on nonexistent device\n"); - return; - } - sbi = EXT3_SB(sb); - - ino = inode->i_ino; - ext3_debug ("freeing inode %lu\n", ino); - trace_ext3_free_inode(inode); - - is_directory = S_ISDIR(inode->i_mode); - - es = EXT3_SB(sb)->s_es; - if (ino < EXT3_FIRST_INO(sb) || ino > le32_to_cpu(es->s_inodes_count)) { - ext3_error (sb, "ext3_free_inode", - "reserved or nonexistent inode %lu", ino); - goto error_return; - } - block_group = (ino - 1) / EXT3_INODES_PER_GROUP(sb); - bit = (ino - 1) % EXT3_INODES_PER_GROUP(sb); - bitmap_bh = read_inode_bitmap(sb, block_group); - if (!bitmap_bh) - goto error_return; - - BUFFER_TRACE(bitmap_bh, "get_write_access"); - fatal = ext3_journal_get_write_access(handle, bitmap_bh); - if (fatal) - goto error_return; - - /* Ok, now we can actually update the inode bitmaps.. */ - if (!ext3_clear_bit_atomic(sb_bgl_lock(sbi, block_group), - bit, bitmap_bh->b_data)) - ext3_error (sb, "ext3_free_inode", - "bit already cleared for inode %lu", ino); - else { - gdp = ext3_get_group_desc (sb, block_group, &bh2); - - BUFFER_TRACE(bh2, "get_write_access"); - fatal = ext3_journal_get_write_access(handle, bh2); - if (fatal) goto error_return; - - if (gdp) { - spin_lock(sb_bgl_lock(sbi, block_group)); - le16_add_cpu(&gdp->bg_free_inodes_count, 1); - if (is_directory) - le16_add_cpu(&gdp->bg_used_dirs_count, -1); - spin_unlock(sb_bgl_lock(sbi, block_group)); - percpu_counter_inc(&sbi->s_freeinodes_counter); - if (is_directory) - percpu_counter_dec(&sbi->s_dirs_counter); - - } - BUFFER_TRACE(bh2, "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, bh2); - if (!fatal) fatal = err; - } - BUFFER_TRACE(bitmap_bh, "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, bitmap_bh); - if (!fatal) - fatal = err; - -error_return: - brelse(bitmap_bh); - ext3_std_error(sb, fatal); -} - -/* - * Orlov's allocator for directories. - * - * We always try to spread first-level directories. - * - * If there are blockgroups with both free inodes and free blocks counts - * not worse than average we return one with smallest directory count. - * Otherwise we simply return a random group. - * - * For the rest rules look so: - * - * It's OK to put directory into a group unless - * it has too many directories already (max_dirs) or - * it has too few free inodes left (min_inodes) or - * it has too few free blocks left (min_blocks). - * Parent's group is preferred, if it doesn't satisfy these - * conditions we search cyclically through the rest. If none - * of the groups look good we just look for a group with more - * free inodes than average (starting at parent's group). - * - * Debt is incremented each time we allocate a directory and decremented - * when we allocate an inode, within 0--255. - */ - -static int find_group_orlov(struct super_block *sb, struct inode *parent) -{ - int parent_group = EXT3_I(parent)->i_block_group; - struct ext3_sb_info *sbi = EXT3_SB(sb); - int ngroups = sbi->s_groups_count; - int inodes_per_group = EXT3_INODES_PER_GROUP(sb); - unsigned int freei, avefreei; - ext3_fsblk_t freeb, avefreeb; - unsigned int ndirs; - int max_dirs, min_inodes; - ext3_grpblk_t min_blocks; - int group = -1, i; - struct ext3_group_desc *desc; - - freei = percpu_counter_read_positive(&sbi->s_freeinodes_counter); - avefreei = freei / ngroups; - freeb = percpu_counter_read_positive(&sbi->s_freeblocks_counter); - avefreeb = freeb / ngroups; - ndirs = percpu_counter_read_positive(&sbi->s_dirs_counter); - - if ((parent == d_inode(sb->s_root)) || - (EXT3_I(parent)->i_flags & EXT3_TOPDIR_FL)) { - int best_ndir = inodes_per_group; - int best_group = -1; - - group = prandom_u32(); - parent_group = (unsigned)group % ngroups; - for (i = 0; i < ngroups; i++) { - group = (parent_group + i) % ngroups; - desc = ext3_get_group_desc (sb, group, NULL); - if (!desc || !desc->bg_free_inodes_count) - continue; - if (le16_to_cpu(desc->bg_used_dirs_count) >= best_ndir) - continue; - if (le16_to_cpu(desc->bg_free_inodes_count) < avefreei) - continue; - if (le16_to_cpu(desc->bg_free_blocks_count) < avefreeb) - continue; - best_group = group; - best_ndir = le16_to_cpu(desc->bg_used_dirs_count); - } - if (best_group >= 0) - return best_group; - goto fallback; - } - - max_dirs = ndirs / ngroups + inodes_per_group / 16; - min_inodes = avefreei - inodes_per_group / 4; - min_blocks = avefreeb - EXT3_BLOCKS_PER_GROUP(sb) / 4; - - for (i = 0; i < ngroups; i++) { - group = (parent_group + i) % ngroups; - desc = ext3_get_group_desc (sb, group, NULL); - if (!desc || !desc->bg_free_inodes_count) - continue; - if (le16_to_cpu(desc->bg_used_dirs_count) >= max_dirs) - continue; - if (le16_to_cpu(desc->bg_free_inodes_count) < min_inodes) - continue; - if (le16_to_cpu(desc->bg_free_blocks_count) < min_blocks) - continue; - return group; - } - -fallback: - for (i = 0; i < ngroups; i++) { - group = (parent_group + i) % ngroups; - desc = ext3_get_group_desc (sb, group, NULL); - if (!desc || !desc->bg_free_inodes_count) - continue; - if (le16_to_cpu(desc->bg_free_inodes_count) >= avefreei) - return group; - } - - if (avefreei) { - /* - * The free-inodes counter is approximate, and for really small - * filesystems the above test can fail to find any blockgroups - */ - avefreei = 0; - goto fallback; - } - - return -1; -} - -static int find_group_other(struct super_block *sb, struct inode *parent) -{ - int parent_group = EXT3_I(parent)->i_block_group; - int ngroups = EXT3_SB(sb)->s_groups_count; - struct ext3_group_desc *desc; - int group, i; - - /* - * Try to place the inode in its parent directory - */ - group = parent_group; - desc = ext3_get_group_desc (sb, group, NULL); - if (desc && le16_to_cpu(desc->bg_free_inodes_count) && - le16_to_cpu(desc->bg_free_blocks_count)) - return group; - - /* - * We're going to place this inode in a different blockgroup from its - * parent. We want to cause files in a common directory to all land in - * the same blockgroup. But we want files which are in a different - * directory which shares a blockgroup with our parent to land in a - * different blockgroup. - * - * So add our directory's i_ino into the starting point for the hash. - */ - group = (group + parent->i_ino) % ngroups; - - /* - * Use a quadratic hash to find a group with a free inode and some free - * blocks. - */ - for (i = 1; i < ngroups; i <<= 1) { - group += i; - if (group >= ngroups) - group -= ngroups; - desc = ext3_get_group_desc (sb, group, NULL); - if (desc && le16_to_cpu(desc->bg_free_inodes_count) && - le16_to_cpu(desc->bg_free_blocks_count)) - return group; - } - - /* - * That failed: try linear search for a free inode, even if that group - * has no free blocks. - */ - group = parent_group; - for (i = 0; i < ngroups; i++) { - if (++group >= ngroups) - group = 0; - desc = ext3_get_group_desc (sb, group, NULL); - if (desc && le16_to_cpu(desc->bg_free_inodes_count)) - return group; - } - - return -1; -} - -/* - * There are two policies for allocating an inode. If the new inode is - * a directory, then a forward search is made for a block group with both - * free space and a low directory-to-inode ratio; if that fails, then of - * the groups with above-average free space, that group with the fewest - * directories already is chosen. - * - * For other inodes, search forward from the parent directory's block - * group to find a free inode. - */ -struct inode *ext3_new_inode(handle_t *handle, struct inode * dir, - const struct qstr *qstr, umode_t mode) -{ - struct super_block *sb; - struct buffer_head *bitmap_bh = NULL; - struct buffer_head *bh2; - int group; - unsigned long ino = 0; - struct inode * inode; - struct ext3_group_desc * gdp = NULL; - struct ext3_super_block * es; - struct ext3_inode_info *ei; - struct ext3_sb_info *sbi; - int err = 0; - struct inode *ret; - int i; - - /* Cannot create files in a deleted directory */ - if (!dir || !dir->i_nlink) - return ERR_PTR(-EPERM); - - sb = dir->i_sb; - trace_ext3_request_inode(dir, mode); - inode = new_inode(sb); - if (!inode) - return ERR_PTR(-ENOMEM); - ei = EXT3_I(inode); - - sbi = EXT3_SB(sb); - es = sbi->s_es; - if (S_ISDIR(mode)) - group = find_group_orlov(sb, dir); - else - group = find_group_other(sb, dir); - - err = -ENOSPC; - if (group == -1) - goto out; - - for (i = 0; i < sbi->s_groups_count; i++) { - err = -EIO; - - gdp = ext3_get_group_desc(sb, group, &bh2); - if (!gdp) - goto fail; - - brelse(bitmap_bh); - bitmap_bh = read_inode_bitmap(sb, group); - if (!bitmap_bh) - goto fail; - - ino = 0; - -repeat_in_this_group: - ino = ext3_find_next_zero_bit((unsigned long *) - bitmap_bh->b_data, EXT3_INODES_PER_GROUP(sb), ino); - if (ino < EXT3_INODES_PER_GROUP(sb)) { - - BUFFER_TRACE(bitmap_bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, bitmap_bh); - if (err) - goto fail; - - if (!ext3_set_bit_atomic(sb_bgl_lock(sbi, group), - ino, bitmap_bh->b_data)) { - /* we won it */ - BUFFER_TRACE(bitmap_bh, - "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, - bitmap_bh); - if (err) - goto fail; - goto got; - } - /* we lost it */ - journal_release_buffer(handle, bitmap_bh); - - if (++ino < EXT3_INODES_PER_GROUP(sb)) - goto repeat_in_this_group; - } - - /* - * This case is possible in concurrent environment. It is very - * rare. We cannot repeat the find_group_xxx() call because - * that will simply return the same blockgroup, because the - * group descriptor metadata has not yet been updated. - * So we just go onto the next blockgroup. - */ - if (++group == sbi->s_groups_count) - group = 0; - } - err = -ENOSPC; - goto out; - -got: - ino += group * EXT3_INODES_PER_GROUP(sb) + 1; - if (ino < EXT3_FIRST_INO(sb) || ino > le32_to_cpu(es->s_inodes_count)) { - ext3_error (sb, "ext3_new_inode", - "reserved inode or inode > inodes count - " - "block_group = %d, inode=%lu", group, ino); - err = -EIO; - goto fail; - } - - BUFFER_TRACE(bh2, "get_write_access"); - err = ext3_journal_get_write_access(handle, bh2); - if (err) goto fail; - spin_lock(sb_bgl_lock(sbi, group)); - le16_add_cpu(&gdp->bg_free_inodes_count, -1); - if (S_ISDIR(mode)) { - le16_add_cpu(&gdp->bg_used_dirs_count, 1); - } - spin_unlock(sb_bgl_lock(sbi, group)); - BUFFER_TRACE(bh2, "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, bh2); - if (err) goto fail; - - percpu_counter_dec(&sbi->s_freeinodes_counter); - if (S_ISDIR(mode)) - percpu_counter_inc(&sbi->s_dirs_counter); - - - if (test_opt(sb, GRPID)) { - inode->i_mode = mode; - inode->i_uid = current_fsuid(); - inode->i_gid = dir->i_gid; - } else - inode_init_owner(inode, dir, mode); - - inode->i_ino = ino; - /* This is the optimal IO size (for stat), not the fs block size */ - inode->i_blocks = 0; - inode->i_mtime = inode->i_atime = inode->i_ctime = CURRENT_TIME_SEC; - - memset(ei->i_data, 0, sizeof(ei->i_data)); - ei->i_dir_start_lookup = 0; - ei->i_disksize = 0; - - ei->i_flags = - ext3_mask_flags(mode, EXT3_I(dir)->i_flags & EXT3_FL_INHERITED); -#ifdef EXT3_FRAGMENTS - ei->i_faddr = 0; - ei->i_frag_no = 0; - ei->i_frag_size = 0; -#endif - ei->i_file_acl = 0; - ei->i_dir_acl = 0; - ei->i_dtime = 0; - ei->i_block_alloc_info = NULL; - ei->i_block_group = group; - - ext3_set_inode_flags(inode); - if (IS_DIRSYNC(inode)) - handle->h_sync = 1; - if (insert_inode_locked(inode) < 0) { - /* - * Likely a bitmap corruption causing inode to be allocated - * twice. - */ - err = -EIO; - goto fail; - } - spin_lock(&sbi->s_next_gen_lock); - inode->i_generation = sbi->s_next_generation++; - spin_unlock(&sbi->s_next_gen_lock); - - ei->i_state_flags = 0; - ext3_set_inode_state(inode, EXT3_STATE_NEW); - - /* See comment in ext3_iget for explanation */ - if (ino >= EXT3_FIRST_INO(sb) + 1 && - EXT3_INODE_SIZE(sb) > EXT3_GOOD_OLD_INODE_SIZE) { - ei->i_extra_isize = - sizeof(struct ext3_inode) - EXT3_GOOD_OLD_INODE_SIZE; - } else { - ei->i_extra_isize = 0; - } - - ret = inode; - dquot_initialize(inode); - err = dquot_alloc_inode(inode); - if (err) - goto fail_drop; - - err = ext3_init_acl(handle, inode, dir); - if (err) - goto fail_free_drop; - - err = ext3_init_security(handle, inode, dir, qstr); - if (err) - goto fail_free_drop; - - err = ext3_mark_inode_dirty(handle, inode); - if (err) { - ext3_std_error(sb, err); - goto fail_free_drop; - } - - ext3_debug("allocating inode %lu\n", inode->i_ino); - trace_ext3_allocate_inode(inode, dir, mode); - goto really_out; -fail: - ext3_std_error(sb, err); -out: - iput(inode); - ret = ERR_PTR(err); -really_out: - brelse(bitmap_bh); - return ret; - -fail_free_drop: - dquot_free_inode(inode); - -fail_drop: - dquot_drop(inode); - inode->i_flags |= S_NOQUOTA; - clear_nlink(inode); - unlock_new_inode(inode); - iput(inode); - brelse(bitmap_bh); - return ERR_PTR(err); -} - -/* Verify that we are loading a valid orphan from disk */ -struct inode *ext3_orphan_get(struct super_block *sb, unsigned long ino) -{ - unsigned long max_ino = le32_to_cpu(EXT3_SB(sb)->s_es->s_inodes_count); - unsigned long block_group; - int bit; - struct buffer_head *bitmap_bh; - struct inode *inode = NULL; - long err = -EIO; - - /* Error cases - e2fsck has already cleaned up for us */ - if (ino > max_ino) { - ext3_warning(sb, __func__, - "bad orphan ino %lu! e2fsck was run?", ino); - goto error; - } - - block_group = (ino - 1) / EXT3_INODES_PER_GROUP(sb); - bit = (ino - 1) % EXT3_INODES_PER_GROUP(sb); - bitmap_bh = read_inode_bitmap(sb, block_group); - if (!bitmap_bh) { - ext3_warning(sb, __func__, - "inode bitmap error for orphan %lu", ino); - goto error; - } - - /* Having the inode bit set should be a 100% indicator that this - * is a valid orphan (no e2fsck run on fs). Orphans also include - * inodes that were being truncated, so we can't check i_nlink==0. - */ - if (!ext3_test_bit(bit, bitmap_bh->b_data)) - goto bad_orphan; - - inode = ext3_iget(sb, ino); - if (IS_ERR(inode)) - goto iget_failed; - - /* - * If the orphans has i_nlinks > 0 then it should be able to be - * truncated, otherwise it won't be removed from the orphan list - * during processing and an infinite loop will result. - */ - if (inode->i_nlink && !ext3_can_truncate(inode)) - goto bad_orphan; - - if (NEXT_ORPHAN(inode) > max_ino) - goto bad_orphan; - brelse(bitmap_bh); - return inode; - -iget_failed: - err = PTR_ERR(inode); - inode = NULL; -bad_orphan: - ext3_warning(sb, __func__, - "bad orphan inode %lu! e2fsck was run?", ino); - printk(KERN_NOTICE "ext3_test_bit(bit=%d, block=%llu) = %d\n", - bit, (unsigned long long)bitmap_bh->b_blocknr, - ext3_test_bit(bit, bitmap_bh->b_data)); - printk(KERN_NOTICE "inode=%p\n", inode); - if (inode) { - printk(KERN_NOTICE "is_bad_inode(inode)=%d\n", - is_bad_inode(inode)); - printk(KERN_NOTICE "NEXT_ORPHAN(inode)=%u\n", - NEXT_ORPHAN(inode)); - printk(KERN_NOTICE "max_ino=%lu\n", max_ino); - printk(KERN_NOTICE "i_nlink=%u\n", inode->i_nlink); - /* Avoid freeing blocks if we got a bad deleted inode */ - if (inode->i_nlink == 0) - inode->i_blocks = 0; - iput(inode); - } - brelse(bitmap_bh); -error: - return ERR_PTR(err); -} - -unsigned long ext3_count_free_inodes (struct super_block * sb) -{ - unsigned long desc_count; - struct ext3_group_desc *gdp; - int i; -#ifdef EXT3FS_DEBUG - struct ext3_super_block *es; - unsigned long bitmap_count, x; - struct buffer_head *bitmap_bh = NULL; - - es = EXT3_SB(sb)->s_es; - desc_count = 0; - bitmap_count = 0; - gdp = NULL; - for (i = 0; i < EXT3_SB(sb)->s_groups_count; i++) { - gdp = ext3_get_group_desc (sb, i, NULL); - if (!gdp) - continue; - desc_count += le16_to_cpu(gdp->bg_free_inodes_count); - brelse(bitmap_bh); - bitmap_bh = read_inode_bitmap(sb, i); - if (!bitmap_bh) - continue; - - x = ext3_count_free(bitmap_bh, EXT3_INODES_PER_GROUP(sb) / 8); - printk("group %d: stored = %d, counted = %lu\n", - i, le16_to_cpu(gdp->bg_free_inodes_count), x); - bitmap_count += x; - } - brelse(bitmap_bh); - printk("ext3_count_free_inodes: stored = %u, computed = %lu, %lu\n", - le32_to_cpu(es->s_free_inodes_count), desc_count, bitmap_count); - return desc_count; -#else - desc_count = 0; - for (i = 0; i < EXT3_SB(sb)->s_groups_count; i++) { - gdp = ext3_get_group_desc (sb, i, NULL); - if (!gdp) - continue; - desc_count += le16_to_cpu(gdp->bg_free_inodes_count); - cond_resched(); - } - return desc_count; -#endif -} - -/* Called at mount-time, super-block is locked */ -unsigned long ext3_count_dirs (struct super_block * sb) -{ - unsigned long count = 0; - int i; - - for (i = 0; i < EXT3_SB(sb)->s_groups_count; i++) { - struct ext3_group_desc *gdp = ext3_get_group_desc (sb, i, NULL); - if (!gdp) - continue; - count += le16_to_cpu(gdp->bg_used_dirs_count); - } - return count; -} - diff --git a/fs/ext3/inode.c b/fs/ext3/inode.c deleted file mode 100644 index 6c7e5468a2f807..00000000000000 --- a/fs/ext3/inode.c +++ /dev/null @@ -1,3574 +0,0 @@ -/* - * linux/fs/ext3/inode.c - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * - * from - * - * linux/fs/minix/inode.c - * - * Copyright (C) 1991, 1992 Linus Torvalds - * - * Goal-directed block allocation by Stephen Tweedie - * (sct@redhat.com), 1993, 1998 - * Big-endian to little-endian byte-swapping/bitmaps by - * David S. Miller (davem@caip.rutgers.edu), 1995 - * 64-bit file support on 64-bit platforms by Jakub Jelinek - * (jj@sunsite.ms.mff.cuni.cz) - * - * Assorted race fixes, rewrite of ext3_get_block() by Al Viro, 2000 - */ - -#include -#include -#include -#include -#include -#include -#include "ext3.h" -#include "xattr.h" -#include "acl.h" - -static int ext3_writepage_trans_blocks(struct inode *inode); -static int ext3_block_truncate_page(struct inode *inode, loff_t from); - -/* - * Test whether an inode is a fast symlink. - */ -static int ext3_inode_is_fast_symlink(struct inode *inode) -{ - int ea_blocks = EXT3_I(inode)->i_file_acl ? - (inode->i_sb->s_blocksize >> 9) : 0; - - return (S_ISLNK(inode->i_mode) && inode->i_blocks - ea_blocks == 0); -} - -/* - * The ext3 forget function must perform a revoke if we are freeing data - * which has been journaled. Metadata (eg. indirect blocks) must be - * revoked in all cases. - * - * "bh" may be NULL: a metadata block may have been freed from memory - * but there may still be a record of it in the journal, and that record - * still needs to be revoked. - */ -int ext3_forget(handle_t *handle, int is_metadata, struct inode *inode, - struct buffer_head *bh, ext3_fsblk_t blocknr) -{ - int err; - - might_sleep(); - - trace_ext3_forget(inode, is_metadata, blocknr); - BUFFER_TRACE(bh, "enter"); - - jbd_debug(4, "forgetting bh %p: is_metadata = %d, mode %o, " - "data mode %lx\n", - bh, is_metadata, inode->i_mode, - test_opt(inode->i_sb, DATA_FLAGS)); - - /* Never use the revoke function if we are doing full data - * journaling: there is no need to, and a V1 superblock won't - * support it. Otherwise, only skip the revoke on un-journaled - * data blocks. */ - - if (test_opt(inode->i_sb, DATA_FLAGS) == EXT3_MOUNT_JOURNAL_DATA || - (!is_metadata && !ext3_should_journal_data(inode))) { - if (bh) { - BUFFER_TRACE(bh, "call journal_forget"); - return ext3_journal_forget(handle, bh); - } - return 0; - } - - /* - * data!=journal && (is_metadata || should_journal_data(inode)) - */ - BUFFER_TRACE(bh, "call ext3_journal_revoke"); - err = ext3_journal_revoke(handle, blocknr, bh); - if (err) - ext3_abort(inode->i_sb, __func__, - "error %d when attempting revoke", err); - BUFFER_TRACE(bh, "exit"); - return err; -} - -/* - * Work out how many blocks we need to proceed with the next chunk of a - * truncate transaction. - */ -static unsigned long blocks_for_truncate(struct inode *inode) -{ - unsigned long needed; - - needed = inode->i_blocks >> (inode->i_sb->s_blocksize_bits - 9); - - /* Give ourselves just enough room to cope with inodes in which - * i_blocks is corrupt: we've seen disk corruptions in the past - * which resulted in random data in an inode which looked enough - * like a regular file for ext3 to try to delete it. Things - * will go a bit crazy if that happens, but at least we should - * try not to panic the whole kernel. */ - if (needed < 2) - needed = 2; - - /* But we need to bound the transaction so we don't overflow the - * journal. */ - if (needed > EXT3_MAX_TRANS_DATA) - needed = EXT3_MAX_TRANS_DATA; - - return EXT3_DATA_TRANS_BLOCKS(inode->i_sb) + needed; -} - -/* - * Truncate transactions can be complex and absolutely huge. So we need to - * be able to restart the transaction at a conventient checkpoint to make - * sure we don't overflow the journal. - * - * start_transaction gets us a new handle for a truncate transaction, - * and extend_transaction tries to extend the existing one a bit. If - * extend fails, we need to propagate the failure up and restart the - * transaction in the top-level truncate loop. --sct - */ -static handle_t *start_transaction(struct inode *inode) -{ - handle_t *result; - - result = ext3_journal_start(inode, blocks_for_truncate(inode)); - if (!IS_ERR(result)) - return result; - - ext3_std_error(inode->i_sb, PTR_ERR(result)); - return result; -} - -/* - * Try to extend this transaction for the purposes of truncation. - * - * Returns 0 if we managed to create more room. If we can't create more - * room, and the transaction must be restarted we return 1. - */ -static int try_to_extend_transaction(handle_t *handle, struct inode *inode) -{ - if (handle->h_buffer_credits > EXT3_RESERVE_TRANS_BLOCKS) - return 0; - if (!ext3_journal_extend(handle, blocks_for_truncate(inode))) - return 0; - return 1; -} - -/* - * Restart the transaction associated with *handle. This does a commit, - * so before we call here everything must be consistently dirtied against - * this transaction. - */ -static int truncate_restart_transaction(handle_t *handle, struct inode *inode) -{ - int ret; - - jbd_debug(2, "restarting handle %p\n", handle); - /* - * Drop truncate_mutex to avoid deadlock with ext3_get_blocks_handle - * At this moment, get_block can be called only for blocks inside - * i_size since page cache has been already dropped and writes are - * blocked by i_mutex. So we can safely drop the truncate_mutex. - */ - mutex_unlock(&EXT3_I(inode)->truncate_mutex); - ret = ext3_journal_restart(handle, blocks_for_truncate(inode)); - mutex_lock(&EXT3_I(inode)->truncate_mutex); - return ret; -} - -/* - * Called at inode eviction from icache - */ -void ext3_evict_inode (struct inode *inode) -{ - struct ext3_inode_info *ei = EXT3_I(inode); - struct ext3_block_alloc_info *rsv; - handle_t *handle; - int want_delete = 0; - - trace_ext3_evict_inode(inode); - if (!inode->i_nlink && !is_bad_inode(inode)) { - dquot_initialize(inode); - want_delete = 1; - } - - /* - * When journalling data dirty buffers are tracked only in the journal. - * So although mm thinks everything is clean and ready for reaping the - * inode might still have some pages to write in the running - * transaction or waiting to be checkpointed. Thus calling - * journal_invalidatepage() (via truncate_inode_pages()) to discard - * these buffers can cause data loss. Also even if we did not discard - * these buffers, we would have no way to find them after the inode - * is reaped and thus user could see stale data if he tries to read - * them before the transaction is checkpointed. So be careful and - * force everything to disk here... We use ei->i_datasync_tid to - * store the newest transaction containing inode's data. - * - * Note that directories do not have this problem because they don't - * use page cache. - * - * The s_journal check handles the case when ext3_get_journal() fails - * and puts the journal inode. - */ - if (inode->i_nlink && ext3_should_journal_data(inode) && - EXT3_SB(inode->i_sb)->s_journal && - (S_ISLNK(inode->i_mode) || S_ISREG(inode->i_mode)) && - inode->i_ino != EXT3_JOURNAL_INO) { - tid_t commit_tid = atomic_read(&ei->i_datasync_tid); - journal_t *journal = EXT3_SB(inode->i_sb)->s_journal; - - log_start_commit(journal, commit_tid); - log_wait_commit(journal, commit_tid); - filemap_write_and_wait(&inode->i_data); - } - truncate_inode_pages_final(&inode->i_data); - - ext3_discard_reservation(inode); - rsv = ei->i_block_alloc_info; - ei->i_block_alloc_info = NULL; - if (unlikely(rsv)) - kfree(rsv); - - if (!want_delete) - goto no_delete; - - handle = start_transaction(inode); - if (IS_ERR(handle)) { - /* - * If we're going to skip the normal cleanup, we still need to - * make sure that the in-core orphan linked list is properly - * cleaned up. - */ - ext3_orphan_del(NULL, inode); - goto no_delete; - } - - if (IS_SYNC(inode)) - handle->h_sync = 1; - inode->i_size = 0; - if (inode->i_blocks) - ext3_truncate(inode); - /* - * Kill off the orphan record created when the inode lost the last - * link. Note that ext3_orphan_del() has to be able to cope with the - * deletion of a non-existent orphan - ext3_truncate() could - * have removed the record. - */ - ext3_orphan_del(handle, inode); - ei->i_dtime = get_seconds(); - - /* - * One subtle ordering requirement: if anything has gone wrong - * (transaction abort, IO errors, whatever), then we can still - * do these next steps (the fs will already have been marked as - * having errors), but we can't free the inode if the mark_dirty - * fails. - */ - if (ext3_mark_inode_dirty(handle, inode)) { - /* If that failed, just dquot_drop() and be done with that */ - dquot_drop(inode); - clear_inode(inode); - } else { - ext3_xattr_delete_inode(handle, inode); - dquot_free_inode(inode); - dquot_drop(inode); - clear_inode(inode); - ext3_free_inode(handle, inode); - } - ext3_journal_stop(handle); - return; -no_delete: - clear_inode(inode); - dquot_drop(inode); -} - -typedef struct { - __le32 *p; - __le32 key; - struct buffer_head *bh; -} Indirect; - -static inline void add_chain(Indirect *p, struct buffer_head *bh, __le32 *v) -{ - p->key = *(p->p = v); - p->bh = bh; -} - -static int verify_chain(Indirect *from, Indirect *to) -{ - while (from <= to && from->key == *from->p) - from++; - return (from > to); -} - -/** - * ext3_block_to_path - parse the block number into array of offsets - * @inode: inode in question (we are only interested in its superblock) - * @i_block: block number to be parsed - * @offsets: array to store the offsets in - * @boundary: set this non-zero if the referred-to block is likely to be - * followed (on disk) by an indirect block. - * - * To store the locations of file's data ext3 uses a data structure common - * for UNIX filesystems - tree of pointers anchored in the inode, with - * data blocks at leaves and indirect blocks in intermediate nodes. - * This function translates the block number into path in that tree - - * return value is the path length and @offsets[n] is the offset of - * pointer to (n+1)th node in the nth one. If @block is out of range - * (negative or too large) warning is printed and zero returned. - * - * Note: function doesn't find node addresses, so no IO is needed. All - * we need to know is the capacity of indirect blocks (taken from the - * inode->i_sb). - */ - -/* - * Portability note: the last comparison (check that we fit into triple - * indirect block) is spelled differently, because otherwise on an - * architecture with 32-bit longs and 8Kb pages we might get into trouble - * if our filesystem had 8Kb blocks. We might use long long, but that would - * kill us on x86. Oh, well, at least the sign propagation does not matter - - * i_block would have to be negative in the very beginning, so we would not - * get there at all. - */ - -static int ext3_block_to_path(struct inode *inode, - long i_block, int offsets[4], int *boundary) -{ - int ptrs = EXT3_ADDR_PER_BLOCK(inode->i_sb); - int ptrs_bits = EXT3_ADDR_PER_BLOCK_BITS(inode->i_sb); - const long direct_blocks = EXT3_NDIR_BLOCKS, - indirect_blocks = ptrs, - double_blocks = (1 << (ptrs_bits * 2)); - int n = 0; - int final = 0; - - if (i_block < 0) { - ext3_warning (inode->i_sb, "ext3_block_to_path", "block < 0"); - } else if (i_block < direct_blocks) { - offsets[n++] = i_block; - final = direct_blocks; - } else if ( (i_block -= direct_blocks) < indirect_blocks) { - offsets[n++] = EXT3_IND_BLOCK; - offsets[n++] = i_block; - final = ptrs; - } else if ((i_block -= indirect_blocks) < double_blocks) { - offsets[n++] = EXT3_DIND_BLOCK; - offsets[n++] = i_block >> ptrs_bits; - offsets[n++] = i_block & (ptrs - 1); - final = ptrs; - } else if (((i_block -= double_blocks) >> (ptrs_bits * 2)) < ptrs) { - offsets[n++] = EXT3_TIND_BLOCK; - offsets[n++] = i_block >> (ptrs_bits * 2); - offsets[n++] = (i_block >> ptrs_bits) & (ptrs - 1); - offsets[n++] = i_block & (ptrs - 1); - final = ptrs; - } else { - ext3_warning(inode->i_sb, "ext3_block_to_path", "block > big"); - } - if (boundary) - *boundary = final - 1 - (i_block & (ptrs - 1)); - return n; -} - -/** - * ext3_get_branch - read the chain of indirect blocks leading to data - * @inode: inode in question - * @depth: depth of the chain (1 - direct pointer, etc.) - * @offsets: offsets of pointers in inode/indirect blocks - * @chain: place to store the result - * @err: here we store the error value - * - * Function fills the array of triples and returns %NULL - * if everything went OK or the pointer to the last filled triple - * (incomplete one) otherwise. Upon the return chain[i].key contains - * the number of (i+1)-th block in the chain (as it is stored in memory, - * i.e. little-endian 32-bit), chain[i].p contains the address of that - * number (it points into struct inode for i==0 and into the bh->b_data - * for i>0) and chain[i].bh points to the buffer_head of i-th indirect - * block for i>0 and NULL for i==0. In other words, it holds the block - * numbers of the chain, addresses they were taken from (and where we can - * verify that chain did not change) and buffer_heads hosting these - * numbers. - * - * Function stops when it stumbles upon zero pointer (absent block) - * (pointer to last triple returned, *@err == 0) - * or when it gets an IO error reading an indirect block - * (ditto, *@err == -EIO) - * or when it notices that chain had been changed while it was reading - * (ditto, *@err == -EAGAIN) - * or when it reads all @depth-1 indirect blocks successfully and finds - * the whole chain, all way to the data (returns %NULL, *err == 0). - */ -static Indirect *ext3_get_branch(struct inode *inode, int depth, int *offsets, - Indirect chain[4], int *err) -{ - struct super_block *sb = inode->i_sb; - Indirect *p = chain; - struct buffer_head *bh; - - *err = 0; - /* i_data is not going away, no lock needed */ - add_chain (chain, NULL, EXT3_I(inode)->i_data + *offsets); - if (!p->key) - goto no_block; - while (--depth) { - bh = sb_bread(sb, le32_to_cpu(p->key)); - if (!bh) - goto failure; - /* Reader: pointers */ - if (!verify_chain(chain, p)) - goto changed; - add_chain(++p, bh, (__le32*)bh->b_data + *++offsets); - /* Reader: end */ - if (!p->key) - goto no_block; - } - return NULL; - -changed: - brelse(bh); - *err = -EAGAIN; - goto no_block; -failure: - *err = -EIO; -no_block: - return p; -} - -/** - * ext3_find_near - find a place for allocation with sufficient locality - * @inode: owner - * @ind: descriptor of indirect block. - * - * This function returns the preferred place for block allocation. - * It is used when heuristic for sequential allocation fails. - * Rules are: - * + if there is a block to the left of our position - allocate near it. - * + if pointer will live in indirect block - allocate near that block. - * + if pointer will live in inode - allocate in the same - * cylinder group. - * - * In the latter case we colour the starting block by the callers PID to - * prevent it from clashing with concurrent allocations for a different inode - * in the same block group. The PID is used here so that functionally related - * files will be close-by on-disk. - * - * Caller must make sure that @ind is valid and will stay that way. - */ -static ext3_fsblk_t ext3_find_near(struct inode *inode, Indirect *ind) -{ - struct ext3_inode_info *ei = EXT3_I(inode); - __le32 *start = ind->bh ? (__le32*) ind->bh->b_data : ei->i_data; - __le32 *p; - ext3_fsblk_t bg_start; - ext3_grpblk_t colour; - - /* Try to find previous block */ - for (p = ind->p - 1; p >= start; p--) { - if (*p) - return le32_to_cpu(*p); - } - - /* No such thing, so let's try location of indirect block */ - if (ind->bh) - return ind->bh->b_blocknr; - - /* - * It is going to be referred to from the inode itself? OK, just put it - * into the same cylinder group then. - */ - bg_start = ext3_group_first_block_no(inode->i_sb, ei->i_block_group); - colour = (current->pid % 16) * - (EXT3_BLOCKS_PER_GROUP(inode->i_sb) / 16); - return bg_start + colour; -} - -/** - * ext3_find_goal - find a preferred place for allocation. - * @inode: owner - * @block: block we want - * @partial: pointer to the last triple within a chain - * - * Normally this function find the preferred place for block allocation, - * returns it. - */ - -static ext3_fsblk_t ext3_find_goal(struct inode *inode, long block, - Indirect *partial) -{ - struct ext3_block_alloc_info *block_i; - - block_i = EXT3_I(inode)->i_block_alloc_info; - - /* - * try the heuristic for sequential allocation, - * failing that at least try to get decent locality. - */ - if (block_i && (block == block_i->last_alloc_logical_block + 1) - && (block_i->last_alloc_physical_block != 0)) { - return block_i->last_alloc_physical_block + 1; - } - - return ext3_find_near(inode, partial); -} - -/** - * ext3_blks_to_allocate - Look up the block map and count the number - * of direct blocks need to be allocated for the given branch. - * - * @branch: chain of indirect blocks - * @k: number of blocks need for indirect blocks - * @blks: number of data blocks to be mapped. - * @blocks_to_boundary: the offset in the indirect block - * - * return the total number of blocks to be allocate, including the - * direct and indirect blocks. - */ -static int ext3_blks_to_allocate(Indirect *branch, int k, unsigned long blks, - int blocks_to_boundary) -{ - unsigned long count = 0; - - /* - * Simple case, [t,d]Indirect block(s) has not allocated yet - * then it's clear blocks on that path have not allocated - */ - if (k > 0) { - /* right now we don't handle cross boundary allocation */ - if (blks < blocks_to_boundary + 1) - count += blks; - else - count += blocks_to_boundary + 1; - return count; - } - - count++; - while (count < blks && count <= blocks_to_boundary && - le32_to_cpu(*(branch[0].p + count)) == 0) { - count++; - } - return count; -} - -/** - * ext3_alloc_blocks - multiple allocate blocks needed for a branch - * @handle: handle for this transaction - * @inode: owner - * @goal: preferred place for allocation - * @indirect_blks: the number of blocks need to allocate for indirect - * blocks - * @blks: number of blocks need to allocated for direct blocks - * @new_blocks: on return it will store the new block numbers for - * the indirect blocks(if needed) and the first direct block, - * @err: here we store the error value - * - * return the number of direct blocks allocated - */ -static int ext3_alloc_blocks(handle_t *handle, struct inode *inode, - ext3_fsblk_t goal, int indirect_blks, int blks, - ext3_fsblk_t new_blocks[4], int *err) -{ - int target, i; - unsigned long count = 0; - int index = 0; - ext3_fsblk_t current_block = 0; - int ret = 0; - - /* - * Here we try to allocate the requested multiple blocks at once, - * on a best-effort basis. - * To build a branch, we should allocate blocks for - * the indirect blocks(if not allocated yet), and at least - * the first direct block of this branch. That's the - * minimum number of blocks need to allocate(required) - */ - target = blks + indirect_blks; - - while (1) { - count = target; - /* allocating blocks for indirect blocks and direct blocks */ - current_block = ext3_new_blocks(handle,inode,goal,&count,err); - if (*err) - goto failed_out; - - target -= count; - /* allocate blocks for indirect blocks */ - while (index < indirect_blks && count) { - new_blocks[index++] = current_block++; - count--; - } - - if (count > 0) - break; - } - - /* save the new block number for the first direct block */ - new_blocks[index] = current_block; - - /* total number of blocks allocated for direct blocks */ - ret = count; - *err = 0; - return ret; -failed_out: - for (i = 0; i key). Upon the exit we have the same - * picture as after the successful ext3_get_block(), except that in one - * place chain is disconnected - *branch->p is still zero (we did not - * set the last link), but branch->key contains the number that should - * be placed into *branch->p to fill that gap. - * - * If allocation fails we free all blocks we've allocated (and forget - * their buffer_heads) and return the error value the from failed - * ext3_alloc_block() (normally -ENOSPC). Otherwise we set the chain - * as described above and return 0. - */ -static int ext3_alloc_branch(handle_t *handle, struct inode *inode, - int indirect_blks, int *blks, ext3_fsblk_t goal, - int *offsets, Indirect *branch) -{ - int blocksize = inode->i_sb->s_blocksize; - int i, n = 0; - int err = 0; - struct buffer_head *bh; - int num; - ext3_fsblk_t new_blocks[4]; - ext3_fsblk_t current_block; - - num = ext3_alloc_blocks(handle, inode, goal, indirect_blks, - *blks, new_blocks, &err); - if (err) - return err; - - branch[0].key = cpu_to_le32(new_blocks[0]); - /* - * metadata blocks and data blocks are allocated. - */ - for (n = 1; n <= indirect_blks; n++) { - /* - * Get buffer_head for parent block, zero it out - * and set the pointer to new one, then send - * parent to disk. - */ - bh = sb_getblk(inode->i_sb, new_blocks[n-1]); - if (unlikely(!bh)) { - err = -ENOMEM; - goto failed; - } - branch[n].bh = bh; - lock_buffer(bh); - BUFFER_TRACE(bh, "call get_create_access"); - err = ext3_journal_get_create_access(handle, bh); - if (err) { - unlock_buffer(bh); - brelse(bh); - goto failed; - } - - memset(bh->b_data, 0, blocksize); - branch[n].p = (__le32 *) bh->b_data + offsets[n]; - branch[n].key = cpu_to_le32(new_blocks[n]); - *branch[n].p = branch[n].key; - if ( n == indirect_blks) { - current_block = new_blocks[n]; - /* - * End of chain, update the last new metablock of - * the chain to point to the new allocated - * data blocks numbers - */ - for (i=1; i < num; i++) - *(branch[n].p + i) = cpu_to_le32(++current_block); - } - BUFFER_TRACE(bh, "marking uptodate"); - set_buffer_uptodate(bh); - unlock_buffer(bh); - - BUFFER_TRACE(bh, "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, bh); - if (err) - goto failed; - } - *blks = num; - return err; -failed: - /* Allocation failed, free what we already allocated */ - for (i = 1; i <= n ; i++) { - BUFFER_TRACE(branch[i].bh, "call journal_forget"); - ext3_journal_forget(handle, branch[i].bh); - } - for (i = 0; i < indirect_blks; i++) - ext3_free_blocks(handle, inode, new_blocks[i], 1); - - ext3_free_blocks(handle, inode, new_blocks[i], num); - - return err; -} - -/** - * ext3_splice_branch - splice the allocated branch onto inode. - * @handle: handle for this transaction - * @inode: owner - * @block: (logical) number of block we are adding - * @where: location of missing link - * @num: number of indirect blocks we are adding - * @blks: number of direct blocks we are adding - * - * This function fills the missing link and does all housekeeping needed in - * inode (->i_blocks, etc.). In case of success we end up with the full - * chain to new block and return 0. - */ -static int ext3_splice_branch(handle_t *handle, struct inode *inode, - long block, Indirect *where, int num, int blks) -{ - int i; - int err = 0; - struct ext3_block_alloc_info *block_i; - ext3_fsblk_t current_block; - struct ext3_inode_info *ei = EXT3_I(inode); - struct timespec now; - - block_i = ei->i_block_alloc_info; - /* - * If we're splicing into a [td]indirect block (as opposed to the - * inode) then we need to get write access to the [td]indirect block - * before the splice. - */ - if (where->bh) { - BUFFER_TRACE(where->bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, where->bh); - if (err) - goto err_out; - } - /* That's it */ - - *where->p = where->key; - - /* - * Update the host buffer_head or inode to point to more just allocated - * direct blocks blocks - */ - if (num == 0 && blks > 1) { - current_block = le32_to_cpu(where->key) + 1; - for (i = 1; i < blks; i++) - *(where->p + i ) = cpu_to_le32(current_block++); - } - - /* - * update the most recently allocated logical & physical block - * in i_block_alloc_info, to assist find the proper goal block for next - * allocation - */ - if (block_i) { - block_i->last_alloc_logical_block = block + blks - 1; - block_i->last_alloc_physical_block = - le32_to_cpu(where[num].key) + blks - 1; - } - - /* We are done with atomic stuff, now do the rest of housekeeping */ - now = CURRENT_TIME_SEC; - if (!timespec_equal(&inode->i_ctime, &now) || !where->bh) { - inode->i_ctime = now; - ext3_mark_inode_dirty(handle, inode); - } - /* ext3_mark_inode_dirty already updated i_sync_tid */ - atomic_set(&ei->i_datasync_tid, handle->h_transaction->t_tid); - - /* had we spliced it onto indirect block? */ - if (where->bh) { - /* - * If we spliced it onto an indirect block, we haven't - * altered the inode. Note however that if it is being spliced - * onto an indirect block at the very end of the file (the - * file is growing) then we *will* alter the inode to reflect - * the new i_size. But that is not done here - it is done in - * generic_commit_write->__mark_inode_dirty->ext3_dirty_inode. - */ - jbd_debug(5, "splicing indirect only\n"); - BUFFER_TRACE(where->bh, "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, where->bh); - if (err) - goto err_out; - } else { - /* - * OK, we spliced it into the inode itself on a direct block. - * Inode was dirtied above. - */ - jbd_debug(5, "splicing direct\n"); - } - return err; - -err_out: - for (i = 1; i <= num; i++) { - BUFFER_TRACE(where[i].bh, "call journal_forget"); - ext3_journal_forget(handle, where[i].bh); - ext3_free_blocks(handle,inode,le32_to_cpu(where[i-1].key),1); - } - ext3_free_blocks(handle, inode, le32_to_cpu(where[num].key), blks); - - return err; -} - -/* - * Allocation strategy is simple: if we have to allocate something, we will - * have to go the whole way to leaf. So let's do it before attaching anything - * to tree, set linkage between the newborn blocks, write them if sync is - * required, recheck the path, free and repeat if check fails, otherwise - * set the last missing link (that will protect us from any truncate-generated - * removals - all blocks on the path are immune now) and possibly force the - * write on the parent block. - * That has a nice additional property: no special recovery from the failed - * allocations is needed - we simply release blocks and do not touch anything - * reachable from inode. - * - * `handle' can be NULL if create == 0. - * - * The BKL may not be held on entry here. Be sure to take it early. - * return > 0, # of blocks mapped or allocated. - * return = 0, if plain lookup failed. - * return < 0, error case. - */ -int ext3_get_blocks_handle(handle_t *handle, struct inode *inode, - sector_t iblock, unsigned long maxblocks, - struct buffer_head *bh_result, - int create) -{ - int err = -EIO; - int offsets[4]; - Indirect chain[4]; - Indirect *partial; - ext3_fsblk_t goal; - int indirect_blks; - int blocks_to_boundary = 0; - int depth; - struct ext3_inode_info *ei = EXT3_I(inode); - int count = 0; - ext3_fsblk_t first_block = 0; - - - trace_ext3_get_blocks_enter(inode, iblock, maxblocks, create); - J_ASSERT(handle != NULL || create == 0); - depth = ext3_block_to_path(inode,iblock,offsets,&blocks_to_boundary); - - if (depth == 0) - goto out; - - partial = ext3_get_branch(inode, depth, offsets, chain, &err); - - /* Simplest case - block found, no allocation needed */ - if (!partial) { - first_block = le32_to_cpu(chain[depth - 1].key); - clear_buffer_new(bh_result); - count++; - /*map more blocks*/ - while (count < maxblocks && count <= blocks_to_boundary) { - ext3_fsblk_t blk; - - if (!verify_chain(chain, chain + depth - 1)) { - /* - * Indirect block might be removed by - * truncate while we were reading it. - * Handling of that case: forget what we've - * got now. Flag the err as EAGAIN, so it - * will reread. - */ - err = -EAGAIN; - count = 0; - break; - } - blk = le32_to_cpu(*(chain[depth-1].p + count)); - - if (blk == first_block + count) - count++; - else - break; - } - if (err != -EAGAIN) - goto got_it; - } - - /* Next simple case - plain lookup or failed read of indirect block */ - if (!create || err == -EIO) - goto cleanup; - - /* - * Block out ext3_truncate while we alter the tree - */ - mutex_lock(&ei->truncate_mutex); - - /* - * If the indirect block is missing while we are reading - * the chain(ext3_get_branch() returns -EAGAIN err), or - * if the chain has been changed after we grab the semaphore, - * (either because another process truncated this branch, or - * another get_block allocated this branch) re-grab the chain to see if - * the request block has been allocated or not. - * - * Since we already block the truncate/other get_block - * at this point, we will have the current copy of the chain when we - * splice the branch into the tree. - */ - if (err == -EAGAIN || !verify_chain(chain, partial)) { - while (partial > chain) { - brelse(partial->bh); - partial--; - } - partial = ext3_get_branch(inode, depth, offsets, chain, &err); - if (!partial) { - count++; - mutex_unlock(&ei->truncate_mutex); - if (err) - goto cleanup; - clear_buffer_new(bh_result); - goto got_it; - } - } - - /* - * Okay, we need to do block allocation. Lazily initialize the block - * allocation info here if necessary - */ - if (S_ISREG(inode->i_mode) && (!ei->i_block_alloc_info)) - ext3_init_block_alloc_info(inode); - - goal = ext3_find_goal(inode, iblock, partial); - - /* the number of blocks need to allocate for [d,t]indirect blocks */ - indirect_blks = (chain + depth) - partial - 1; - - /* - * Next look up the indirect map to count the totoal number of - * direct blocks to allocate for this branch. - */ - count = ext3_blks_to_allocate(partial, indirect_blks, - maxblocks, blocks_to_boundary); - err = ext3_alloc_branch(handle, inode, indirect_blks, &count, goal, - offsets + (partial - chain), partial); - - /* - * The ext3_splice_branch call will free and forget any buffers - * on the new chain if there is a failure, but that risks using - * up transaction credits, especially for bitmaps where the - * credits cannot be returned. Can we handle this somehow? We - * may need to return -EAGAIN upwards in the worst case. --sct - */ - if (!err) - err = ext3_splice_branch(handle, inode, iblock, - partial, indirect_blks, count); - mutex_unlock(&ei->truncate_mutex); - if (err) - goto cleanup; - - set_buffer_new(bh_result); -got_it: - map_bh(bh_result, inode->i_sb, le32_to_cpu(chain[depth-1].key)); - if (count > blocks_to_boundary) - set_buffer_boundary(bh_result); - err = count; - /* Clean up and exit */ - partial = chain + depth - 1; /* the whole chain */ -cleanup: - while (partial > chain) { - BUFFER_TRACE(partial->bh, "call brelse"); - brelse(partial->bh); - partial--; - } - BUFFER_TRACE(bh_result, "returned"); -out: - trace_ext3_get_blocks_exit(inode, iblock, - depth ? le32_to_cpu(chain[depth-1].key) : 0, - count, err); - return err; -} - -/* Maximum number of blocks we map for direct IO at once. */ -#define DIO_MAX_BLOCKS 4096 -/* - * Number of credits we need for writing DIO_MAX_BLOCKS: - * We need sb + group descriptor + bitmap + inode -> 4 - * For B blocks with A block pointers per block we need: - * 1 (triple ind.) + (B/A/A + 2) (doubly ind.) + (B/A + 2) (indirect). - * If we plug in 4096 for B and 256 for A (for 1KB block size), we get 25. - */ -#define DIO_CREDITS 25 - -static int ext3_get_block(struct inode *inode, sector_t iblock, - struct buffer_head *bh_result, int create) -{ - handle_t *handle = ext3_journal_current_handle(); - int ret = 0, started = 0; - unsigned max_blocks = bh_result->b_size >> inode->i_blkbits; - - if (create && !handle) { /* Direct IO write... */ - if (max_blocks > DIO_MAX_BLOCKS) - max_blocks = DIO_MAX_BLOCKS; - handle = ext3_journal_start(inode, DIO_CREDITS + - EXT3_MAXQUOTAS_TRANS_BLOCKS(inode->i_sb)); - if (IS_ERR(handle)) { - ret = PTR_ERR(handle); - goto out; - } - started = 1; - } - - ret = ext3_get_blocks_handle(handle, inode, iblock, - max_blocks, bh_result, create); - if (ret > 0) { - bh_result->b_size = (ret << inode->i_blkbits); - ret = 0; - } - if (started) - ext3_journal_stop(handle); -out: - return ret; -} - -int ext3_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, - u64 start, u64 len) -{ - return generic_block_fiemap(inode, fieinfo, start, len, - ext3_get_block); -} - -/* - * `handle' can be NULL if create is zero - */ -struct buffer_head *ext3_getblk(handle_t *handle, struct inode *inode, - long block, int create, int *errp) -{ - struct buffer_head dummy; - int fatal = 0, err; - - J_ASSERT(handle != NULL || create == 0); - - dummy.b_state = 0; - dummy.b_blocknr = -1000; - buffer_trace_init(&dummy.b_history); - err = ext3_get_blocks_handle(handle, inode, block, 1, - &dummy, create); - /* - * ext3_get_blocks_handle() returns number of blocks - * mapped. 0 in case of a HOLE. - */ - if (err > 0) { - WARN_ON(err > 1); - err = 0; - } - *errp = err; - if (!err && buffer_mapped(&dummy)) { - struct buffer_head *bh; - bh = sb_getblk(inode->i_sb, dummy.b_blocknr); - if (unlikely(!bh)) { - *errp = -ENOMEM; - goto err; - } - if (buffer_new(&dummy)) { - J_ASSERT(create != 0); - J_ASSERT(handle != NULL); - - /* - * Now that we do not always journal data, we should - * keep in mind whether this should always journal the - * new buffer as metadata. For now, regular file - * writes use ext3_get_block instead, so it's not a - * problem. - */ - lock_buffer(bh); - BUFFER_TRACE(bh, "call get_create_access"); - fatal = ext3_journal_get_create_access(handle, bh); - if (!fatal && !buffer_uptodate(bh)) { - memset(bh->b_data,0,inode->i_sb->s_blocksize); - set_buffer_uptodate(bh); - } - unlock_buffer(bh); - BUFFER_TRACE(bh, "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, bh); - if (!fatal) - fatal = err; - } else { - BUFFER_TRACE(bh, "not a new buffer"); - } - if (fatal) { - *errp = fatal; - brelse(bh); - bh = NULL; - } - return bh; - } -err: - return NULL; -} - -struct buffer_head *ext3_bread(handle_t *handle, struct inode *inode, - int block, int create, int *err) -{ - struct buffer_head * bh; - - bh = ext3_getblk(handle, inode, block, create, err); - if (!bh) - return bh; - if (bh_uptodate_or_lock(bh)) - return bh; - get_bh(bh); - bh->b_end_io = end_buffer_read_sync; - submit_bh(READ | REQ_META | REQ_PRIO, bh); - wait_on_buffer(bh); - if (buffer_uptodate(bh)) - return bh; - put_bh(bh); - *err = -EIO; - return NULL; -} - -static int walk_page_buffers( handle_t *handle, - struct buffer_head *head, - unsigned from, - unsigned to, - int *partial, - int (*fn)( handle_t *handle, - struct buffer_head *bh)) -{ - struct buffer_head *bh; - unsigned block_start, block_end; - unsigned blocksize = head->b_size; - int err, ret = 0; - struct buffer_head *next; - - for ( bh = head, block_start = 0; - ret == 0 && (bh != head || !block_start); - block_start = block_end, bh = next) - { - next = bh->b_this_page; - block_end = block_start + blocksize; - if (block_end <= from || block_start >= to) { - if (partial && !buffer_uptodate(bh)) - *partial = 1; - continue; - } - err = (*fn)(handle, bh); - if (!ret) - ret = err; - } - return ret; -} - -/* - * To preserve ordering, it is essential that the hole instantiation and - * the data write be encapsulated in a single transaction. We cannot - * close off a transaction and start a new one between the ext3_get_block() - * and the commit_write(). So doing the journal_start at the start of - * prepare_write() is the right place. - * - * Also, this function can nest inside ext3_writepage() -> - * block_write_full_page(). In that case, we *know* that ext3_writepage() - * has generated enough buffer credits to do the whole page. So we won't - * block on the journal in that case, which is good, because the caller may - * be PF_MEMALLOC. - * - * By accident, ext3 can be reentered when a transaction is open via - * quota file writes. If we were to commit the transaction while thus - * reentered, there can be a deadlock - we would be holding a quota - * lock, and the commit would never complete if another thread had a - * transaction open and was blocking on the quota lock - a ranking - * violation. - * - * So what we do is to rely on the fact that journal_stop/journal_start - * will _not_ run commit under these circumstances because handle->h_ref - * is elevated. We'll still have enough credits for the tiny quotafile - * write. - */ -static int do_journal_get_write_access(handle_t *handle, - struct buffer_head *bh) -{ - int dirty = buffer_dirty(bh); - int ret; - - if (!buffer_mapped(bh) || buffer_freed(bh)) - return 0; - /* - * __block_prepare_write() could have dirtied some buffers. Clean - * the dirty bit as jbd2_journal_get_write_access() could complain - * otherwise about fs integrity issues. Setting of the dirty bit - * by __block_prepare_write() isn't a real problem here as we clear - * the bit before releasing a page lock and thus writeback cannot - * ever write the buffer. - */ - if (dirty) - clear_buffer_dirty(bh); - ret = ext3_journal_get_write_access(handle, bh); - if (!ret && dirty) - ret = ext3_journal_dirty_metadata(handle, bh); - return ret; -} - -/* - * Truncate blocks that were not used by write. We have to truncate the - * pagecache as well so that corresponding buffers get properly unmapped. - */ -static void ext3_truncate_failed_write(struct inode *inode) -{ - truncate_inode_pages(inode->i_mapping, inode->i_size); - ext3_truncate(inode); -} - -/* - * Truncate blocks that were not used by direct IO write. We have to zero out - * the last file block as well because direct IO might have written to it. - */ -static void ext3_truncate_failed_direct_write(struct inode *inode) -{ - ext3_block_truncate_page(inode, inode->i_size); - ext3_truncate(inode); -} - -static int ext3_write_begin(struct file *file, struct address_space *mapping, - loff_t pos, unsigned len, unsigned flags, - struct page **pagep, void **fsdata) -{ - struct inode *inode = mapping->host; - int ret; - handle_t *handle; - int retries = 0; - struct page *page; - pgoff_t index; - unsigned from, to; - /* Reserve one block more for addition to orphan list in case - * we allocate blocks but write fails for some reason */ - int needed_blocks = ext3_writepage_trans_blocks(inode) + 1; - - trace_ext3_write_begin(inode, pos, len, flags); - - index = pos >> PAGE_CACHE_SHIFT; - from = pos & (PAGE_CACHE_SIZE - 1); - to = from + len; - -retry: - page = grab_cache_page_write_begin(mapping, index, flags); - if (!page) - return -ENOMEM; - *pagep = page; - - handle = ext3_journal_start(inode, needed_blocks); - if (IS_ERR(handle)) { - unlock_page(page); - page_cache_release(page); - ret = PTR_ERR(handle); - goto out; - } - ret = __block_write_begin(page, pos, len, ext3_get_block); - if (ret) - goto write_begin_failed; - - if (ext3_should_journal_data(inode)) { - ret = walk_page_buffers(handle, page_buffers(page), - from, to, NULL, do_journal_get_write_access); - } -write_begin_failed: - if (ret) { - /* - * block_write_begin may have instantiated a few blocks - * outside i_size. Trim these off again. Don't need - * i_size_read because we hold i_mutex. - * - * Add inode to orphan list in case we crash before truncate - * finishes. Do this only if ext3_can_truncate() agrees so - * that orphan processing code is happy. - */ - if (pos + len > inode->i_size && ext3_can_truncate(inode)) - ext3_orphan_add(handle, inode); - ext3_journal_stop(handle); - unlock_page(page); - page_cache_release(page); - if (pos + len > inode->i_size) - ext3_truncate_failed_write(inode); - } - if (ret == -ENOSPC && ext3_should_retry_alloc(inode->i_sb, &retries)) - goto retry; -out: - return ret; -} - - -int ext3_journal_dirty_data(handle_t *handle, struct buffer_head *bh) -{ - int err = journal_dirty_data(handle, bh); - if (err) - ext3_journal_abort_handle(__func__, __func__, - bh, handle, err); - return err; -} - -/* For ordered writepage and write_end functions */ -static int journal_dirty_data_fn(handle_t *handle, struct buffer_head *bh) -{ - /* - * Write could have mapped the buffer but it didn't copy the data in - * yet. So avoid filing such buffer into a transaction. - */ - if (buffer_mapped(bh) && buffer_uptodate(bh)) - return ext3_journal_dirty_data(handle, bh); - return 0; -} - -/* For write_end() in data=journal mode */ -static int write_end_fn(handle_t *handle, struct buffer_head *bh) -{ - if (!buffer_mapped(bh) || buffer_freed(bh)) - return 0; - set_buffer_uptodate(bh); - return ext3_journal_dirty_metadata(handle, bh); -} - -/* - * This is nasty and subtle: ext3_write_begin() could have allocated blocks - * for the whole page but later we failed to copy the data in. Update inode - * size according to what we managed to copy. The rest is going to be - * truncated in write_end function. - */ -static void update_file_sizes(struct inode *inode, loff_t pos, unsigned copied) -{ - /* What matters to us is i_disksize. We don't write i_size anywhere */ - if (pos + copied > inode->i_size) - i_size_write(inode, pos + copied); - if (pos + copied > EXT3_I(inode)->i_disksize) { - EXT3_I(inode)->i_disksize = pos + copied; - mark_inode_dirty(inode); - } -} - -/* - * We need to pick up the new inode size which generic_commit_write gave us - * `file' can be NULL - eg, when called from page_symlink(). - * - * ext3 never places buffers on inode->i_mapping->private_list. metadata - * buffers are managed internally. - */ -static int ext3_ordered_write_end(struct file *file, - struct address_space *mapping, - loff_t pos, unsigned len, unsigned copied, - struct page *page, void *fsdata) -{ - handle_t *handle = ext3_journal_current_handle(); - struct inode *inode = file->f_mapping->host; - unsigned from, to; - int ret = 0, ret2; - - trace_ext3_ordered_write_end(inode, pos, len, copied); - copied = block_write_end(file, mapping, pos, len, copied, page, fsdata); - - from = pos & (PAGE_CACHE_SIZE - 1); - to = from + copied; - ret = walk_page_buffers(handle, page_buffers(page), - from, to, NULL, journal_dirty_data_fn); - - if (ret == 0) - update_file_sizes(inode, pos, copied); - /* - * There may be allocated blocks outside of i_size because - * we failed to copy some data. Prepare for truncate. - */ - if (pos + len > inode->i_size && ext3_can_truncate(inode)) - ext3_orphan_add(handle, inode); - ret2 = ext3_journal_stop(handle); - if (!ret) - ret = ret2; - unlock_page(page); - page_cache_release(page); - - if (pos + len > inode->i_size) - ext3_truncate_failed_write(inode); - return ret ? ret : copied; -} - -static int ext3_writeback_write_end(struct file *file, - struct address_space *mapping, - loff_t pos, unsigned len, unsigned copied, - struct page *page, void *fsdata) -{ - handle_t *handle = ext3_journal_current_handle(); - struct inode *inode = file->f_mapping->host; - int ret; - - trace_ext3_writeback_write_end(inode, pos, len, copied); - copied = block_write_end(file, mapping, pos, len, copied, page, fsdata); - update_file_sizes(inode, pos, copied); - /* - * There may be allocated blocks outside of i_size because - * we failed to copy some data. Prepare for truncate. - */ - if (pos + len > inode->i_size && ext3_can_truncate(inode)) - ext3_orphan_add(handle, inode); - ret = ext3_journal_stop(handle); - unlock_page(page); - page_cache_release(page); - - if (pos + len > inode->i_size) - ext3_truncate_failed_write(inode); - return ret ? ret : copied; -} - -static int ext3_journalled_write_end(struct file *file, - struct address_space *mapping, - loff_t pos, unsigned len, unsigned copied, - struct page *page, void *fsdata) -{ - handle_t *handle = ext3_journal_current_handle(); - struct inode *inode = mapping->host; - struct ext3_inode_info *ei = EXT3_I(inode); - int ret = 0, ret2; - int partial = 0; - unsigned from, to; - - trace_ext3_journalled_write_end(inode, pos, len, copied); - from = pos & (PAGE_CACHE_SIZE - 1); - to = from + len; - - if (copied < len) { - if (!PageUptodate(page)) - copied = 0; - page_zero_new_buffers(page, from + copied, to); - to = from + copied; - } - - ret = walk_page_buffers(handle, page_buffers(page), from, - to, &partial, write_end_fn); - if (!partial) - SetPageUptodate(page); - - if (pos + copied > inode->i_size) - i_size_write(inode, pos + copied); - /* - * There may be allocated blocks outside of i_size because - * we failed to copy some data. Prepare for truncate. - */ - if (pos + len > inode->i_size && ext3_can_truncate(inode)) - ext3_orphan_add(handle, inode); - ext3_set_inode_state(inode, EXT3_STATE_JDATA); - atomic_set(&ei->i_datasync_tid, handle->h_transaction->t_tid); - if (inode->i_size > ei->i_disksize) { - ei->i_disksize = inode->i_size; - ret2 = ext3_mark_inode_dirty(handle, inode); - if (!ret) - ret = ret2; - } - - ret2 = ext3_journal_stop(handle); - if (!ret) - ret = ret2; - unlock_page(page); - page_cache_release(page); - - if (pos + len > inode->i_size) - ext3_truncate_failed_write(inode); - return ret ? ret : copied; -} - -/* - * bmap() is special. It gets used by applications such as lilo and by - * the swapper to find the on-disk block of a specific piece of data. - * - * Naturally, this is dangerous if the block concerned is still in the - * journal. If somebody makes a swapfile on an ext3 data-journaling - * filesystem and enables swap, then they may get a nasty shock when the - * data getting swapped to that swapfile suddenly gets overwritten by - * the original zero's written out previously to the journal and - * awaiting writeback in the kernel's buffer cache. - * - * So, if we see any bmap calls here on a modified, data-journaled file, - * take extra steps to flush any blocks which might be in the cache. - */ -static sector_t ext3_bmap(struct address_space *mapping, sector_t block) -{ - struct inode *inode = mapping->host; - journal_t *journal; - int err; - - if (ext3_test_inode_state(inode, EXT3_STATE_JDATA)) { - /* - * This is a REALLY heavyweight approach, but the use of - * bmap on dirty files is expected to be extremely rare: - * only if we run lilo or swapon on a freshly made file - * do we expect this to happen. - * - * (bmap requires CAP_SYS_RAWIO so this does not - * represent an unprivileged user DOS attack --- we'd be - * in trouble if mortal users could trigger this path at - * will.) - * - * NB. EXT3_STATE_JDATA is not set on files other than - * regular files. If somebody wants to bmap a directory - * or symlink and gets confused because the buffer - * hasn't yet been flushed to disk, they deserve - * everything they get. - */ - - ext3_clear_inode_state(inode, EXT3_STATE_JDATA); - journal = EXT3_JOURNAL(inode); - journal_lock_updates(journal); - err = journal_flush(journal); - journal_unlock_updates(journal); - - if (err) - return 0; - } - - return generic_block_bmap(mapping,block,ext3_get_block); -} - -static int bget_one(handle_t *handle, struct buffer_head *bh) -{ - get_bh(bh); - return 0; -} - -static int bput_one(handle_t *handle, struct buffer_head *bh) -{ - put_bh(bh); - return 0; -} - -static int buffer_unmapped(handle_t *handle, struct buffer_head *bh) -{ - return !buffer_mapped(bh); -} - -/* - * Note that whenever we need to map blocks we start a transaction even if - * we're not journalling data. This is to preserve ordering: any hole - * instantiation within __block_write_full_page -> ext3_get_block() should be - * journalled along with the data so we don't crash and then get metadata which - * refers to old data. - * - * In all journalling modes block_write_full_page() will start the I/O. - * - * We don't honour synchronous mounts for writepage(). That would be - * disastrous. Any write() or metadata operation will sync the fs for - * us. - */ -static int ext3_ordered_writepage(struct page *page, - struct writeback_control *wbc) -{ - struct inode *inode = page->mapping->host; - struct buffer_head *page_bufs; - handle_t *handle = NULL; - int ret = 0; - int err; - - J_ASSERT(PageLocked(page)); - /* - * We don't want to warn for emergency remount. The condition is - * ordered to avoid dereferencing inode->i_sb in non-error case to - * avoid slow-downs. - */ - WARN_ON_ONCE(IS_RDONLY(inode) && - !(EXT3_SB(inode->i_sb)->s_mount_state & EXT3_ERROR_FS)); - - /* - * We give up here if we're reentered, because it might be for a - * different filesystem. - */ - if (ext3_journal_current_handle()) - goto out_fail; - - trace_ext3_ordered_writepage(page); - if (!page_has_buffers(page)) { - create_empty_buffers(page, inode->i_sb->s_blocksize, - (1 << BH_Dirty)|(1 << BH_Uptodate)); - page_bufs = page_buffers(page); - } else { - page_bufs = page_buffers(page); - if (!walk_page_buffers(NULL, page_bufs, 0, PAGE_CACHE_SIZE, - NULL, buffer_unmapped)) { - /* Provide NULL get_block() to catch bugs if buffers - * weren't really mapped */ - return block_write_full_page(page, NULL, wbc); - } - } - handle = ext3_journal_start(inode, ext3_writepage_trans_blocks(inode)); - - if (IS_ERR(handle)) { - ret = PTR_ERR(handle); - goto out_fail; - } - - walk_page_buffers(handle, page_bufs, 0, - PAGE_CACHE_SIZE, NULL, bget_one); - - ret = block_write_full_page(page, ext3_get_block, wbc); - - /* - * The page can become unlocked at any point now, and - * truncate can then come in and change things. So we - * can't touch *page from now on. But *page_bufs is - * safe due to elevated refcount. - */ - - /* - * And attach them to the current transaction. But only if - * block_write_full_page() succeeded. Otherwise they are unmapped, - * and generally junk. - */ - if (ret == 0) - ret = walk_page_buffers(handle, page_bufs, 0, PAGE_CACHE_SIZE, - NULL, journal_dirty_data_fn); - walk_page_buffers(handle, page_bufs, 0, - PAGE_CACHE_SIZE, NULL, bput_one); - err = ext3_journal_stop(handle); - if (!ret) - ret = err; - return ret; - -out_fail: - redirty_page_for_writepage(wbc, page); - unlock_page(page); - return ret; -} - -static int ext3_writeback_writepage(struct page *page, - struct writeback_control *wbc) -{ - struct inode *inode = page->mapping->host; - handle_t *handle = NULL; - int ret = 0; - int err; - - J_ASSERT(PageLocked(page)); - /* - * We don't want to warn for emergency remount. The condition is - * ordered to avoid dereferencing inode->i_sb in non-error case to - * avoid slow-downs. - */ - WARN_ON_ONCE(IS_RDONLY(inode) && - !(EXT3_SB(inode->i_sb)->s_mount_state & EXT3_ERROR_FS)); - - if (ext3_journal_current_handle()) - goto out_fail; - - trace_ext3_writeback_writepage(page); - if (page_has_buffers(page)) { - if (!walk_page_buffers(NULL, page_buffers(page), 0, - PAGE_CACHE_SIZE, NULL, buffer_unmapped)) { - /* Provide NULL get_block() to catch bugs if buffers - * weren't really mapped */ - return block_write_full_page(page, NULL, wbc); - } - } - - handle = ext3_journal_start(inode, ext3_writepage_trans_blocks(inode)); - if (IS_ERR(handle)) { - ret = PTR_ERR(handle); - goto out_fail; - } - - ret = block_write_full_page(page, ext3_get_block, wbc); - - err = ext3_journal_stop(handle); - if (!ret) - ret = err; - return ret; - -out_fail: - redirty_page_for_writepage(wbc, page); - unlock_page(page); - return ret; -} - -static int ext3_journalled_writepage(struct page *page, - struct writeback_control *wbc) -{ - struct inode *inode = page->mapping->host; - handle_t *handle = NULL; - int ret = 0; - int err; - - J_ASSERT(PageLocked(page)); - /* - * We don't want to warn for emergency remount. The condition is - * ordered to avoid dereferencing inode->i_sb in non-error case to - * avoid slow-downs. - */ - WARN_ON_ONCE(IS_RDONLY(inode) && - !(EXT3_SB(inode->i_sb)->s_mount_state & EXT3_ERROR_FS)); - - trace_ext3_journalled_writepage(page); - if (!page_has_buffers(page) || PageChecked(page)) { - if (ext3_journal_current_handle()) - goto no_write; - - handle = ext3_journal_start(inode, - ext3_writepage_trans_blocks(inode)); - if (IS_ERR(handle)) { - ret = PTR_ERR(handle); - goto no_write; - } - /* - * It's mmapped pagecache. Add buffers and journal it. There - * doesn't seem much point in redirtying the page here. - */ - ClearPageChecked(page); - ret = __block_write_begin(page, 0, PAGE_CACHE_SIZE, - ext3_get_block); - if (ret != 0) { - ext3_journal_stop(handle); - goto out_unlock; - } - ret = walk_page_buffers(handle, page_buffers(page), 0, - PAGE_CACHE_SIZE, NULL, do_journal_get_write_access); - - err = walk_page_buffers(handle, page_buffers(page), 0, - PAGE_CACHE_SIZE, NULL, write_end_fn); - if (ret == 0) - ret = err; - ext3_set_inode_state(inode, EXT3_STATE_JDATA); - atomic_set(&EXT3_I(inode)->i_datasync_tid, - handle->h_transaction->t_tid); - unlock_page(page); - err = ext3_journal_stop(handle); - if (!ret) - ret = err; - } else { - /* - * It is a page full of checkpoint-mode buffers. Go and write - * them. They should have been already mapped when they went - * to the journal so provide NULL get_block function to catch - * errors. - */ - ret = block_write_full_page(page, NULL, wbc); - } -out: - return ret; - -no_write: - redirty_page_for_writepage(wbc, page); -out_unlock: - unlock_page(page); - goto out; -} - -static int ext3_readpage(struct file *file, struct page *page) -{ - trace_ext3_readpage(page); - return mpage_readpage(page, ext3_get_block); -} - -static int -ext3_readpages(struct file *file, struct address_space *mapping, - struct list_head *pages, unsigned nr_pages) -{ - return mpage_readpages(mapping, pages, nr_pages, ext3_get_block); -} - -static void ext3_invalidatepage(struct page *page, unsigned int offset, - unsigned int length) -{ - journal_t *journal = EXT3_JOURNAL(page->mapping->host); - - trace_ext3_invalidatepage(page, offset, length); - - /* - * If it's a full truncate we just forget about the pending dirtying - */ - if (offset == 0 && length == PAGE_CACHE_SIZE) - ClearPageChecked(page); - - journal_invalidatepage(journal, page, offset, length); -} - -static int ext3_releasepage(struct page *page, gfp_t wait) -{ - journal_t *journal = EXT3_JOURNAL(page->mapping->host); - - trace_ext3_releasepage(page); - WARN_ON(PageChecked(page)); - if (!page_has_buffers(page)) - return 0; - return journal_try_to_free_buffers(journal, page, wait); -} - -/* - * If the O_DIRECT write will extend the file then add this inode to the - * orphan list. So recovery will truncate it back to the original size - * if the machine crashes during the write. - * - * If the O_DIRECT write is intantiating holes inside i_size and the machine - * crashes then stale disk data _may_ be exposed inside the file. But current - * VFS code falls back into buffered path in that case so we are safe. - */ -static ssize_t ext3_direct_IO(struct kiocb *iocb, struct iov_iter *iter, - loff_t offset) -{ - struct file *file = iocb->ki_filp; - struct inode *inode = file->f_mapping->host; - struct ext3_inode_info *ei = EXT3_I(inode); - handle_t *handle; - ssize_t ret; - int orphan = 0; - size_t count = iov_iter_count(iter); - int retries = 0; - - trace_ext3_direct_IO_enter(inode, offset, count, iov_iter_rw(iter)); - - if (iov_iter_rw(iter) == WRITE) { - loff_t final_size = offset + count; - - if (final_size > inode->i_size) { - /* Credits for sb + inode write */ - handle = ext3_journal_start(inode, 2); - if (IS_ERR(handle)) { - ret = PTR_ERR(handle); - goto out; - } - ret = ext3_orphan_add(handle, inode); - if (ret) { - ext3_journal_stop(handle); - goto out; - } - orphan = 1; - ei->i_disksize = inode->i_size; - ext3_journal_stop(handle); - } - } - -retry: - ret = blockdev_direct_IO(iocb, inode, iter, offset, ext3_get_block); - /* - * In case of error extending write may have instantiated a few - * blocks outside i_size. Trim these off again. - */ - if (unlikely(iov_iter_rw(iter) == WRITE && ret < 0)) { - loff_t isize = i_size_read(inode); - loff_t end = offset + count; - - if (end > isize) - ext3_truncate_failed_direct_write(inode); - } - if (ret == -ENOSPC && ext3_should_retry_alloc(inode->i_sb, &retries)) - goto retry; - - if (orphan) { - int err; - - /* Credits for sb + inode write */ - handle = ext3_journal_start(inode, 2); - if (IS_ERR(handle)) { - /* This is really bad luck. We've written the data - * but cannot extend i_size. Truncate allocated blocks - * and pretend the write failed... */ - ext3_truncate_failed_direct_write(inode); - ret = PTR_ERR(handle); - if (inode->i_nlink) - ext3_orphan_del(NULL, inode); - goto out; - } - if (inode->i_nlink) - ext3_orphan_del(handle, inode); - if (ret > 0) { - loff_t end = offset + ret; - if (end > inode->i_size) { - ei->i_disksize = end; - i_size_write(inode, end); - /* - * We're going to return a positive `ret' - * here due to non-zero-length I/O, so there's - * no way of reporting error returns from - * ext3_mark_inode_dirty() to userspace. So - * ignore it. - */ - ext3_mark_inode_dirty(handle, inode); - } - } - err = ext3_journal_stop(handle); - if (ret == 0) - ret = err; - } -out: - trace_ext3_direct_IO_exit(inode, offset, count, iov_iter_rw(iter), ret); - return ret; -} - -/* - * Pages can be marked dirty completely asynchronously from ext3's journalling - * activity. By filemap_sync_pte(), try_to_unmap_one(), etc. We cannot do - * much here because ->set_page_dirty is called under VFS locks. The page is - * not necessarily locked. - * - * We cannot just dirty the page and leave attached buffers clean, because the - * buffers' dirty state is "definitive". We cannot just set the buffers dirty - * or jbddirty because all the journalling code will explode. - * - * So what we do is to mark the page "pending dirty" and next time writepage - * is called, propagate that into the buffers appropriately. - */ -static int ext3_journalled_set_page_dirty(struct page *page) -{ - SetPageChecked(page); - return __set_page_dirty_nobuffers(page); -} - -static const struct address_space_operations ext3_ordered_aops = { - .readpage = ext3_readpage, - .readpages = ext3_readpages, - .writepage = ext3_ordered_writepage, - .write_begin = ext3_write_begin, - .write_end = ext3_ordered_write_end, - .bmap = ext3_bmap, - .invalidatepage = ext3_invalidatepage, - .releasepage = ext3_releasepage, - .direct_IO = ext3_direct_IO, - .migratepage = buffer_migrate_page, - .is_partially_uptodate = block_is_partially_uptodate, - .is_dirty_writeback = buffer_check_dirty_writeback, - .error_remove_page = generic_error_remove_page, -}; - -static const struct address_space_operations ext3_writeback_aops = { - .readpage = ext3_readpage, - .readpages = ext3_readpages, - .writepage = ext3_writeback_writepage, - .write_begin = ext3_write_begin, - .write_end = ext3_writeback_write_end, - .bmap = ext3_bmap, - .invalidatepage = ext3_invalidatepage, - .releasepage = ext3_releasepage, - .direct_IO = ext3_direct_IO, - .migratepage = buffer_migrate_page, - .is_partially_uptodate = block_is_partially_uptodate, - .error_remove_page = generic_error_remove_page, -}; - -static const struct address_space_operations ext3_journalled_aops = { - .readpage = ext3_readpage, - .readpages = ext3_readpages, - .writepage = ext3_journalled_writepage, - .write_begin = ext3_write_begin, - .write_end = ext3_journalled_write_end, - .set_page_dirty = ext3_journalled_set_page_dirty, - .bmap = ext3_bmap, - .invalidatepage = ext3_invalidatepage, - .releasepage = ext3_releasepage, - .is_partially_uptodate = block_is_partially_uptodate, - .error_remove_page = generic_error_remove_page, -}; - -void ext3_set_aops(struct inode *inode) -{ - if (ext3_should_order_data(inode)) - inode->i_mapping->a_ops = &ext3_ordered_aops; - else if (ext3_should_writeback_data(inode)) - inode->i_mapping->a_ops = &ext3_writeback_aops; - else - inode->i_mapping->a_ops = &ext3_journalled_aops; -} - -/* - * ext3_block_truncate_page() zeroes out a mapping from file offset `from' - * up to the end of the block which corresponds to `from'. - * This required during truncate. We need to physically zero the tail end - * of that block so it doesn't yield old data if the file is later grown. - */ -static int ext3_block_truncate_page(struct inode *inode, loff_t from) -{ - ext3_fsblk_t index = from >> PAGE_CACHE_SHIFT; - unsigned offset = from & (PAGE_CACHE_SIZE - 1); - unsigned blocksize, iblock, length, pos; - struct page *page; - handle_t *handle = NULL; - struct buffer_head *bh; - int err = 0; - - /* Truncated on block boundary - nothing to do */ - blocksize = inode->i_sb->s_blocksize; - if ((from & (blocksize - 1)) == 0) - return 0; - - page = grab_cache_page(inode->i_mapping, index); - if (!page) - return -ENOMEM; - length = blocksize - (offset & (blocksize - 1)); - iblock = index << (PAGE_CACHE_SHIFT - inode->i_sb->s_blocksize_bits); - - if (!page_has_buffers(page)) - create_empty_buffers(page, blocksize, 0); - - /* Find the buffer that contains "offset" */ - bh = page_buffers(page); - pos = blocksize; - while (offset >= pos) { - bh = bh->b_this_page; - iblock++; - pos += blocksize; - } - - err = 0; - if (buffer_freed(bh)) { - BUFFER_TRACE(bh, "freed: skip"); - goto unlock; - } - - if (!buffer_mapped(bh)) { - BUFFER_TRACE(bh, "unmapped"); - ext3_get_block(inode, iblock, bh, 0); - /* unmapped? It's a hole - nothing to do */ - if (!buffer_mapped(bh)) { - BUFFER_TRACE(bh, "still unmapped"); - goto unlock; - } - } - - /* Ok, it's mapped. Make sure it's up-to-date */ - if (PageUptodate(page)) - set_buffer_uptodate(bh); - - if (!bh_uptodate_or_lock(bh)) { - err = bh_submit_read(bh); - /* Uhhuh. Read error. Complain and punt. */ - if (err) - goto unlock; - } - - /* data=writeback mode doesn't need transaction to zero-out data */ - if (!ext3_should_writeback_data(inode)) { - /* We journal at most one block */ - handle = ext3_journal_start(inode, 1); - if (IS_ERR(handle)) { - clear_highpage(page); - flush_dcache_page(page); - err = PTR_ERR(handle); - goto unlock; - } - } - - if (ext3_should_journal_data(inode)) { - BUFFER_TRACE(bh, "get write access"); - err = ext3_journal_get_write_access(handle, bh); - if (err) - goto stop; - } - - zero_user(page, offset, length); - BUFFER_TRACE(bh, "zeroed end of block"); - - err = 0; - if (ext3_should_journal_data(inode)) { - err = ext3_journal_dirty_metadata(handle, bh); - } else { - if (ext3_should_order_data(inode)) - err = ext3_journal_dirty_data(handle, bh); - mark_buffer_dirty(bh); - } -stop: - if (handle) - ext3_journal_stop(handle); - -unlock: - unlock_page(page); - page_cache_release(page); - return err; -} - -/* - * Probably it should be a library function... search for first non-zero word - * or memcmp with zero_page, whatever is better for particular architecture. - * Linus? - */ -static inline int all_zeroes(__le32 *p, __le32 *q) -{ - while (p < q) - if (*p++) - return 0; - return 1; -} - -/** - * ext3_find_shared - find the indirect blocks for partial truncation. - * @inode: inode in question - * @depth: depth of the affected branch - * @offsets: offsets of pointers in that branch (see ext3_block_to_path) - * @chain: place to store the pointers to partial indirect blocks - * @top: place to the (detached) top of branch - * - * This is a helper function used by ext3_truncate(). - * - * When we do truncate() we may have to clean the ends of several - * indirect blocks but leave the blocks themselves alive. Block is - * partially truncated if some data below the new i_size is referred - * from it (and it is on the path to the first completely truncated - * data block, indeed). We have to free the top of that path along - * with everything to the right of the path. Since no allocation - * past the truncation point is possible until ext3_truncate() - * finishes, we may safely do the latter, but top of branch may - * require special attention - pageout below the truncation point - * might try to populate it. - * - * We atomically detach the top of branch from the tree, store the - * block number of its root in *@top, pointers to buffer_heads of - * partially truncated blocks - in @chain[].bh and pointers to - * their last elements that should not be removed - in - * @chain[].p. Return value is the pointer to last filled element - * of @chain. - * - * The work left to caller to do the actual freeing of subtrees: - * a) free the subtree starting from *@top - * b) free the subtrees whose roots are stored in - * (@chain[i].p+1 .. end of @chain[i].bh->b_data) - * c) free the subtrees growing from the inode past the @chain[0]. - * (no partially truncated stuff there). */ - -static Indirect *ext3_find_shared(struct inode *inode, int depth, - int offsets[4], Indirect chain[4], __le32 *top) -{ - Indirect *partial, *p; - int k, err; - - *top = 0; - /* Make k index the deepest non-null offset + 1 */ - for (k = depth; k > 1 && !offsets[k-1]; k--) - ; - partial = ext3_get_branch(inode, k, offsets, chain, &err); - /* Writer: pointers */ - if (!partial) - partial = chain + k-1; - /* - * If the branch acquired continuation since we've looked at it - - * fine, it should all survive and (new) top doesn't belong to us. - */ - if (!partial->key && *partial->p) - /* Writer: end */ - goto no_top; - for (p=partial; p>chain && all_zeroes((__le32*)p->bh->b_data,p->p); p--) - ; - /* - * OK, we've found the last block that must survive. The rest of our - * branch should be detached before unlocking. However, if that rest - * of branch is all ours and does not grow immediately from the inode - * it's easier to cheat and just decrement partial->p. - */ - if (p == chain + k - 1 && p > chain) { - p->p--; - } else { - *top = *p->p; - /* Nope, don't do this in ext3. Must leave the tree intact */ -#if 0 - *p->p = 0; -#endif - } - /* Writer: end */ - - while(partial > p) { - brelse(partial->bh); - partial--; - } -no_top: - return partial; -} - -/* - * Zero a number of block pointers in either an inode or an indirect block. - * If we restart the transaction we must again get write access to the - * indirect block for further modification. - * - * We release `count' blocks on disk, but (last - first) may be greater - * than `count' because there can be holes in there. - */ -static void ext3_clear_blocks(handle_t *handle, struct inode *inode, - struct buffer_head *bh, ext3_fsblk_t block_to_free, - unsigned long count, __le32 *first, __le32 *last) -{ - __le32 *p; - if (try_to_extend_transaction(handle, inode)) { - if (bh) { - BUFFER_TRACE(bh, "call ext3_journal_dirty_metadata"); - if (ext3_journal_dirty_metadata(handle, bh)) - return; - } - ext3_mark_inode_dirty(handle, inode); - truncate_restart_transaction(handle, inode); - if (bh) { - BUFFER_TRACE(bh, "retaking write access"); - if (ext3_journal_get_write_access(handle, bh)) - return; - } - } - - /* - * Any buffers which are on the journal will be in memory. We find - * them on the hash table so journal_revoke() will run journal_forget() - * on them. We've already detached each block from the file, so - * bforget() in journal_forget() should be safe. - * - * AKPM: turn on bforget in journal_forget()!!! - */ - for (p = first; p < last; p++) { - u32 nr = le32_to_cpu(*p); - if (nr) { - struct buffer_head *bh; - - *p = 0; - bh = sb_find_get_block(inode->i_sb, nr); - ext3_forget(handle, 0, inode, bh, nr); - } - } - - ext3_free_blocks(handle, inode, block_to_free, count); -} - -/** - * ext3_free_data - free a list of data blocks - * @handle: handle for this transaction - * @inode: inode we are dealing with - * @this_bh: indirect buffer_head which contains *@first and *@last - * @first: array of block numbers - * @last: points immediately past the end of array - * - * We are freeing all blocks referred from that array (numbers are stored as - * little-endian 32-bit) and updating @inode->i_blocks appropriately. - * - * We accumulate contiguous runs of blocks to free. Conveniently, if these - * blocks are contiguous then releasing them at one time will only affect one - * or two bitmap blocks (+ group descriptor(s) and superblock) and we won't - * actually use a lot of journal space. - * - * @this_bh will be %NULL if @first and @last point into the inode's direct - * block pointers. - */ -static void ext3_free_data(handle_t *handle, struct inode *inode, - struct buffer_head *this_bh, - __le32 *first, __le32 *last) -{ - ext3_fsblk_t block_to_free = 0; /* Starting block # of a run */ - unsigned long count = 0; /* Number of blocks in the run */ - __le32 *block_to_free_p = NULL; /* Pointer into inode/ind - corresponding to - block_to_free */ - ext3_fsblk_t nr; /* Current block # */ - __le32 *p; /* Pointer into inode/ind - for current block */ - int err; - - if (this_bh) { /* For indirect block */ - BUFFER_TRACE(this_bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, this_bh); - /* Important: if we can't update the indirect pointers - * to the blocks, we can't free them. */ - if (err) - return; - } - - for (p = first; p < last; p++) { - nr = le32_to_cpu(*p); - if (nr) { - /* accumulate blocks to free if they're contiguous */ - if (count == 0) { - block_to_free = nr; - block_to_free_p = p; - count = 1; - } else if (nr == block_to_free + count) { - count++; - } else { - ext3_clear_blocks(handle, inode, this_bh, - block_to_free, - count, block_to_free_p, p); - block_to_free = nr; - block_to_free_p = p; - count = 1; - } - } - } - - if (count > 0) - ext3_clear_blocks(handle, inode, this_bh, block_to_free, - count, block_to_free_p, p); - - if (this_bh) { - BUFFER_TRACE(this_bh, "call ext3_journal_dirty_metadata"); - - /* - * The buffer head should have an attached journal head at this - * point. However, if the data is corrupted and an indirect - * block pointed to itself, it would have been detached when - * the block was cleared. Check for this instead of OOPSing. - */ - if (bh2jh(this_bh)) - ext3_journal_dirty_metadata(handle, this_bh); - else - ext3_error(inode->i_sb, "ext3_free_data", - "circular indirect block detected, " - "inode=%lu, block=%llu", - inode->i_ino, - (unsigned long long)this_bh->b_blocknr); - } -} - -/** - * ext3_free_branches - free an array of branches - * @handle: JBD handle for this transaction - * @inode: inode we are dealing with - * @parent_bh: the buffer_head which contains *@first and *@last - * @first: array of block numbers - * @last: pointer immediately past the end of array - * @depth: depth of the branches to free - * - * We are freeing all blocks referred from these branches (numbers are - * stored as little-endian 32-bit) and updating @inode->i_blocks - * appropriately. - */ -static void ext3_free_branches(handle_t *handle, struct inode *inode, - struct buffer_head *parent_bh, - __le32 *first, __le32 *last, int depth) -{ - ext3_fsblk_t nr; - __le32 *p; - - if (is_handle_aborted(handle)) - return; - - if (depth--) { - struct buffer_head *bh; - int addr_per_block = EXT3_ADDR_PER_BLOCK(inode->i_sb); - p = last; - while (--p >= first) { - nr = le32_to_cpu(*p); - if (!nr) - continue; /* A hole */ - - /* Go read the buffer for the next level down */ - bh = sb_bread(inode->i_sb, nr); - - /* - * A read failure? Report error and clear slot - * (should be rare). - */ - if (!bh) { - ext3_error(inode->i_sb, "ext3_free_branches", - "Read failure, inode=%lu, block="E3FSBLK, - inode->i_ino, nr); - continue; - } - - /* This zaps the entire block. Bottom up. */ - BUFFER_TRACE(bh, "free child branches"); - ext3_free_branches(handle, inode, bh, - (__le32*)bh->b_data, - (__le32*)bh->b_data + addr_per_block, - depth); - - /* - * Everything below this this pointer has been - * released. Now let this top-of-subtree go. - * - * We want the freeing of this indirect block to be - * atomic in the journal with the updating of the - * bitmap block which owns it. So make some room in - * the journal. - * - * We zero the parent pointer *after* freeing its - * pointee in the bitmaps, so if extend_transaction() - * for some reason fails to put the bitmap changes and - * the release into the same transaction, recovery - * will merely complain about releasing a free block, - * rather than leaking blocks. - */ - if (is_handle_aborted(handle)) - return; - if (try_to_extend_transaction(handle, inode)) { - ext3_mark_inode_dirty(handle, inode); - truncate_restart_transaction(handle, inode); - } - - /* - * We've probably journalled the indirect block several - * times during the truncate. But it's no longer - * needed and we now drop it from the transaction via - * journal_revoke(). - * - * That's easy if it's exclusively part of this - * transaction. But if it's part of the committing - * transaction then journal_forget() will simply - * brelse() it. That means that if the underlying - * block is reallocated in ext3_get_block(), - * unmap_underlying_metadata() will find this block - * and will try to get rid of it. damn, damn. Thus - * we don't allow a block to be reallocated until - * a transaction freeing it has fully committed. - * - * We also have to make sure journal replay after a - * crash does not overwrite non-journaled data blocks - * with old metadata when the block got reallocated for - * data. Thus we have to store a revoke record for a - * block in the same transaction in which we free the - * block. - */ - ext3_forget(handle, 1, inode, bh, bh->b_blocknr); - - ext3_free_blocks(handle, inode, nr, 1); - - if (parent_bh) { - /* - * The block which we have just freed is - * pointed to by an indirect block: journal it - */ - BUFFER_TRACE(parent_bh, "get_write_access"); - if (!ext3_journal_get_write_access(handle, - parent_bh)){ - *p = 0; - BUFFER_TRACE(parent_bh, - "call ext3_journal_dirty_metadata"); - ext3_journal_dirty_metadata(handle, - parent_bh); - } - } - } - } else { - /* We have reached the bottom of the tree. */ - BUFFER_TRACE(parent_bh, "free data blocks"); - ext3_free_data(handle, inode, parent_bh, first, last); - } -} - -int ext3_can_truncate(struct inode *inode) -{ - if (S_ISREG(inode->i_mode)) - return 1; - if (S_ISDIR(inode->i_mode)) - return 1; - if (S_ISLNK(inode->i_mode)) - return !ext3_inode_is_fast_symlink(inode); - return 0; -} - -/* - * ext3_truncate() - * - * We block out ext3_get_block() block instantiations across the entire - * transaction, and VFS/VM ensures that ext3_truncate() cannot run - * simultaneously on behalf of the same inode. - * - * As we work through the truncate and commit bits of it to the journal there - * is one core, guiding principle: the file's tree must always be consistent on - * disk. We must be able to restart the truncate after a crash. - * - * The file's tree may be transiently inconsistent in memory (although it - * probably isn't), but whenever we close off and commit a journal transaction, - * the contents of (the filesystem + the journal) must be consistent and - * restartable. It's pretty simple, really: bottom up, right to left (although - * left-to-right works OK too). - * - * Note that at recovery time, journal replay occurs *before* the restart of - * truncate against the orphan inode list. - * - * The committed inode has the new, desired i_size (which is the same as - * i_disksize in this case). After a crash, ext3_orphan_cleanup() will see - * that this inode's truncate did not complete and it will again call - * ext3_truncate() to have another go. So there will be instantiated blocks - * to the right of the truncation point in a crashed ext3 filesystem. But - * that's fine - as long as they are linked from the inode, the post-crash - * ext3_truncate() run will find them and release them. - */ -void ext3_truncate(struct inode *inode) -{ - handle_t *handle; - struct ext3_inode_info *ei = EXT3_I(inode); - __le32 *i_data = ei->i_data; - int addr_per_block = EXT3_ADDR_PER_BLOCK(inode->i_sb); - int offsets[4]; - Indirect chain[4]; - Indirect *partial; - __le32 nr = 0; - int n; - long last_block; - unsigned blocksize = inode->i_sb->s_blocksize; - - trace_ext3_truncate_enter(inode); - - if (!ext3_can_truncate(inode)) - goto out_notrans; - - if (inode->i_size == 0 && ext3_should_writeback_data(inode)) - ext3_set_inode_state(inode, EXT3_STATE_FLUSH_ON_CLOSE); - - handle = start_transaction(inode); - if (IS_ERR(handle)) - goto out_notrans; - - last_block = (inode->i_size + blocksize-1) - >> EXT3_BLOCK_SIZE_BITS(inode->i_sb); - n = ext3_block_to_path(inode, last_block, offsets, NULL); - if (n == 0) - goto out_stop; /* error */ - - /* - * OK. This truncate is going to happen. We add the inode to the - * orphan list, so that if this truncate spans multiple transactions, - * and we crash, we will resume the truncate when the filesystem - * recovers. It also marks the inode dirty, to catch the new size. - * - * Implication: the file must always be in a sane, consistent - * truncatable state while each transaction commits. - */ - if (ext3_orphan_add(handle, inode)) - goto out_stop; - - /* - * The orphan list entry will now protect us from any crash which - * occurs before the truncate completes, so it is now safe to propagate - * the new, shorter inode size (held for now in i_size) into the - * on-disk inode. We do this via i_disksize, which is the value which - * ext3 *really* writes onto the disk inode. - */ - ei->i_disksize = inode->i_size; - - /* - * From here we block out all ext3_get_block() callers who want to - * modify the block allocation tree. - */ - mutex_lock(&ei->truncate_mutex); - - if (n == 1) { /* direct blocks */ - ext3_free_data(handle, inode, NULL, i_data+offsets[0], - i_data + EXT3_NDIR_BLOCKS); - goto do_indirects; - } - - partial = ext3_find_shared(inode, n, offsets, chain, &nr); - /* Kill the top of shared branch (not detached) */ - if (nr) { - if (partial == chain) { - /* Shared branch grows from the inode */ - ext3_free_branches(handle, inode, NULL, - &nr, &nr+1, (chain+n-1) - partial); - *partial->p = 0; - /* - * We mark the inode dirty prior to restart, - * and prior to stop. No need for it here. - */ - } else { - /* Shared branch grows from an indirect block */ - ext3_free_branches(handle, inode, partial->bh, - partial->p, - partial->p+1, (chain+n-1) - partial); - } - } - /* Clear the ends of indirect blocks on the shared branch */ - while (partial > chain) { - ext3_free_branches(handle, inode, partial->bh, partial->p + 1, - (__le32*)partial->bh->b_data+addr_per_block, - (chain+n-1) - partial); - BUFFER_TRACE(partial->bh, "call brelse"); - brelse (partial->bh); - partial--; - } -do_indirects: - /* Kill the remaining (whole) subtrees */ - switch (offsets[0]) { - default: - nr = i_data[EXT3_IND_BLOCK]; - if (nr) { - ext3_free_branches(handle, inode, NULL, &nr, &nr+1, 1); - i_data[EXT3_IND_BLOCK] = 0; - } - case EXT3_IND_BLOCK: - nr = i_data[EXT3_DIND_BLOCK]; - if (nr) { - ext3_free_branches(handle, inode, NULL, &nr, &nr+1, 2); - i_data[EXT3_DIND_BLOCK] = 0; - } - case EXT3_DIND_BLOCK: - nr = i_data[EXT3_TIND_BLOCK]; - if (nr) { - ext3_free_branches(handle, inode, NULL, &nr, &nr+1, 3); - i_data[EXT3_TIND_BLOCK] = 0; - } - case EXT3_TIND_BLOCK: - ; - } - - ext3_discard_reservation(inode); - - mutex_unlock(&ei->truncate_mutex); - inode->i_mtime = inode->i_ctime = CURRENT_TIME_SEC; - ext3_mark_inode_dirty(handle, inode); - - /* - * In a multi-transaction truncate, we only make the final transaction - * synchronous - */ - if (IS_SYNC(inode)) - handle->h_sync = 1; -out_stop: - /* - * If this was a simple ftruncate(), and the file will remain alive - * then we need to clear up the orphan record which we created above. - * However, if this was a real unlink then we were called by - * ext3_evict_inode(), and we allow that function to clean up the - * orphan info for us. - */ - if (inode->i_nlink) - ext3_orphan_del(handle, inode); - - ext3_journal_stop(handle); - trace_ext3_truncate_exit(inode); - return; -out_notrans: - /* - * Delete the inode from orphan list so that it doesn't stay there - * forever and trigger assertion on umount. - */ - if (inode->i_nlink) - ext3_orphan_del(NULL, inode); - trace_ext3_truncate_exit(inode); -} - -static ext3_fsblk_t ext3_get_inode_block(struct super_block *sb, - unsigned long ino, struct ext3_iloc *iloc) -{ - unsigned long block_group; - unsigned long offset; - ext3_fsblk_t block; - struct ext3_group_desc *gdp; - - if (!ext3_valid_inum(sb, ino)) { - /* - * This error is already checked for in namei.c unless we are - * looking at an NFS filehandle, in which case no error - * report is needed - */ - return 0; - } - - block_group = (ino - 1) / EXT3_INODES_PER_GROUP(sb); - gdp = ext3_get_group_desc(sb, block_group, NULL); - if (!gdp) - return 0; - /* - * Figure out the offset within the block group inode table - */ - offset = ((ino - 1) % EXT3_INODES_PER_GROUP(sb)) * - EXT3_INODE_SIZE(sb); - block = le32_to_cpu(gdp->bg_inode_table) + - (offset >> EXT3_BLOCK_SIZE_BITS(sb)); - - iloc->block_group = block_group; - iloc->offset = offset & (EXT3_BLOCK_SIZE(sb) - 1); - return block; -} - -/* - * ext3_get_inode_loc returns with an extra refcount against the inode's - * underlying buffer_head on success. If 'in_mem' is true, we have all - * data in memory that is needed to recreate the on-disk version of this - * inode. - */ -static int __ext3_get_inode_loc(struct inode *inode, - struct ext3_iloc *iloc, int in_mem) -{ - ext3_fsblk_t block; - struct buffer_head *bh; - - block = ext3_get_inode_block(inode->i_sb, inode->i_ino, iloc); - if (!block) - return -EIO; - - bh = sb_getblk(inode->i_sb, block); - if (unlikely(!bh)) { - ext3_error (inode->i_sb, "ext3_get_inode_loc", - "unable to read inode block - " - "inode=%lu, block="E3FSBLK, - inode->i_ino, block); - return -ENOMEM; - } - if (!buffer_uptodate(bh)) { - lock_buffer(bh); - - /* - * If the buffer has the write error flag, we have failed - * to write out another inode in the same block. In this - * case, we don't have to read the block because we may - * read the old inode data successfully. - */ - if (buffer_write_io_error(bh) && !buffer_uptodate(bh)) - set_buffer_uptodate(bh); - - if (buffer_uptodate(bh)) { - /* someone brought it uptodate while we waited */ - unlock_buffer(bh); - goto has_buffer; - } - - /* - * If we have all information of the inode in memory and this - * is the only valid inode in the block, we need not read the - * block. - */ - if (in_mem) { - struct buffer_head *bitmap_bh; - struct ext3_group_desc *desc; - int inodes_per_buffer; - int inode_offset, i; - int block_group; - int start; - - block_group = (inode->i_ino - 1) / - EXT3_INODES_PER_GROUP(inode->i_sb); - inodes_per_buffer = bh->b_size / - EXT3_INODE_SIZE(inode->i_sb); - inode_offset = ((inode->i_ino - 1) % - EXT3_INODES_PER_GROUP(inode->i_sb)); - start = inode_offset & ~(inodes_per_buffer - 1); - - /* Is the inode bitmap in cache? */ - desc = ext3_get_group_desc(inode->i_sb, - block_group, NULL); - if (!desc) - goto make_io; - - bitmap_bh = sb_getblk(inode->i_sb, - le32_to_cpu(desc->bg_inode_bitmap)); - if (unlikely(!bitmap_bh)) - goto make_io; - - /* - * If the inode bitmap isn't in cache then the - * optimisation may end up performing two reads instead - * of one, so skip it. - */ - if (!buffer_uptodate(bitmap_bh)) { - brelse(bitmap_bh); - goto make_io; - } - for (i = start; i < start + inodes_per_buffer; i++) { - if (i == inode_offset) - continue; - if (ext3_test_bit(i, bitmap_bh->b_data)) - break; - } - brelse(bitmap_bh); - if (i == start + inodes_per_buffer) { - /* all other inodes are free, so skip I/O */ - memset(bh->b_data, 0, bh->b_size); - set_buffer_uptodate(bh); - unlock_buffer(bh); - goto has_buffer; - } - } - -make_io: - /* - * There are other valid inodes in the buffer, this inode - * has in-inode xattrs, or we don't have this inode in memory. - * Read the block from disk. - */ - trace_ext3_load_inode(inode); - get_bh(bh); - bh->b_end_io = end_buffer_read_sync; - submit_bh(READ | REQ_META | REQ_PRIO, bh); - wait_on_buffer(bh); - if (!buffer_uptodate(bh)) { - ext3_error(inode->i_sb, "ext3_get_inode_loc", - "unable to read inode block - " - "inode=%lu, block="E3FSBLK, - inode->i_ino, block); - brelse(bh); - return -EIO; - } - } -has_buffer: - iloc->bh = bh; - return 0; -} - -int ext3_get_inode_loc(struct inode *inode, struct ext3_iloc *iloc) -{ - /* We have all inode data except xattrs in memory here. */ - return __ext3_get_inode_loc(inode, iloc, - !ext3_test_inode_state(inode, EXT3_STATE_XATTR)); -} - -void ext3_set_inode_flags(struct inode *inode) -{ - unsigned int flags = EXT3_I(inode)->i_flags; - - inode->i_flags &= ~(S_SYNC|S_APPEND|S_IMMUTABLE|S_NOATIME|S_DIRSYNC); - if (flags & EXT3_SYNC_FL) - inode->i_flags |= S_SYNC; - if (flags & EXT3_APPEND_FL) - inode->i_flags |= S_APPEND; - if (flags & EXT3_IMMUTABLE_FL) - inode->i_flags |= S_IMMUTABLE; - if (flags & EXT3_NOATIME_FL) - inode->i_flags |= S_NOATIME; - if (flags & EXT3_DIRSYNC_FL) - inode->i_flags |= S_DIRSYNC; -} - -/* Propagate flags from i_flags to EXT3_I(inode)->i_flags */ -void ext3_get_inode_flags(struct ext3_inode_info *ei) -{ - unsigned int flags = ei->vfs_inode.i_flags; - - ei->i_flags &= ~(EXT3_SYNC_FL|EXT3_APPEND_FL| - EXT3_IMMUTABLE_FL|EXT3_NOATIME_FL|EXT3_DIRSYNC_FL); - if (flags & S_SYNC) - ei->i_flags |= EXT3_SYNC_FL; - if (flags & S_APPEND) - ei->i_flags |= EXT3_APPEND_FL; - if (flags & S_IMMUTABLE) - ei->i_flags |= EXT3_IMMUTABLE_FL; - if (flags & S_NOATIME) - ei->i_flags |= EXT3_NOATIME_FL; - if (flags & S_DIRSYNC) - ei->i_flags |= EXT3_DIRSYNC_FL; -} - -struct inode *ext3_iget(struct super_block *sb, unsigned long ino) -{ - struct ext3_iloc iloc; - struct ext3_inode *raw_inode; - struct ext3_inode_info *ei; - struct buffer_head *bh; - struct inode *inode; - journal_t *journal = EXT3_SB(sb)->s_journal; - transaction_t *transaction; - long ret; - int block; - uid_t i_uid; - gid_t i_gid; - - inode = iget_locked(sb, ino); - if (!inode) - return ERR_PTR(-ENOMEM); - if (!(inode->i_state & I_NEW)) - return inode; - - ei = EXT3_I(inode); - ei->i_block_alloc_info = NULL; - - ret = __ext3_get_inode_loc(inode, &iloc, 0); - if (ret < 0) - goto bad_inode; - bh = iloc.bh; - raw_inode = ext3_raw_inode(&iloc); - inode->i_mode = le16_to_cpu(raw_inode->i_mode); - i_uid = (uid_t)le16_to_cpu(raw_inode->i_uid_low); - i_gid = (gid_t)le16_to_cpu(raw_inode->i_gid_low); - if(!(test_opt (inode->i_sb, NO_UID32))) { - i_uid |= le16_to_cpu(raw_inode->i_uid_high) << 16; - i_gid |= le16_to_cpu(raw_inode->i_gid_high) << 16; - } - i_uid_write(inode, i_uid); - i_gid_write(inode, i_gid); - set_nlink(inode, le16_to_cpu(raw_inode->i_links_count)); - inode->i_size = le32_to_cpu(raw_inode->i_size); - inode->i_atime.tv_sec = (signed)le32_to_cpu(raw_inode->i_atime); - inode->i_ctime.tv_sec = (signed)le32_to_cpu(raw_inode->i_ctime); - inode->i_mtime.tv_sec = (signed)le32_to_cpu(raw_inode->i_mtime); - inode->i_atime.tv_nsec = inode->i_ctime.tv_nsec = inode->i_mtime.tv_nsec = 0; - - ei->i_state_flags = 0; - ei->i_dir_start_lookup = 0; - ei->i_dtime = le32_to_cpu(raw_inode->i_dtime); - /* We now have enough fields to check if the inode was active or not. - * This is needed because nfsd might try to access dead inodes - * the test is that same one that e2fsck uses - * NeilBrown 1999oct15 - */ - if (inode->i_nlink == 0) { - if (inode->i_mode == 0 || - !(EXT3_SB(inode->i_sb)->s_mount_state & EXT3_ORPHAN_FS)) { - /* this inode is deleted */ - brelse (bh); - ret = -ESTALE; - goto bad_inode; - } - /* The only unlinked inodes we let through here have - * valid i_mode and are being read by the orphan - * recovery code: that's fine, we're about to complete - * the process of deleting those. */ - } - inode->i_blocks = le32_to_cpu(raw_inode->i_blocks); - ei->i_flags = le32_to_cpu(raw_inode->i_flags); -#ifdef EXT3_FRAGMENTS - ei->i_faddr = le32_to_cpu(raw_inode->i_faddr); - ei->i_frag_no = raw_inode->i_frag; - ei->i_frag_size = raw_inode->i_fsize; -#endif - ei->i_file_acl = le32_to_cpu(raw_inode->i_file_acl); - if (!S_ISREG(inode->i_mode)) { - ei->i_dir_acl = le32_to_cpu(raw_inode->i_dir_acl); - } else { - inode->i_size |= - ((__u64)le32_to_cpu(raw_inode->i_size_high)) << 32; - } - ei->i_disksize = inode->i_size; - inode->i_generation = le32_to_cpu(raw_inode->i_generation); - ei->i_block_group = iloc.block_group; - /* - * NOTE! The in-memory inode i_data array is in little-endian order - * even on big-endian machines: we do NOT byteswap the block numbers! - */ - for (block = 0; block < EXT3_N_BLOCKS; block++) - ei->i_data[block] = raw_inode->i_block[block]; - INIT_LIST_HEAD(&ei->i_orphan); - - /* - * Set transaction id's of transactions that have to be committed - * to finish f[data]sync. We set them to currently running transaction - * as we cannot be sure that the inode or some of its metadata isn't - * part of the transaction - the inode could have been reclaimed and - * now it is reread from disk. - */ - if (journal) { - tid_t tid; - - spin_lock(&journal->j_state_lock); - if (journal->j_running_transaction) - transaction = journal->j_running_transaction; - else - transaction = journal->j_committing_transaction; - if (transaction) - tid = transaction->t_tid; - else - tid = journal->j_commit_sequence; - spin_unlock(&journal->j_state_lock); - atomic_set(&ei->i_sync_tid, tid); - atomic_set(&ei->i_datasync_tid, tid); - } - - if (inode->i_ino >= EXT3_FIRST_INO(inode->i_sb) + 1 && - EXT3_INODE_SIZE(inode->i_sb) > EXT3_GOOD_OLD_INODE_SIZE) { - /* - * When mke2fs creates big inodes it does not zero out - * the unused bytes above EXT3_GOOD_OLD_INODE_SIZE, - * so ignore those first few inodes. - */ - ei->i_extra_isize = le16_to_cpu(raw_inode->i_extra_isize); - if (EXT3_GOOD_OLD_INODE_SIZE + ei->i_extra_isize > - EXT3_INODE_SIZE(inode->i_sb)) { - brelse (bh); - ret = -EIO; - goto bad_inode; - } - if (ei->i_extra_isize == 0) { - /* The extra space is currently unused. Use it. */ - ei->i_extra_isize = sizeof(struct ext3_inode) - - EXT3_GOOD_OLD_INODE_SIZE; - } else { - __le32 *magic = (void *)raw_inode + - EXT3_GOOD_OLD_INODE_SIZE + - ei->i_extra_isize; - if (*magic == cpu_to_le32(EXT3_XATTR_MAGIC)) - ext3_set_inode_state(inode, EXT3_STATE_XATTR); - } - } else - ei->i_extra_isize = 0; - - if (S_ISREG(inode->i_mode)) { - inode->i_op = &ext3_file_inode_operations; - inode->i_fop = &ext3_file_operations; - ext3_set_aops(inode); - } else if (S_ISDIR(inode->i_mode)) { - inode->i_op = &ext3_dir_inode_operations; - inode->i_fop = &ext3_dir_operations; - } else if (S_ISLNK(inode->i_mode)) { - if (ext3_inode_is_fast_symlink(inode)) { - inode->i_op = &ext3_fast_symlink_inode_operations; - nd_terminate_link(ei->i_data, inode->i_size, - sizeof(ei->i_data) - 1); - inode->i_link = (char *)ei->i_data; - } else { - inode->i_op = &ext3_symlink_inode_operations; - ext3_set_aops(inode); - } - } else { - inode->i_op = &ext3_special_inode_operations; - if (raw_inode->i_block[0]) - init_special_inode(inode, inode->i_mode, - old_decode_dev(le32_to_cpu(raw_inode->i_block[0]))); - else - init_special_inode(inode, inode->i_mode, - new_decode_dev(le32_to_cpu(raw_inode->i_block[1]))); - } - brelse (iloc.bh); - ext3_set_inode_flags(inode); - unlock_new_inode(inode); - return inode; - -bad_inode: - iget_failed(inode); - return ERR_PTR(ret); -} - -/* - * Post the struct inode info into an on-disk inode location in the - * buffer-cache. This gobbles the caller's reference to the - * buffer_head in the inode location struct. - * - * The caller must have write access to iloc->bh. - */ -static int ext3_do_update_inode(handle_t *handle, - struct inode *inode, - struct ext3_iloc *iloc) -{ - struct ext3_inode *raw_inode = ext3_raw_inode(iloc); - struct ext3_inode_info *ei = EXT3_I(inode); - struct buffer_head *bh = iloc->bh; - int err = 0, rc, block; - int need_datasync = 0; - __le32 disksize; - uid_t i_uid; - gid_t i_gid; - -again: - /* we can't allow multiple procs in here at once, its a bit racey */ - lock_buffer(bh); - - /* For fields not not tracking in the in-memory inode, - * initialise them to zero for new inodes. */ - if (ext3_test_inode_state(inode, EXT3_STATE_NEW)) - memset(raw_inode, 0, EXT3_SB(inode->i_sb)->s_inode_size); - - ext3_get_inode_flags(ei); - raw_inode->i_mode = cpu_to_le16(inode->i_mode); - i_uid = i_uid_read(inode); - i_gid = i_gid_read(inode); - if(!(test_opt(inode->i_sb, NO_UID32))) { - raw_inode->i_uid_low = cpu_to_le16(low_16_bits(i_uid)); - raw_inode->i_gid_low = cpu_to_le16(low_16_bits(i_gid)); -/* - * Fix up interoperability with old kernels. Otherwise, old inodes get - * re-used with the upper 16 bits of the uid/gid intact - */ - if(!ei->i_dtime) { - raw_inode->i_uid_high = - cpu_to_le16(high_16_bits(i_uid)); - raw_inode->i_gid_high = - cpu_to_le16(high_16_bits(i_gid)); - } else { - raw_inode->i_uid_high = 0; - raw_inode->i_gid_high = 0; - } - } else { - raw_inode->i_uid_low = - cpu_to_le16(fs_high2lowuid(i_uid)); - raw_inode->i_gid_low = - cpu_to_le16(fs_high2lowgid(i_gid)); - raw_inode->i_uid_high = 0; - raw_inode->i_gid_high = 0; - } - raw_inode->i_links_count = cpu_to_le16(inode->i_nlink); - disksize = cpu_to_le32(ei->i_disksize); - if (disksize != raw_inode->i_size) { - need_datasync = 1; - raw_inode->i_size = disksize; - } - raw_inode->i_atime = cpu_to_le32(inode->i_atime.tv_sec); - raw_inode->i_ctime = cpu_to_le32(inode->i_ctime.tv_sec); - raw_inode->i_mtime = cpu_to_le32(inode->i_mtime.tv_sec); - raw_inode->i_blocks = cpu_to_le32(inode->i_blocks); - raw_inode->i_dtime = cpu_to_le32(ei->i_dtime); - raw_inode->i_flags = cpu_to_le32(ei->i_flags); -#ifdef EXT3_FRAGMENTS - raw_inode->i_faddr = cpu_to_le32(ei->i_faddr); - raw_inode->i_frag = ei->i_frag_no; - raw_inode->i_fsize = ei->i_frag_size; -#endif - raw_inode->i_file_acl = cpu_to_le32(ei->i_file_acl); - if (!S_ISREG(inode->i_mode)) { - raw_inode->i_dir_acl = cpu_to_le32(ei->i_dir_acl); - } else { - disksize = cpu_to_le32(ei->i_disksize >> 32); - if (disksize != raw_inode->i_size_high) { - raw_inode->i_size_high = disksize; - need_datasync = 1; - } - if (ei->i_disksize > 0x7fffffffULL) { - struct super_block *sb = inode->i_sb; - if (!EXT3_HAS_RO_COMPAT_FEATURE(sb, - EXT3_FEATURE_RO_COMPAT_LARGE_FILE) || - EXT3_SB(sb)->s_es->s_rev_level == - cpu_to_le32(EXT3_GOOD_OLD_REV)) { - /* If this is the first large file - * created, add a flag to the superblock. - */ - unlock_buffer(bh); - err = ext3_journal_get_write_access(handle, - EXT3_SB(sb)->s_sbh); - if (err) - goto out_brelse; - - ext3_update_dynamic_rev(sb); - EXT3_SET_RO_COMPAT_FEATURE(sb, - EXT3_FEATURE_RO_COMPAT_LARGE_FILE); - handle->h_sync = 1; - err = ext3_journal_dirty_metadata(handle, - EXT3_SB(sb)->s_sbh); - /* get our lock and start over */ - goto again; - } - } - } - raw_inode->i_generation = cpu_to_le32(inode->i_generation); - if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode)) { - if (old_valid_dev(inode->i_rdev)) { - raw_inode->i_block[0] = - cpu_to_le32(old_encode_dev(inode->i_rdev)); - raw_inode->i_block[1] = 0; - } else { - raw_inode->i_block[0] = 0; - raw_inode->i_block[1] = - cpu_to_le32(new_encode_dev(inode->i_rdev)); - raw_inode->i_block[2] = 0; - } - } else for (block = 0; block < EXT3_N_BLOCKS; block++) - raw_inode->i_block[block] = ei->i_data[block]; - - if (ei->i_extra_isize) - raw_inode->i_extra_isize = cpu_to_le16(ei->i_extra_isize); - - BUFFER_TRACE(bh, "call ext3_journal_dirty_metadata"); - unlock_buffer(bh); - rc = ext3_journal_dirty_metadata(handle, bh); - if (!err) - err = rc; - ext3_clear_inode_state(inode, EXT3_STATE_NEW); - - atomic_set(&ei->i_sync_tid, handle->h_transaction->t_tid); - if (need_datasync) - atomic_set(&ei->i_datasync_tid, handle->h_transaction->t_tid); -out_brelse: - brelse (bh); - ext3_std_error(inode->i_sb, err); - return err; -} - -/* - * ext3_write_inode() - * - * We are called from a few places: - * - * - Within generic_file_aio_write() -> generic_write_sync() for O_SYNC files. - * Here, there will be no transaction running. We wait for any running - * transaction to commit. - * - * - Within flush work (for sys_sync(), kupdate and such). - * We wait on commit, if told to. - * - * - Within iput_final() -> write_inode_now() - * We wait on commit, if told to. - * - * In all cases it is actually safe for us to return without doing anything, - * because the inode has been copied into a raw inode buffer in - * ext3_mark_inode_dirty(). This is a correctness thing for WB_SYNC_ALL - * writeback. - * - * Note that we are absolutely dependent upon all inode dirtiers doing the - * right thing: they *must* call mark_inode_dirty() after dirtying info in - * which we are interested. - * - * It would be a bug for them to not do this. The code: - * - * mark_inode_dirty(inode) - * stuff(); - * inode->i_size = expr; - * - * is in error because write_inode() could occur while `stuff()' is running, - * and the new i_size will be lost. Plus the inode will no longer be on the - * superblock's dirty inode list. - */ -int ext3_write_inode(struct inode *inode, struct writeback_control *wbc) -{ - if (WARN_ON_ONCE(current->flags & PF_MEMALLOC)) - return 0; - - if (ext3_journal_current_handle()) { - jbd_debug(1, "called recursively, non-PF_MEMALLOC!\n"); - dump_stack(); - return -EIO; - } - - /* - * No need to force transaction in WB_SYNC_NONE mode. Also - * ext3_sync_fs() will force the commit after everything is - * written. - */ - if (wbc->sync_mode != WB_SYNC_ALL || wbc->for_sync) - return 0; - - return ext3_force_commit(inode->i_sb); -} - -/* - * ext3_setattr() - * - * Called from notify_change. - * - * We want to trap VFS attempts to truncate the file as soon as - * possible. In particular, we want to make sure that when the VFS - * shrinks i_size, we put the inode on the orphan list and modify - * i_disksize immediately, so that during the subsequent flushing of - * dirty pages and freeing of disk blocks, we can guarantee that any - * commit will leave the blocks being flushed in an unused state on - * disk. (On recovery, the inode will get truncated and the blocks will - * be freed, so we have a strong guarantee that no future commit will - * leave these blocks visible to the user.) - * - * Called with inode->sem down. - */ -int ext3_setattr(struct dentry *dentry, struct iattr *attr) -{ - struct inode *inode = d_inode(dentry); - int error, rc = 0; - const unsigned int ia_valid = attr->ia_valid; - - error = inode_change_ok(inode, attr); - if (error) - return error; - - if (is_quota_modification(inode, attr)) - dquot_initialize(inode); - if ((ia_valid & ATTR_UID && !uid_eq(attr->ia_uid, inode->i_uid)) || - (ia_valid & ATTR_GID && !gid_eq(attr->ia_gid, inode->i_gid))) { - handle_t *handle; - - /* (user+group)*(old+new) structure, inode write (sb, - * inode block, ? - but truncate inode update has it) */ - handle = ext3_journal_start(inode, EXT3_MAXQUOTAS_INIT_BLOCKS(inode->i_sb)+ - EXT3_MAXQUOTAS_DEL_BLOCKS(inode->i_sb)+3); - if (IS_ERR(handle)) { - error = PTR_ERR(handle); - goto err_out; - } - error = dquot_transfer(inode, attr); - if (error) { - ext3_journal_stop(handle); - return error; - } - /* Update corresponding info in inode so that everything is in - * one transaction */ - if (attr->ia_valid & ATTR_UID) - inode->i_uid = attr->ia_uid; - if (attr->ia_valid & ATTR_GID) - inode->i_gid = attr->ia_gid; - error = ext3_mark_inode_dirty(handle, inode); - ext3_journal_stop(handle); - } - - if (attr->ia_valid & ATTR_SIZE) - inode_dio_wait(inode); - - if (S_ISREG(inode->i_mode) && - attr->ia_valid & ATTR_SIZE && attr->ia_size < inode->i_size) { - handle_t *handle; - - handle = ext3_journal_start(inode, 3); - if (IS_ERR(handle)) { - error = PTR_ERR(handle); - goto err_out; - } - - error = ext3_orphan_add(handle, inode); - if (error) { - ext3_journal_stop(handle); - goto err_out; - } - EXT3_I(inode)->i_disksize = attr->ia_size; - error = ext3_mark_inode_dirty(handle, inode); - ext3_journal_stop(handle); - if (error) { - /* Some hard fs error must have happened. Bail out. */ - ext3_orphan_del(NULL, inode); - goto err_out; - } - rc = ext3_block_truncate_page(inode, attr->ia_size); - if (rc) { - /* Cleanup orphan list and exit */ - handle = ext3_journal_start(inode, 3); - if (IS_ERR(handle)) { - ext3_orphan_del(NULL, inode); - goto err_out; - } - ext3_orphan_del(handle, inode); - ext3_journal_stop(handle); - goto err_out; - } - } - - if ((attr->ia_valid & ATTR_SIZE) && - attr->ia_size != i_size_read(inode)) { - truncate_setsize(inode, attr->ia_size); - ext3_truncate(inode); - } - - setattr_copy(inode, attr); - mark_inode_dirty(inode); - - if (ia_valid & ATTR_MODE) - rc = posix_acl_chmod(inode, inode->i_mode); - -err_out: - ext3_std_error(inode->i_sb, error); - if (!error) - error = rc; - return error; -} - - -/* - * How many blocks doth make a writepage()? - * - * With N blocks per page, it may be: - * N data blocks - * 2 indirect block - * 2 dindirect - * 1 tindirect - * N+5 bitmap blocks (from the above) - * N+5 group descriptor summary blocks - * 1 inode block - * 1 superblock. - * 2 * EXT3_SINGLEDATA_TRANS_BLOCKS for the quote files - * - * 3 * (N + 5) + 2 + 2 * EXT3_SINGLEDATA_TRANS_BLOCKS - * - * With ordered or writeback data it's the same, less the N data blocks. - * - * If the inode's direct blocks can hold an integral number of pages then a - * page cannot straddle two indirect blocks, and we can only touch one indirect - * and dindirect block, and the "5" above becomes "3". - * - * This still overestimates under most circumstances. If we were to pass the - * start and end offsets in here as well we could do block_to_path() on each - * block and work out the exact number of indirects which are touched. Pah. - */ - -static int ext3_writepage_trans_blocks(struct inode *inode) -{ - int bpp = ext3_journal_blocks_per_page(inode); - int indirects = (EXT3_NDIR_BLOCKS % bpp) ? 5 : 3; - int ret; - - if (ext3_should_journal_data(inode)) - ret = 3 * (bpp + indirects) + 2; - else - ret = 2 * (bpp + indirects) + indirects + 2; - -#ifdef CONFIG_QUOTA - /* We know that structure was already allocated during dquot_initialize so - * we will be updating only the data blocks + inodes */ - ret += EXT3_MAXQUOTAS_TRANS_BLOCKS(inode->i_sb); -#endif - - return ret; -} - -/* - * The caller must have previously called ext3_reserve_inode_write(). - * Give this, we know that the caller already has write access to iloc->bh. - */ -int ext3_mark_iloc_dirty(handle_t *handle, - struct inode *inode, struct ext3_iloc *iloc) -{ - int err = 0; - - /* the do_update_inode consumes one bh->b_count */ - get_bh(iloc->bh); - - /* ext3_do_update_inode() does journal_dirty_metadata */ - err = ext3_do_update_inode(handle, inode, iloc); - put_bh(iloc->bh); - return err; -} - -/* - * On success, We end up with an outstanding reference count against - * iloc->bh. This _must_ be cleaned up later. - */ - -int -ext3_reserve_inode_write(handle_t *handle, struct inode *inode, - struct ext3_iloc *iloc) -{ - int err = 0; - if (handle) { - err = ext3_get_inode_loc(inode, iloc); - if (!err) { - BUFFER_TRACE(iloc->bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, iloc->bh); - if (err) { - brelse(iloc->bh); - iloc->bh = NULL; - } - } - } - ext3_std_error(inode->i_sb, err); - return err; -} - -/* - * What we do here is to mark the in-core inode as clean with respect to inode - * dirtiness (it may still be data-dirty). - * This means that the in-core inode may be reaped by prune_icache - * without having to perform any I/O. This is a very good thing, - * because *any* task may call prune_icache - even ones which - * have a transaction open against a different journal. - * - * Is this cheating? Not really. Sure, we haven't written the - * inode out, but prune_icache isn't a user-visible syncing function. - * Whenever the user wants stuff synced (sys_sync, sys_msync, sys_fsync) - * we start and wait on commits. - */ -int ext3_mark_inode_dirty(handle_t *handle, struct inode *inode) -{ - struct ext3_iloc iloc; - int err; - - might_sleep(); - trace_ext3_mark_inode_dirty(inode, _RET_IP_); - err = ext3_reserve_inode_write(handle, inode, &iloc); - if (!err) - err = ext3_mark_iloc_dirty(handle, inode, &iloc); - return err; -} - -/* - * ext3_dirty_inode() is called from __mark_inode_dirty() - * - * We're really interested in the case where a file is being extended. - * i_size has been changed by generic_commit_write() and we thus need - * to include the updated inode in the current transaction. - * - * Also, dquot_alloc_space() will always dirty the inode when blocks - * are allocated to the file. - * - * If the inode is marked synchronous, we don't honour that here - doing - * so would cause a commit on atime updates, which we don't bother doing. - * We handle synchronous inodes at the highest possible level. - */ -void ext3_dirty_inode(struct inode *inode, int flags) -{ - handle_t *current_handle = ext3_journal_current_handle(); - handle_t *handle; - - handle = ext3_journal_start(inode, 2); - if (IS_ERR(handle)) - goto out; - if (current_handle && - current_handle->h_transaction != handle->h_transaction) { - /* This task has a transaction open against a different fs */ - printk(KERN_EMERG "%s: transactions do not match!\n", - __func__); - } else { - jbd_debug(5, "marking dirty. outer handle=%p\n", - current_handle); - ext3_mark_inode_dirty(handle, inode); - } - ext3_journal_stop(handle); -out: - return; -} - -#if 0 -/* - * Bind an inode's backing buffer_head into this transaction, to prevent - * it from being flushed to disk early. Unlike - * ext3_reserve_inode_write, this leaves behind no bh reference and - * returns no iloc structure, so the caller needs to repeat the iloc - * lookup to mark the inode dirty later. - */ -static int ext3_pin_inode(handle_t *handle, struct inode *inode) -{ - struct ext3_iloc iloc; - - int err = 0; - if (handle) { - err = ext3_get_inode_loc(inode, &iloc); - if (!err) { - BUFFER_TRACE(iloc.bh, "get_write_access"); - err = journal_get_write_access(handle, iloc.bh); - if (!err) - err = ext3_journal_dirty_metadata(handle, - iloc.bh); - brelse(iloc.bh); - } - } - ext3_std_error(inode->i_sb, err); - return err; -} -#endif - -int ext3_change_inode_journal_flag(struct inode *inode, int val) -{ - journal_t *journal; - handle_t *handle; - int err; - - /* - * We have to be very careful here: changing a data block's - * journaling status dynamically is dangerous. If we write a - * data block to the journal, change the status and then delete - * that block, we risk forgetting to revoke the old log record - * from the journal and so a subsequent replay can corrupt data. - * So, first we make sure that the journal is empty and that - * nobody is changing anything. - */ - - journal = EXT3_JOURNAL(inode); - if (is_journal_aborted(journal)) - return -EROFS; - - journal_lock_updates(journal); - journal_flush(journal); - - /* - * OK, there are no updates running now, and all cached data is - * synced to disk. We are now in a completely consistent state - * which doesn't have anything in the journal, and we know that - * no filesystem updates are running, so it is safe to modify - * the inode's in-core data-journaling state flag now. - */ - - if (val) - EXT3_I(inode)->i_flags |= EXT3_JOURNAL_DATA_FL; - else - EXT3_I(inode)->i_flags &= ~EXT3_JOURNAL_DATA_FL; - ext3_set_aops(inode); - - journal_unlock_updates(journal); - - /* Finally we can mark the inode as dirty. */ - - handle = ext3_journal_start(inode, 1); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - err = ext3_mark_inode_dirty(handle, inode); - handle->h_sync = 1; - ext3_journal_stop(handle); - ext3_std_error(inode->i_sb, err); - - return err; -} diff --git a/fs/ext3/ioctl.c b/fs/ext3/ioctl.c deleted file mode 100644 index 4d96e9a6453268..00000000000000 --- a/fs/ext3/ioctl.c +++ /dev/null @@ -1,327 +0,0 @@ -/* - * linux/fs/ext3/ioctl.c - * - * Copyright (C) 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - */ - -#include -#include -#include -#include "ext3.h" - -long ext3_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) -{ - struct inode *inode = file_inode(filp); - struct ext3_inode_info *ei = EXT3_I(inode); - unsigned int flags; - unsigned short rsv_window_size; - - ext3_debug ("cmd = %u, arg = %lu\n", cmd, arg); - - switch (cmd) { - case EXT3_IOC_GETFLAGS: - ext3_get_inode_flags(ei); - flags = ei->i_flags & EXT3_FL_USER_VISIBLE; - return put_user(flags, (int __user *) arg); - case EXT3_IOC_SETFLAGS: { - handle_t *handle = NULL; - int err; - struct ext3_iloc iloc; - unsigned int oldflags; - unsigned int jflag; - - if (!inode_owner_or_capable(inode)) - return -EACCES; - - if (get_user(flags, (int __user *) arg)) - return -EFAULT; - - err = mnt_want_write_file(filp); - if (err) - return err; - - flags = ext3_mask_flags(inode->i_mode, flags); - - mutex_lock(&inode->i_mutex); - - /* Is it quota file? Do not allow user to mess with it */ - err = -EPERM; - if (IS_NOQUOTA(inode)) - goto flags_out; - - oldflags = ei->i_flags; - - /* The JOURNAL_DATA flag is modifiable only by root */ - jflag = flags & EXT3_JOURNAL_DATA_FL; - - /* - * The IMMUTABLE and APPEND_ONLY flags can only be changed by - * the relevant capability. - * - * This test looks nicer. Thanks to Pauline Middelink - */ - if ((flags ^ oldflags) & (EXT3_APPEND_FL | EXT3_IMMUTABLE_FL)) { - if (!capable(CAP_LINUX_IMMUTABLE)) - goto flags_out; - } - - /* - * The JOURNAL_DATA flag can only be changed by - * the relevant capability. - */ - if ((jflag ^ oldflags) & (EXT3_JOURNAL_DATA_FL)) { - if (!capable(CAP_SYS_RESOURCE)) - goto flags_out; - } - - handle = ext3_journal_start(inode, 1); - if (IS_ERR(handle)) { - err = PTR_ERR(handle); - goto flags_out; - } - if (IS_SYNC(inode)) - handle->h_sync = 1; - err = ext3_reserve_inode_write(handle, inode, &iloc); - if (err) - goto flags_err; - - flags = flags & EXT3_FL_USER_MODIFIABLE; - flags |= oldflags & ~EXT3_FL_USER_MODIFIABLE; - ei->i_flags = flags; - - ext3_set_inode_flags(inode); - inode->i_ctime = CURRENT_TIME_SEC; - - err = ext3_mark_iloc_dirty(handle, inode, &iloc); -flags_err: - ext3_journal_stop(handle); - if (err) - goto flags_out; - - if ((jflag ^ oldflags) & (EXT3_JOURNAL_DATA_FL)) - err = ext3_change_inode_journal_flag(inode, jflag); -flags_out: - mutex_unlock(&inode->i_mutex); - mnt_drop_write_file(filp); - return err; - } - case EXT3_IOC_GETVERSION: - case EXT3_IOC_GETVERSION_OLD: - return put_user(inode->i_generation, (int __user *) arg); - case EXT3_IOC_SETVERSION: - case EXT3_IOC_SETVERSION_OLD: { - handle_t *handle; - struct ext3_iloc iloc; - __u32 generation; - int err; - - if (!inode_owner_or_capable(inode)) - return -EPERM; - - err = mnt_want_write_file(filp); - if (err) - return err; - if (get_user(generation, (int __user *) arg)) { - err = -EFAULT; - goto setversion_out; - } - - mutex_lock(&inode->i_mutex); - handle = ext3_journal_start(inode, 1); - if (IS_ERR(handle)) { - err = PTR_ERR(handle); - goto unlock_out; - } - err = ext3_reserve_inode_write(handle, inode, &iloc); - if (err == 0) { - inode->i_ctime = CURRENT_TIME_SEC; - inode->i_generation = generation; - err = ext3_mark_iloc_dirty(handle, inode, &iloc); - } - ext3_journal_stop(handle); - -unlock_out: - mutex_unlock(&inode->i_mutex); -setversion_out: - mnt_drop_write_file(filp); - return err; - } - case EXT3_IOC_GETRSVSZ: - if (test_opt(inode->i_sb, RESERVATION) - && S_ISREG(inode->i_mode) - && ei->i_block_alloc_info) { - rsv_window_size = ei->i_block_alloc_info->rsv_window_node.rsv_goal_size; - return put_user(rsv_window_size, (int __user *)arg); - } - return -ENOTTY; - case EXT3_IOC_SETRSVSZ: { - int err; - - if (!test_opt(inode->i_sb, RESERVATION) ||!S_ISREG(inode->i_mode)) - return -ENOTTY; - - err = mnt_want_write_file(filp); - if (err) - return err; - - if (!inode_owner_or_capable(inode)) { - err = -EACCES; - goto setrsvsz_out; - } - - if (get_user(rsv_window_size, (int __user *)arg)) { - err = -EFAULT; - goto setrsvsz_out; - } - - if (rsv_window_size > EXT3_MAX_RESERVE_BLOCKS) - rsv_window_size = EXT3_MAX_RESERVE_BLOCKS; - - /* - * need to allocate reservation structure for this inode - * before set the window size - */ - mutex_lock(&ei->truncate_mutex); - if (!ei->i_block_alloc_info) - ext3_init_block_alloc_info(inode); - - if (ei->i_block_alloc_info){ - struct ext3_reserve_window_node *rsv = &ei->i_block_alloc_info->rsv_window_node; - rsv->rsv_goal_size = rsv_window_size; - } - mutex_unlock(&ei->truncate_mutex); -setrsvsz_out: - mnt_drop_write_file(filp); - return err; - } - case EXT3_IOC_GROUP_EXTEND: { - ext3_fsblk_t n_blocks_count; - struct super_block *sb = inode->i_sb; - int err, err2; - - if (!capable(CAP_SYS_RESOURCE)) - return -EPERM; - - err = mnt_want_write_file(filp); - if (err) - return err; - - if (get_user(n_blocks_count, (__u32 __user *)arg)) { - err = -EFAULT; - goto group_extend_out; - } - err = ext3_group_extend(sb, EXT3_SB(sb)->s_es, n_blocks_count); - journal_lock_updates(EXT3_SB(sb)->s_journal); - err2 = journal_flush(EXT3_SB(sb)->s_journal); - journal_unlock_updates(EXT3_SB(sb)->s_journal); - if (err == 0) - err = err2; -group_extend_out: - mnt_drop_write_file(filp); - return err; - } - case EXT3_IOC_GROUP_ADD: { - struct ext3_new_group_data input; - struct super_block *sb = inode->i_sb; - int err, err2; - - if (!capable(CAP_SYS_RESOURCE)) - return -EPERM; - - err = mnt_want_write_file(filp); - if (err) - return err; - - if (copy_from_user(&input, (struct ext3_new_group_input __user *)arg, - sizeof(input))) { - err = -EFAULT; - goto group_add_out; - } - - err = ext3_group_add(sb, &input); - journal_lock_updates(EXT3_SB(sb)->s_journal); - err2 = journal_flush(EXT3_SB(sb)->s_journal); - journal_unlock_updates(EXT3_SB(sb)->s_journal); - if (err == 0) - err = err2; -group_add_out: - mnt_drop_write_file(filp); - return err; - } - case FITRIM: { - - struct super_block *sb = inode->i_sb; - struct fstrim_range range; - int ret = 0; - - if (!capable(CAP_SYS_ADMIN)) - return -EPERM; - - if (copy_from_user(&range, (struct fstrim_range __user *)arg, - sizeof(range))) - return -EFAULT; - - ret = ext3_trim_fs(sb, &range); - if (ret < 0) - return ret; - - if (copy_to_user((struct fstrim_range __user *)arg, &range, - sizeof(range))) - return -EFAULT; - - return 0; - } - - default: - return -ENOTTY; - } -} - -#ifdef CONFIG_COMPAT -long ext3_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg) -{ - /* These are just misnamed, they actually get/put from/to user an int */ - switch (cmd) { - case EXT3_IOC32_GETFLAGS: - cmd = EXT3_IOC_GETFLAGS; - break; - case EXT3_IOC32_SETFLAGS: - cmd = EXT3_IOC_SETFLAGS; - break; - case EXT3_IOC32_GETVERSION: - cmd = EXT3_IOC_GETVERSION; - break; - case EXT3_IOC32_SETVERSION: - cmd = EXT3_IOC_SETVERSION; - break; - case EXT3_IOC32_GROUP_EXTEND: - cmd = EXT3_IOC_GROUP_EXTEND; - break; - case EXT3_IOC32_GETVERSION_OLD: - cmd = EXT3_IOC_GETVERSION_OLD; - break; - case EXT3_IOC32_SETVERSION_OLD: - cmd = EXT3_IOC_SETVERSION_OLD; - break; -#ifdef CONFIG_JBD_DEBUG - case EXT3_IOC32_WAIT_FOR_READONLY: - cmd = EXT3_IOC_WAIT_FOR_READONLY; - break; -#endif - case EXT3_IOC32_GETRSVSZ: - cmd = EXT3_IOC_GETRSVSZ; - break; - case EXT3_IOC32_SETRSVSZ: - cmd = EXT3_IOC_SETRSVSZ; - break; - case EXT3_IOC_GROUP_ADD: - break; - default: - return -ENOIOCTLCMD; - } - return ext3_ioctl(file, cmd, (unsigned long) compat_ptr(arg)); -} -#endif diff --git a/fs/ext3/namei.c b/fs/ext3/namei.c deleted file mode 100644 index c9e767cd4b6799..00000000000000 --- a/fs/ext3/namei.c +++ /dev/null @@ -1,2586 +0,0 @@ -/* - * linux/fs/ext3/namei.c - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * - * from - * - * linux/fs/minix/namei.c - * - * Copyright (C) 1991, 1992 Linus Torvalds - * - * Big-endian to little-endian byte-swapping/bitmaps by - * David S. Miller (davem@caip.rutgers.edu), 1995 - * Directory entry file type support and forward compatibility hooks - * for B-tree directories by Theodore Ts'o (tytso@mit.edu), 1998 - * Hash Tree Directory indexing (c) - * Daniel Phillips, 2001 - * Hash Tree Directory indexing porting - * Christopher Li, 2002 - * Hash Tree Directory indexing cleanup - * Theodore Ts'o, 2002 - */ - -#include -#include "ext3.h" -#include "namei.h" -#include "xattr.h" -#include "acl.h" - -/* - * define how far ahead to read directories while searching them. - */ -#define NAMEI_RA_CHUNKS 2 -#define NAMEI_RA_BLOCKS 4 -#define NAMEI_RA_SIZE (NAMEI_RA_CHUNKS * NAMEI_RA_BLOCKS) - -static struct buffer_head *ext3_append(handle_t *handle, - struct inode *inode, - u32 *block, int *err) -{ - struct buffer_head *bh; - - *block = inode->i_size >> inode->i_sb->s_blocksize_bits; - - if ((bh = ext3_dir_bread(handle, inode, *block, 1, err))) { - inode->i_size += inode->i_sb->s_blocksize; - EXT3_I(inode)->i_disksize = inode->i_size; - *err = ext3_journal_get_write_access(handle, bh); - if (*err) { - brelse(bh); - bh = NULL; - } - } - return bh; -} - -#ifndef assert -#define assert(test) J_ASSERT(test) -#endif - -#ifdef DX_DEBUG -#define dxtrace(command) command -#else -#define dxtrace(command) -#endif - -struct fake_dirent -{ - __le32 inode; - __le16 rec_len; - u8 name_len; - u8 file_type; -}; - -struct dx_countlimit -{ - __le16 limit; - __le16 count; -}; - -struct dx_entry -{ - __le32 hash; - __le32 block; -}; - -/* - * dx_root_info is laid out so that if it should somehow get overlaid by a - * dirent the two low bits of the hash version will be zero. Therefore, the - * hash version mod 4 should never be 0. Sincerely, the paranoia department. - */ - -struct dx_root -{ - struct fake_dirent dot; - char dot_name[4]; - struct fake_dirent dotdot; - char dotdot_name[4]; - struct dx_root_info - { - __le32 reserved_zero; - u8 hash_version; - u8 info_length; /* 8 */ - u8 indirect_levels; - u8 unused_flags; - } - info; - struct dx_entry entries[0]; -}; - -struct dx_node -{ - struct fake_dirent fake; - struct dx_entry entries[0]; -}; - - -struct dx_frame -{ - struct buffer_head *bh; - struct dx_entry *entries; - struct dx_entry *at; -}; - -struct dx_map_entry -{ - u32 hash; - u16 offs; - u16 size; -}; - -static inline unsigned dx_get_block (struct dx_entry *entry); -static void dx_set_block (struct dx_entry *entry, unsigned value); -static inline unsigned dx_get_hash (struct dx_entry *entry); -static void dx_set_hash (struct dx_entry *entry, unsigned value); -static unsigned dx_get_count (struct dx_entry *entries); -static unsigned dx_get_limit (struct dx_entry *entries); -static void dx_set_count (struct dx_entry *entries, unsigned value); -static void dx_set_limit (struct dx_entry *entries, unsigned value); -static unsigned dx_root_limit (struct inode *dir, unsigned infosize); -static unsigned dx_node_limit (struct inode *dir); -static struct dx_frame *dx_probe(struct qstr *entry, - struct inode *dir, - struct dx_hash_info *hinfo, - struct dx_frame *frame, - int *err); -static void dx_release (struct dx_frame *frames); -static int dx_make_map(struct ext3_dir_entry_2 *de, unsigned blocksize, - struct dx_hash_info *hinfo, struct dx_map_entry map[]); -static void dx_sort_map(struct dx_map_entry *map, unsigned count); -static struct ext3_dir_entry_2 *dx_move_dirents (char *from, char *to, - struct dx_map_entry *offsets, int count); -static struct ext3_dir_entry_2 *dx_pack_dirents(char *base, unsigned blocksize); -static void dx_insert_block (struct dx_frame *frame, u32 hash, u32 block); -static int ext3_htree_next_block(struct inode *dir, __u32 hash, - struct dx_frame *frame, - struct dx_frame *frames, - __u32 *start_hash); -static struct buffer_head * ext3_dx_find_entry(struct inode *dir, - struct qstr *entry, struct ext3_dir_entry_2 **res_dir, - int *err); -static int ext3_dx_add_entry(handle_t *handle, struct dentry *dentry, - struct inode *inode); - -/* - * p is at least 6 bytes before the end of page - */ -static inline struct ext3_dir_entry_2 * -ext3_next_entry(struct ext3_dir_entry_2 *p) -{ - return (struct ext3_dir_entry_2 *)((char *)p + - ext3_rec_len_from_disk(p->rec_len)); -} - -/* - * Future: use high four bits of block for coalesce-on-delete flags - * Mask them off for now. - */ - -static inline unsigned dx_get_block (struct dx_entry *entry) -{ - return le32_to_cpu(entry->block) & 0x00ffffff; -} - -static inline void dx_set_block (struct dx_entry *entry, unsigned value) -{ - entry->block = cpu_to_le32(value); -} - -static inline unsigned dx_get_hash (struct dx_entry *entry) -{ - return le32_to_cpu(entry->hash); -} - -static inline void dx_set_hash (struct dx_entry *entry, unsigned value) -{ - entry->hash = cpu_to_le32(value); -} - -static inline unsigned dx_get_count (struct dx_entry *entries) -{ - return le16_to_cpu(((struct dx_countlimit *) entries)->count); -} - -static inline unsigned dx_get_limit (struct dx_entry *entries) -{ - return le16_to_cpu(((struct dx_countlimit *) entries)->limit); -} - -static inline void dx_set_count (struct dx_entry *entries, unsigned value) -{ - ((struct dx_countlimit *) entries)->count = cpu_to_le16(value); -} - -static inline void dx_set_limit (struct dx_entry *entries, unsigned value) -{ - ((struct dx_countlimit *) entries)->limit = cpu_to_le16(value); -} - -static inline unsigned dx_root_limit (struct inode *dir, unsigned infosize) -{ - unsigned entry_space = dir->i_sb->s_blocksize - EXT3_DIR_REC_LEN(1) - - EXT3_DIR_REC_LEN(2) - infosize; - return entry_space / sizeof(struct dx_entry); -} - -static inline unsigned dx_node_limit (struct inode *dir) -{ - unsigned entry_space = dir->i_sb->s_blocksize - EXT3_DIR_REC_LEN(0); - return entry_space / sizeof(struct dx_entry); -} - -/* - * Debug - */ -#ifdef DX_DEBUG -static void dx_show_index (char * label, struct dx_entry *entries) -{ - int i, n = dx_get_count (entries); - printk("%s index ", label); - for (i = 0; i < n; i++) - { - printk("%x->%u ", i? dx_get_hash(entries + i): 0, dx_get_block(entries + i)); - } - printk("\n"); -} - -struct stats -{ - unsigned names; - unsigned space; - unsigned bcount; -}; - -static struct stats dx_show_leaf(struct dx_hash_info *hinfo, struct ext3_dir_entry_2 *de, - int size, int show_names) -{ - unsigned names = 0, space = 0; - char *base = (char *) de; - struct dx_hash_info h = *hinfo; - - printk("names: "); - while ((char *) de < base + size) - { - if (de->inode) - { - if (show_names) - { - int len = de->name_len; - char *name = de->name; - while (len--) printk("%c", *name++); - ext3fs_dirhash(de->name, de->name_len, &h); - printk(":%x.%u ", h.hash, - (unsigned) ((char *) de - base)); - } - space += EXT3_DIR_REC_LEN(de->name_len); - names++; - } - de = ext3_next_entry(de); - } - printk("(%i)\n", names); - return (struct stats) { names, space, 1 }; -} - -struct stats dx_show_entries(struct dx_hash_info *hinfo, struct inode *dir, - struct dx_entry *entries, int levels) -{ - unsigned blocksize = dir->i_sb->s_blocksize; - unsigned count = dx_get_count (entries), names = 0, space = 0, i; - unsigned bcount = 0; - struct buffer_head *bh; - int err; - printk("%i indexed blocks...\n", count); - for (i = 0; i < count; i++, entries++) - { - u32 block = dx_get_block(entries), hash = i? dx_get_hash(entries): 0; - u32 range = i < count - 1? (dx_get_hash(entries + 1) - hash): ~hash; - struct stats stats; - printk("%s%3u:%03u hash %8x/%8x ",levels?"":" ", i, block, hash, range); - if (!(bh = ext3_bread (NULL,dir, block, 0,&err))) continue; - stats = levels? - dx_show_entries(hinfo, dir, ((struct dx_node *) bh->b_data)->entries, levels - 1): - dx_show_leaf(hinfo, (struct ext3_dir_entry_2 *) bh->b_data, blocksize, 0); - names += stats.names; - space += stats.space; - bcount += stats.bcount; - brelse (bh); - } - if (bcount) - printk("%snames %u, fullness %u (%u%%)\n", levels?"":" ", - names, space/bcount,(space/bcount)*100/blocksize); - return (struct stats) { names, space, bcount}; -} -#endif /* DX_DEBUG */ - -/* - * Probe for a directory leaf block to search. - * - * dx_probe can return ERR_BAD_DX_DIR, which means there was a format - * error in the directory index, and the caller should fall back to - * searching the directory normally. The callers of dx_probe **MUST** - * check for this error code, and make sure it never gets reflected - * back to userspace. - */ -static struct dx_frame * -dx_probe(struct qstr *entry, struct inode *dir, - struct dx_hash_info *hinfo, struct dx_frame *frame_in, int *err) -{ - unsigned count, indirect; - struct dx_entry *at, *entries, *p, *q, *m; - struct dx_root *root; - struct buffer_head *bh; - struct dx_frame *frame = frame_in; - u32 hash; - - frame->bh = NULL; - if (!(bh = ext3_dir_bread(NULL, dir, 0, 0, err))) { - *err = ERR_BAD_DX_DIR; - goto fail; - } - root = (struct dx_root *) bh->b_data; - if (root->info.hash_version != DX_HASH_TEA && - root->info.hash_version != DX_HASH_HALF_MD4 && - root->info.hash_version != DX_HASH_LEGACY) { - ext3_warning(dir->i_sb, __func__, - "Unrecognised inode hash code %d", - root->info.hash_version); - brelse(bh); - *err = ERR_BAD_DX_DIR; - goto fail; - } - hinfo->hash_version = root->info.hash_version; - if (hinfo->hash_version <= DX_HASH_TEA) - hinfo->hash_version += EXT3_SB(dir->i_sb)->s_hash_unsigned; - hinfo->seed = EXT3_SB(dir->i_sb)->s_hash_seed; - if (entry) - ext3fs_dirhash(entry->name, entry->len, hinfo); - hash = hinfo->hash; - - if (root->info.unused_flags & 1) { - ext3_warning(dir->i_sb, __func__, - "Unimplemented inode hash flags: %#06x", - root->info.unused_flags); - brelse(bh); - *err = ERR_BAD_DX_DIR; - goto fail; - } - - if ((indirect = root->info.indirect_levels) > 1) { - ext3_warning(dir->i_sb, __func__, - "Unimplemented inode hash depth: %#06x", - root->info.indirect_levels); - brelse(bh); - *err = ERR_BAD_DX_DIR; - goto fail; - } - - entries = (struct dx_entry *) (((char *)&root->info) + - root->info.info_length); - - if (dx_get_limit(entries) != dx_root_limit(dir, - root->info.info_length)) { - ext3_warning(dir->i_sb, __func__, - "dx entry: limit != root limit"); - brelse(bh); - *err = ERR_BAD_DX_DIR; - goto fail; - } - - dxtrace (printk("Look up %x", hash)); - while (1) - { - count = dx_get_count(entries); - if (!count || count > dx_get_limit(entries)) { - ext3_warning(dir->i_sb, __func__, - "dx entry: no count or count > limit"); - brelse(bh); - *err = ERR_BAD_DX_DIR; - goto fail2; - } - - p = entries + 1; - q = entries + count - 1; - while (p <= q) - { - m = p + (q - p)/2; - dxtrace(printk(".")); - if (dx_get_hash(m) > hash) - q = m - 1; - else - p = m + 1; - } - - if (0) // linear search cross check - { - unsigned n = count - 1; - at = entries; - while (n--) - { - dxtrace(printk(",")); - if (dx_get_hash(++at) > hash) - { - at--; - break; - } - } - assert (at == p - 1); - } - - at = p - 1; - dxtrace(printk(" %x->%u\n", at == entries? 0: dx_get_hash(at), dx_get_block(at))); - frame->bh = bh; - frame->entries = entries; - frame->at = at; - if (!indirect--) return frame; - if (!(bh = ext3_dir_bread(NULL, dir, dx_get_block(at), 0, err))) { - *err = ERR_BAD_DX_DIR; - goto fail2; - } - at = entries = ((struct dx_node *) bh->b_data)->entries; - if (dx_get_limit(entries) != dx_node_limit (dir)) { - ext3_warning(dir->i_sb, __func__, - "dx entry: limit != node limit"); - brelse(bh); - *err = ERR_BAD_DX_DIR; - goto fail2; - } - frame++; - frame->bh = NULL; - } -fail2: - while (frame >= frame_in) { - brelse(frame->bh); - frame--; - } -fail: - if (*err == ERR_BAD_DX_DIR) - ext3_warning(dir->i_sb, __func__, - "Corrupt dir inode %ld, running e2fsck is " - "recommended.", dir->i_ino); - return NULL; -} - -static void dx_release (struct dx_frame *frames) -{ - if (frames[0].bh == NULL) - return; - - if (((struct dx_root *) frames[0].bh->b_data)->info.indirect_levels) - brelse(frames[1].bh); - brelse(frames[0].bh); -} - -/* - * This function increments the frame pointer to search the next leaf - * block, and reads in the necessary intervening nodes if the search - * should be necessary. Whether or not the search is necessary is - * controlled by the hash parameter. If the hash value is even, then - * the search is only continued if the next block starts with that - * hash value. This is used if we are searching for a specific file. - * - * If the hash value is HASH_NB_ALWAYS, then always go to the next block. - * - * This function returns 1 if the caller should continue to search, - * or 0 if it should not. If there is an error reading one of the - * index blocks, it will a negative error code. - * - * If start_hash is non-null, it will be filled in with the starting - * hash of the next page. - */ -static int ext3_htree_next_block(struct inode *dir, __u32 hash, - struct dx_frame *frame, - struct dx_frame *frames, - __u32 *start_hash) -{ - struct dx_frame *p; - struct buffer_head *bh; - int err, num_frames = 0; - __u32 bhash; - - p = frame; - /* - * Find the next leaf page by incrementing the frame pointer. - * If we run out of entries in the interior node, loop around and - * increment pointer in the parent node. When we break out of - * this loop, num_frames indicates the number of interior - * nodes need to be read. - */ - while (1) { - if (++(p->at) < p->entries + dx_get_count(p->entries)) - break; - if (p == frames) - return 0; - num_frames++; - p--; - } - - /* - * If the hash is 1, then continue only if the next page has a - * continuation hash of any value. This is used for readdir - * handling. Otherwise, check to see if the hash matches the - * desired contiuation hash. If it doesn't, return since - * there's no point to read in the successive index pages. - */ - bhash = dx_get_hash(p->at); - if (start_hash) - *start_hash = bhash; - if ((hash & 1) == 0) { - if ((bhash & ~1) != hash) - return 0; - } - /* - * If the hash is HASH_NB_ALWAYS, we always go to the next - * block so no check is necessary - */ - while (num_frames--) { - if (!(bh = ext3_dir_bread(NULL, dir, dx_get_block(p->at), - 0, &err))) - return err; /* Failure */ - p++; - brelse (p->bh); - p->bh = bh; - p->at = p->entries = ((struct dx_node *) bh->b_data)->entries; - } - return 1; -} - - -/* - * This function fills a red-black tree with information from a - * directory block. It returns the number directory entries loaded - * into the tree. If there is an error it is returned in err. - */ -static int htree_dirblock_to_tree(struct file *dir_file, - struct inode *dir, int block, - struct dx_hash_info *hinfo, - __u32 start_hash, __u32 start_minor_hash) -{ - struct buffer_head *bh; - struct ext3_dir_entry_2 *de, *top; - int err = 0, count = 0; - - dxtrace(printk("In htree dirblock_to_tree: block %d\n", block)); - - if (!(bh = ext3_dir_bread(NULL, dir, block, 0, &err))) - return err; - - de = (struct ext3_dir_entry_2 *) bh->b_data; - top = (struct ext3_dir_entry_2 *) ((char *) de + - dir->i_sb->s_blocksize - - EXT3_DIR_REC_LEN(0)); - for (; de < top; de = ext3_next_entry(de)) { - if (!ext3_check_dir_entry("htree_dirblock_to_tree", dir, de, bh, - (block<i_sb)) - +((char *)de - bh->b_data))) { - /* silently ignore the rest of the block */ - break; - } - ext3fs_dirhash(de->name, de->name_len, hinfo); - if ((hinfo->hash < start_hash) || - ((hinfo->hash == start_hash) && - (hinfo->minor_hash < start_minor_hash))) - continue; - if (de->inode == 0) - continue; - if ((err = ext3_htree_store_dirent(dir_file, - hinfo->hash, hinfo->minor_hash, de)) != 0) { - brelse(bh); - return err; - } - count++; - } - brelse(bh); - return count; -} - - -/* - * This function fills a red-black tree with information from a - * directory. We start scanning the directory in hash order, starting - * at start_hash and start_minor_hash. - * - * This function returns the number of entries inserted into the tree, - * or a negative error code. - */ -int ext3_htree_fill_tree(struct file *dir_file, __u32 start_hash, - __u32 start_minor_hash, __u32 *next_hash) -{ - struct dx_hash_info hinfo; - struct ext3_dir_entry_2 *de; - struct dx_frame frames[2], *frame; - struct inode *dir; - int block, err; - int count = 0; - int ret; - __u32 hashval; - - dxtrace(printk("In htree_fill_tree, start hash: %x:%x\n", start_hash, - start_minor_hash)); - dir = file_inode(dir_file); - if (!(EXT3_I(dir)->i_flags & EXT3_INDEX_FL)) { - hinfo.hash_version = EXT3_SB(dir->i_sb)->s_def_hash_version; - if (hinfo.hash_version <= DX_HASH_TEA) - hinfo.hash_version += - EXT3_SB(dir->i_sb)->s_hash_unsigned; - hinfo.seed = EXT3_SB(dir->i_sb)->s_hash_seed; - count = htree_dirblock_to_tree(dir_file, dir, 0, &hinfo, - start_hash, start_minor_hash); - *next_hash = ~0; - return count; - } - hinfo.hash = start_hash; - hinfo.minor_hash = 0; - frame = dx_probe(NULL, file_inode(dir_file), &hinfo, frames, &err); - if (!frame) - return err; - - /* Add '.' and '..' from the htree header */ - if (!start_hash && !start_minor_hash) { - de = (struct ext3_dir_entry_2 *) frames[0].bh->b_data; - if ((err = ext3_htree_store_dirent(dir_file, 0, 0, de)) != 0) - goto errout; - count++; - } - if (start_hash < 2 || (start_hash ==2 && start_minor_hash==0)) { - de = (struct ext3_dir_entry_2 *) frames[0].bh->b_data; - de = ext3_next_entry(de); - if ((err = ext3_htree_store_dirent(dir_file, 2, 0, de)) != 0) - goto errout; - count++; - } - - while (1) { - block = dx_get_block(frame->at); - ret = htree_dirblock_to_tree(dir_file, dir, block, &hinfo, - start_hash, start_minor_hash); - if (ret < 0) { - err = ret; - goto errout; - } - count += ret; - hashval = ~0; - ret = ext3_htree_next_block(dir, HASH_NB_ALWAYS, - frame, frames, &hashval); - *next_hash = hashval; - if (ret < 0) { - err = ret; - goto errout; - } - /* - * Stop if: (a) there are no more entries, or - * (b) we have inserted at least one entry and the - * next hash value is not a continuation - */ - if ((ret == 0) || - (count && ((hashval & 1) == 0))) - break; - } - dx_release(frames); - dxtrace(printk("Fill tree: returned %d entries, next hash: %x\n", - count, *next_hash)); - return count; -errout: - dx_release(frames); - return (err); -} - - -/* - * Directory block splitting, compacting - */ - -/* - * Create map of hash values, offsets, and sizes, stored at end of block. - * Returns number of entries mapped. - */ -static int dx_make_map(struct ext3_dir_entry_2 *de, unsigned blocksize, - struct dx_hash_info *hinfo, struct dx_map_entry *map_tail) -{ - int count = 0; - char *base = (char *) de; - struct dx_hash_info h = *hinfo; - - while ((char *) de < base + blocksize) - { - if (de->name_len && de->inode) { - ext3fs_dirhash(de->name, de->name_len, &h); - map_tail--; - map_tail->hash = h.hash; - map_tail->offs = (u16) ((char *) de - base); - map_tail->size = le16_to_cpu(de->rec_len); - count++; - cond_resched(); - } - /* XXX: do we need to check rec_len == 0 case? -Chris */ - de = ext3_next_entry(de); - } - return count; -} - -/* Sort map by hash value */ -static void dx_sort_map (struct dx_map_entry *map, unsigned count) -{ - struct dx_map_entry *p, *q, *top = map + count - 1; - int more; - /* Combsort until bubble sort doesn't suck */ - while (count > 2) - { - count = count*10/13; - if (count - 9 < 2) /* 9, 10 -> 11 */ - count = 11; - for (p = top, q = p - count; q >= map; p--, q--) - if (p->hash < q->hash) - swap(*p, *q); - } - /* Garden variety bubble sort */ - do { - more = 0; - q = top; - while (q-- > map) - { - if (q[1].hash >= q[0].hash) - continue; - swap(*(q+1), *q); - more = 1; - } - } while(more); -} - -static void dx_insert_block(struct dx_frame *frame, u32 hash, u32 block) -{ - struct dx_entry *entries = frame->entries; - struct dx_entry *old = frame->at, *new = old + 1; - int count = dx_get_count(entries); - - assert(count < dx_get_limit(entries)); - assert(old < entries + count); - memmove(new + 1, new, (char *)(entries + count) - (char *)(new)); - dx_set_hash(new, hash); - dx_set_block(new, block); - dx_set_count(entries, count + 1); -} - -static void ext3_update_dx_flag(struct inode *inode) -{ - if (!EXT3_HAS_COMPAT_FEATURE(inode->i_sb, - EXT3_FEATURE_COMPAT_DIR_INDEX)) - EXT3_I(inode)->i_flags &= ~EXT3_INDEX_FL; -} - -/* - * NOTE! unlike strncmp, ext3_match returns 1 for success, 0 for failure. - * - * `len <= EXT3_NAME_LEN' is guaranteed by caller. - * `de != NULL' is guaranteed by caller. - */ -static inline int ext3_match (int len, const char * const name, - struct ext3_dir_entry_2 * de) -{ - if (len != de->name_len) - return 0; - if (!de->inode) - return 0; - return !memcmp(name, de->name, len); -} - -/* - * Returns 0 if not found, -1 on failure, and 1 on success - */ -static inline int search_dirblock(struct buffer_head * bh, - struct inode *dir, - struct qstr *child, - unsigned long offset, - struct ext3_dir_entry_2 ** res_dir) -{ - struct ext3_dir_entry_2 * de; - char * dlimit; - int de_len; - const char *name = child->name; - int namelen = child->len; - - de = (struct ext3_dir_entry_2 *) bh->b_data; - dlimit = bh->b_data + dir->i_sb->s_blocksize; - while ((char *) de < dlimit) { - /* this code is executed quadratically often */ - /* do minimal checking `by hand' */ - - if ((char *) de + namelen <= dlimit && - ext3_match (namelen, name, de)) { - /* found a match - just to be sure, do a full check */ - if (!ext3_check_dir_entry("ext3_find_entry", - dir, de, bh, offset)) - return -1; - *res_dir = de; - return 1; - } - /* prevent looping on a bad block */ - de_len = ext3_rec_len_from_disk(de->rec_len); - if (de_len <= 0) - return -1; - offset += de_len; - de = (struct ext3_dir_entry_2 *) ((char *) de + de_len); - } - return 0; -} - - -/* - * ext3_find_entry() - * - * finds an entry in the specified directory with the wanted name. It - * returns the cache buffer in which the entry was found, and the entry - * itself (as a parameter - res_dir). It does NOT read the inode of the - * entry - you'll have to do that yourself if you want to. - * - * The returned buffer_head has ->b_count elevated. The caller is expected - * to brelse() it when appropriate. - */ -static struct buffer_head *ext3_find_entry(struct inode *dir, - struct qstr *entry, - struct ext3_dir_entry_2 **res_dir) -{ - struct super_block * sb; - struct buffer_head * bh_use[NAMEI_RA_SIZE]; - struct buffer_head * bh, *ret = NULL; - unsigned long start, block, b; - const u8 *name = entry->name; - int ra_max = 0; /* Number of bh's in the readahead - buffer, bh_use[] */ - int ra_ptr = 0; /* Current index into readahead - buffer */ - int num = 0; - int nblocks, i, err; - int namelen; - - *res_dir = NULL; - sb = dir->i_sb; - namelen = entry->len; - if (namelen > EXT3_NAME_LEN) - return NULL; - if ((namelen <= 2) && (name[0] == '.') && - (name[1] == '.' || name[1] == 0)) { - /* - * "." or ".." will only be in the first block - * NFS may look up ".."; "." should be handled by the VFS - */ - block = start = 0; - nblocks = 1; - goto restart; - } - if (is_dx(dir)) { - bh = ext3_dx_find_entry(dir, entry, res_dir, &err); - /* - * On success, or if the error was file not found, - * return. Otherwise, fall back to doing a search the - * old fashioned way. - */ - if (bh || (err != ERR_BAD_DX_DIR)) - return bh; - dxtrace(printk("ext3_find_entry: dx failed, falling back\n")); - } - nblocks = dir->i_size >> EXT3_BLOCK_SIZE_BITS(sb); - start = EXT3_I(dir)->i_dir_start_lookup; - if (start >= nblocks) - start = 0; - block = start; -restart: - do { - /* - * We deal with the read-ahead logic here. - */ - if (ra_ptr >= ra_max) { - /* Refill the readahead buffer */ - ra_ptr = 0; - b = block; - for (ra_max = 0; ra_max < NAMEI_RA_SIZE; ra_max++) { - /* - * Terminate if we reach the end of the - * directory and must wrap, or if our - * search has finished at this block. - */ - if (b >= nblocks || (num && block == start)) { - bh_use[ra_max] = NULL; - break; - } - num++; - bh = ext3_getblk(NULL, dir, b++, 0, &err); - bh_use[ra_max] = bh; - if (bh && !bh_uptodate_or_lock(bh)) { - get_bh(bh); - bh->b_end_io = end_buffer_read_sync; - submit_bh(READ | REQ_META | REQ_PRIO, - bh); - } - } - } - if ((bh = bh_use[ra_ptr++]) == NULL) - goto next; - wait_on_buffer(bh); - if (!buffer_uptodate(bh)) { - /* read error, skip block & hope for the best */ - ext3_error(sb, __func__, "reading directory #%lu " - "offset %lu", dir->i_ino, block); - brelse(bh); - goto next; - } - i = search_dirblock(bh, dir, entry, - block << EXT3_BLOCK_SIZE_BITS(sb), res_dir); - if (i == 1) { - EXT3_I(dir)->i_dir_start_lookup = block; - ret = bh; - goto cleanup_and_exit; - } else { - brelse(bh); - if (i < 0) - goto cleanup_and_exit; - } - next: - if (++block >= nblocks) - block = 0; - } while (block != start); - - /* - * If the directory has grown while we were searching, then - * search the last part of the directory before giving up. - */ - block = nblocks; - nblocks = dir->i_size >> EXT3_BLOCK_SIZE_BITS(sb); - if (block < nblocks) { - start = 0; - goto restart; - } - -cleanup_and_exit: - /* Clean up the read-ahead blocks */ - for (; ra_ptr < ra_max; ra_ptr++) - brelse (bh_use[ra_ptr]); - return ret; -} - -static struct buffer_head * ext3_dx_find_entry(struct inode *dir, - struct qstr *entry, struct ext3_dir_entry_2 **res_dir, - int *err) -{ - struct super_block *sb = dir->i_sb; - struct dx_hash_info hinfo; - struct dx_frame frames[2], *frame; - struct buffer_head *bh; - unsigned long block; - int retval; - - if (!(frame = dx_probe(entry, dir, &hinfo, frames, err))) - return NULL; - do { - block = dx_get_block(frame->at); - if (!(bh = ext3_dir_bread (NULL, dir, block, 0, err))) - goto errout; - - retval = search_dirblock(bh, dir, entry, - block << EXT3_BLOCK_SIZE_BITS(sb), - res_dir); - if (retval == 1) { - dx_release(frames); - return bh; - } - brelse(bh); - if (retval == -1) { - *err = ERR_BAD_DX_DIR; - goto errout; - } - - /* Check to see if we should continue to search */ - retval = ext3_htree_next_block(dir, hinfo.hash, frame, - frames, NULL); - if (retval < 0) { - ext3_warning(sb, __func__, - "error reading index page in directory #%lu", - dir->i_ino); - *err = retval; - goto errout; - } - } while (retval == 1); - - *err = -ENOENT; -errout: - dxtrace(printk("%s not found\n", entry->name)); - dx_release (frames); - return NULL; -} - -static struct dentry *ext3_lookup(struct inode * dir, struct dentry *dentry, unsigned int flags) -{ - struct inode * inode; - struct ext3_dir_entry_2 * de; - struct buffer_head * bh; - - if (dentry->d_name.len > EXT3_NAME_LEN) - return ERR_PTR(-ENAMETOOLONG); - - bh = ext3_find_entry(dir, &dentry->d_name, &de); - inode = NULL; - if (bh) { - unsigned long ino = le32_to_cpu(de->inode); - brelse (bh); - if (!ext3_valid_inum(dir->i_sb, ino)) { - ext3_error(dir->i_sb, "ext3_lookup", - "bad inode number: %lu", ino); - return ERR_PTR(-EIO); - } - inode = ext3_iget(dir->i_sb, ino); - if (inode == ERR_PTR(-ESTALE)) { - ext3_error(dir->i_sb, __func__, - "deleted inode referenced: %lu", - ino); - return ERR_PTR(-EIO); - } - } - return d_splice_alias(inode, dentry); -} - - -struct dentry *ext3_get_parent(struct dentry *child) -{ - unsigned long ino; - struct qstr dotdot = QSTR_INIT("..", 2); - struct ext3_dir_entry_2 * de; - struct buffer_head *bh; - - bh = ext3_find_entry(d_inode(child), &dotdot, &de); - if (!bh) - return ERR_PTR(-ENOENT); - ino = le32_to_cpu(de->inode); - brelse(bh); - - if (!ext3_valid_inum(d_inode(child)->i_sb, ino)) { - ext3_error(d_inode(child)->i_sb, "ext3_get_parent", - "bad inode number: %lu", ino); - return ERR_PTR(-EIO); - } - - return d_obtain_alias(ext3_iget(d_inode(child)->i_sb, ino)); -} - -#define S_SHIFT 12 -static unsigned char ext3_type_by_mode[S_IFMT >> S_SHIFT] = { - [S_IFREG >> S_SHIFT] = EXT3_FT_REG_FILE, - [S_IFDIR >> S_SHIFT] = EXT3_FT_DIR, - [S_IFCHR >> S_SHIFT] = EXT3_FT_CHRDEV, - [S_IFBLK >> S_SHIFT] = EXT3_FT_BLKDEV, - [S_IFIFO >> S_SHIFT] = EXT3_FT_FIFO, - [S_IFSOCK >> S_SHIFT] = EXT3_FT_SOCK, - [S_IFLNK >> S_SHIFT] = EXT3_FT_SYMLINK, -}; - -static inline void ext3_set_de_type(struct super_block *sb, - struct ext3_dir_entry_2 *de, - umode_t mode) { - if (EXT3_HAS_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_FILETYPE)) - de->file_type = ext3_type_by_mode[(mode & S_IFMT)>>S_SHIFT]; -} - -/* - * Move count entries from end of map between two memory locations. - * Returns pointer to last entry moved. - */ -static struct ext3_dir_entry_2 * -dx_move_dirents(char *from, char *to, struct dx_map_entry *map, int count) -{ - unsigned rec_len = 0; - - while (count--) { - struct ext3_dir_entry_2 *de = (struct ext3_dir_entry_2 *) (from + map->offs); - rec_len = EXT3_DIR_REC_LEN(de->name_len); - memcpy (to, de, rec_len); - ((struct ext3_dir_entry_2 *) to)->rec_len = - ext3_rec_len_to_disk(rec_len); - de->inode = 0; - map++; - to += rec_len; - } - return (struct ext3_dir_entry_2 *) (to - rec_len); -} - -/* - * Compact each dir entry in the range to the minimal rec_len. - * Returns pointer to last entry in range. - */ -static struct ext3_dir_entry_2 *dx_pack_dirents(char *base, unsigned blocksize) -{ - struct ext3_dir_entry_2 *next, *to, *prev; - struct ext3_dir_entry_2 *de = (struct ext3_dir_entry_2 *)base; - unsigned rec_len = 0; - - prev = to = de; - while ((char *)de < base + blocksize) { - next = ext3_next_entry(de); - if (de->inode && de->name_len) { - rec_len = EXT3_DIR_REC_LEN(de->name_len); - if (de > to) - memmove(to, de, rec_len); - to->rec_len = ext3_rec_len_to_disk(rec_len); - prev = to; - to = (struct ext3_dir_entry_2 *) (((char *) to) + rec_len); - } - de = next; - } - return prev; -} - -/* - * Split a full leaf block to make room for a new dir entry. - * Allocate a new block, and move entries so that they are approx. equally full. - * Returns pointer to de in block into which the new entry will be inserted. - */ -static struct ext3_dir_entry_2 *do_split(handle_t *handle, struct inode *dir, - struct buffer_head **bh,struct dx_frame *frame, - struct dx_hash_info *hinfo, int *error) -{ - unsigned blocksize = dir->i_sb->s_blocksize; - unsigned count, continued; - struct buffer_head *bh2; - u32 newblock; - u32 hash2; - struct dx_map_entry *map; - char *data1 = (*bh)->b_data, *data2; - unsigned split, move, size; - struct ext3_dir_entry_2 *de = NULL, *de2; - int err = 0, i; - - bh2 = ext3_append (handle, dir, &newblock, &err); - if (!(bh2)) { - brelse(*bh); - *bh = NULL; - goto errout; - } - - BUFFER_TRACE(*bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, *bh); - if (err) - goto journal_error; - - BUFFER_TRACE(frame->bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, frame->bh); - if (err) - goto journal_error; - - data2 = bh2->b_data; - - /* create map in the end of data2 block */ - map = (struct dx_map_entry *) (data2 + blocksize); - count = dx_make_map ((struct ext3_dir_entry_2 *) data1, - blocksize, hinfo, map); - map -= count; - dx_sort_map (map, count); - /* Split the existing block in the middle, size-wise */ - size = 0; - move = 0; - for (i = count-1; i >= 0; i--) { - /* is more than half of this entry in 2nd half of the block? */ - if (size + map[i].size/2 > blocksize/2) - break; - size += map[i].size; - move++; - } - /* map index at which we will split */ - split = count - move; - hash2 = map[split].hash; - continued = hash2 == map[split - 1].hash; - dxtrace(printk("Split block %i at %x, %i/%i\n", - dx_get_block(frame->at), hash2, split, count-split)); - - /* Fancy dance to stay within two buffers */ - de2 = dx_move_dirents(data1, data2, map + split, count - split); - de = dx_pack_dirents(data1,blocksize); - de->rec_len = ext3_rec_len_to_disk(data1 + blocksize - (char *) de); - de2->rec_len = ext3_rec_len_to_disk(data2 + blocksize - (char *) de2); - dxtrace(dx_show_leaf (hinfo, (struct ext3_dir_entry_2 *) data1, blocksize, 1)); - dxtrace(dx_show_leaf (hinfo, (struct ext3_dir_entry_2 *) data2, blocksize, 1)); - - /* Which block gets the new entry? */ - if (hinfo->hash >= hash2) - { - swap(*bh, bh2); - de = de2; - } - dx_insert_block (frame, hash2 + continued, newblock); - err = ext3_journal_dirty_metadata (handle, bh2); - if (err) - goto journal_error; - err = ext3_journal_dirty_metadata (handle, frame->bh); - if (err) - goto journal_error; - brelse (bh2); - dxtrace(dx_show_index ("frame", frame->entries)); - return de; - -journal_error: - brelse(*bh); - brelse(bh2); - *bh = NULL; - ext3_std_error(dir->i_sb, err); -errout: - *error = err; - return NULL; -} - - -/* - * Add a new entry into a directory (leaf) block. If de is non-NULL, - * it points to a directory entry which is guaranteed to be large - * enough for new directory entry. If de is NULL, then - * add_dirent_to_buf will attempt search the directory block for - * space. It will return -ENOSPC if no space is available, and -EIO - * and -EEXIST if directory entry already exists. - * - * NOTE! bh is NOT released in the case where ENOSPC is returned. In - * all other cases bh is released. - */ -static int add_dirent_to_buf(handle_t *handle, struct dentry *dentry, - struct inode *inode, struct ext3_dir_entry_2 *de, - struct buffer_head * bh) -{ - struct inode *dir = d_inode(dentry->d_parent); - const char *name = dentry->d_name.name; - int namelen = dentry->d_name.len; - unsigned long offset = 0; - unsigned short reclen; - int nlen, rlen, err; - char *top; - - reclen = EXT3_DIR_REC_LEN(namelen); - if (!de) { - de = (struct ext3_dir_entry_2 *)bh->b_data; - top = bh->b_data + dir->i_sb->s_blocksize - reclen; - while ((char *) de <= top) { - if (!ext3_check_dir_entry("ext3_add_entry", dir, de, - bh, offset)) { - brelse (bh); - return -EIO; - } - if (ext3_match (namelen, name, de)) { - brelse (bh); - return -EEXIST; - } - nlen = EXT3_DIR_REC_LEN(de->name_len); - rlen = ext3_rec_len_from_disk(de->rec_len); - if ((de->inode? rlen - nlen: rlen) >= reclen) - break; - de = (struct ext3_dir_entry_2 *)((char *)de + rlen); - offset += rlen; - } - if ((char *) de > top) - return -ENOSPC; - } - BUFFER_TRACE(bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, bh); - if (err) { - ext3_std_error(dir->i_sb, err); - brelse(bh); - return err; - } - - /* By now the buffer is marked for journaling */ - nlen = EXT3_DIR_REC_LEN(de->name_len); - rlen = ext3_rec_len_from_disk(de->rec_len); - if (de->inode) { - struct ext3_dir_entry_2 *de1 = (struct ext3_dir_entry_2 *)((char *)de + nlen); - de1->rec_len = ext3_rec_len_to_disk(rlen - nlen); - de->rec_len = ext3_rec_len_to_disk(nlen); - de = de1; - } - de->file_type = EXT3_FT_UNKNOWN; - if (inode) { - de->inode = cpu_to_le32(inode->i_ino); - ext3_set_de_type(dir->i_sb, de, inode->i_mode); - } else - de->inode = 0; - de->name_len = namelen; - memcpy (de->name, name, namelen); - /* - * XXX shouldn't update any times until successful - * completion of syscall, but too many callers depend - * on this. - * - * XXX similarly, too many callers depend on - * ext3_new_inode() setting the times, but error - * recovery deletes the inode, so the worst that can - * happen is that the times are slightly out of date - * and/or different from the directory change time. - */ - dir->i_mtime = dir->i_ctime = CURRENT_TIME_SEC; - ext3_update_dx_flag(dir); - dir->i_version++; - ext3_mark_inode_dirty(handle, dir); - BUFFER_TRACE(bh, "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, bh); - if (err) - ext3_std_error(dir->i_sb, err); - brelse(bh); - return 0; -} - -/* - * This converts a one block unindexed directory to a 3 block indexed - * directory, and adds the dentry to the indexed directory. - */ -static int make_indexed_dir(handle_t *handle, struct dentry *dentry, - struct inode *inode, struct buffer_head *bh) -{ - struct inode *dir = d_inode(dentry->d_parent); - const char *name = dentry->d_name.name; - int namelen = dentry->d_name.len; - struct buffer_head *bh2; - struct dx_root *root; - struct dx_frame frames[2], *frame; - struct dx_entry *entries; - struct ext3_dir_entry_2 *de, *de2; - char *data1, *top; - unsigned len; - int retval; - unsigned blocksize; - struct dx_hash_info hinfo; - u32 block; - struct fake_dirent *fde; - - blocksize = dir->i_sb->s_blocksize; - dxtrace(printk(KERN_DEBUG "Creating index: inode %lu\n", dir->i_ino)); - retval = ext3_journal_get_write_access(handle, bh); - if (retval) { - ext3_std_error(dir->i_sb, retval); - brelse(bh); - return retval; - } - root = (struct dx_root *) bh->b_data; - - /* The 0th block becomes the root, move the dirents out */ - fde = &root->dotdot; - de = (struct ext3_dir_entry_2 *)((char *)fde + - ext3_rec_len_from_disk(fde->rec_len)); - if ((char *) de >= (((char *) root) + blocksize)) { - ext3_error(dir->i_sb, __func__, - "invalid rec_len for '..' in inode %lu", - dir->i_ino); - brelse(bh); - return -EIO; - } - len = ((char *) root) + blocksize - (char *) de; - - bh2 = ext3_append (handle, dir, &block, &retval); - if (!(bh2)) { - brelse(bh); - return retval; - } - EXT3_I(dir)->i_flags |= EXT3_INDEX_FL; - data1 = bh2->b_data; - - memcpy (data1, de, len); - de = (struct ext3_dir_entry_2 *) data1; - top = data1 + len; - while ((char *)(de2 = ext3_next_entry(de)) < top) - de = de2; - de->rec_len = ext3_rec_len_to_disk(data1 + blocksize - (char *) de); - /* Initialize the root; the dot dirents already exist */ - de = (struct ext3_dir_entry_2 *) (&root->dotdot); - de->rec_len = ext3_rec_len_to_disk(blocksize - EXT3_DIR_REC_LEN(2)); - memset (&root->info, 0, sizeof(root->info)); - root->info.info_length = sizeof(root->info); - root->info.hash_version = EXT3_SB(dir->i_sb)->s_def_hash_version; - entries = root->entries; - dx_set_block (entries, 1); - dx_set_count (entries, 1); - dx_set_limit (entries, dx_root_limit(dir, sizeof(root->info))); - - /* Initialize as for dx_probe */ - hinfo.hash_version = root->info.hash_version; - if (hinfo.hash_version <= DX_HASH_TEA) - hinfo.hash_version += EXT3_SB(dir->i_sb)->s_hash_unsigned; - hinfo.seed = EXT3_SB(dir->i_sb)->s_hash_seed; - ext3fs_dirhash(name, namelen, &hinfo); - frame = frames; - frame->entries = entries; - frame->at = entries; - frame->bh = bh; - bh = bh2; - /* - * Mark buffers dirty here so that if do_split() fails we write a - * consistent set of buffers to disk. - */ - ext3_journal_dirty_metadata(handle, frame->bh); - ext3_journal_dirty_metadata(handle, bh); - de = do_split(handle,dir, &bh, frame, &hinfo, &retval); - if (!de) { - ext3_mark_inode_dirty(handle, dir); - dx_release(frames); - return retval; - } - dx_release(frames); - - return add_dirent_to_buf(handle, dentry, inode, de, bh); -} - -/* - * ext3_add_entry() - * - * adds a file entry to the specified directory, using the same - * semantics as ext3_find_entry(). It returns NULL if it failed. - * - * NOTE!! The inode part of 'de' is left at 0 - which means you - * may not sleep between calling this and putting something into - * the entry, as someone else might have used it while you slept. - */ -static int ext3_add_entry (handle_t *handle, struct dentry *dentry, - struct inode *inode) -{ - struct inode *dir = d_inode(dentry->d_parent); - struct buffer_head * bh; - struct ext3_dir_entry_2 *de; - struct super_block * sb; - int retval; - int dx_fallback=0; - unsigned blocksize; - u32 block, blocks; - - sb = dir->i_sb; - blocksize = sb->s_blocksize; - if (!dentry->d_name.len) - return -EINVAL; - if (is_dx(dir)) { - retval = ext3_dx_add_entry(handle, dentry, inode); - if (!retval || (retval != ERR_BAD_DX_DIR)) - return retval; - EXT3_I(dir)->i_flags &= ~EXT3_INDEX_FL; - dx_fallback++; - ext3_mark_inode_dirty(handle, dir); - } - blocks = dir->i_size >> sb->s_blocksize_bits; - for (block = 0; block < blocks; block++) { - if (!(bh = ext3_dir_bread(handle, dir, block, 0, &retval))) - return retval; - - retval = add_dirent_to_buf(handle, dentry, inode, NULL, bh); - if (retval != -ENOSPC) - return retval; - - if (blocks == 1 && !dx_fallback && - EXT3_HAS_COMPAT_FEATURE(sb, EXT3_FEATURE_COMPAT_DIR_INDEX)) - return make_indexed_dir(handle, dentry, inode, bh); - brelse(bh); - } - bh = ext3_append(handle, dir, &block, &retval); - if (!bh) - return retval; - de = (struct ext3_dir_entry_2 *) bh->b_data; - de->inode = 0; - de->rec_len = ext3_rec_len_to_disk(blocksize); - return add_dirent_to_buf(handle, dentry, inode, de, bh); -} - -/* - * Returns 0 for success, or a negative error value - */ -static int ext3_dx_add_entry(handle_t *handle, struct dentry *dentry, - struct inode *inode) -{ - struct dx_frame frames[2], *frame; - struct dx_entry *entries, *at; - struct dx_hash_info hinfo; - struct buffer_head * bh; - struct inode *dir = d_inode(dentry->d_parent); - struct super_block * sb = dir->i_sb; - struct ext3_dir_entry_2 *de; - int err; - - frame = dx_probe(&dentry->d_name, dir, &hinfo, frames, &err); - if (!frame) - return err; - entries = frame->entries; - at = frame->at; - - if (!(bh = ext3_dir_bread(handle, dir, dx_get_block(frame->at), 0, &err))) - goto cleanup; - - BUFFER_TRACE(bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, bh); - if (err) - goto journal_error; - - err = add_dirent_to_buf(handle, dentry, inode, NULL, bh); - if (err != -ENOSPC) { - bh = NULL; - goto cleanup; - } - - /* Block full, should compress but for now just split */ - dxtrace(printk("using %u of %u node entries\n", - dx_get_count(entries), dx_get_limit(entries))); - /* Need to split index? */ - if (dx_get_count(entries) == dx_get_limit(entries)) { - u32 newblock; - unsigned icount = dx_get_count(entries); - int levels = frame - frames; - struct dx_entry *entries2; - struct dx_node *node2; - struct buffer_head *bh2; - - if (levels && (dx_get_count(frames->entries) == - dx_get_limit(frames->entries))) { - ext3_warning(sb, __func__, - "Directory index full!"); - err = -ENOSPC; - goto cleanup; - } - bh2 = ext3_append (handle, dir, &newblock, &err); - if (!(bh2)) - goto cleanup; - node2 = (struct dx_node *)(bh2->b_data); - entries2 = node2->entries; - memset(&node2->fake, 0, sizeof(struct fake_dirent)); - node2->fake.rec_len = ext3_rec_len_to_disk(sb->s_blocksize); - BUFFER_TRACE(frame->bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, frame->bh); - if (err) - goto journal_error; - if (levels) { - unsigned icount1 = icount/2, icount2 = icount - icount1; - unsigned hash2 = dx_get_hash(entries + icount1); - dxtrace(printk("Split index %i/%i\n", icount1, icount2)); - - BUFFER_TRACE(frame->bh, "get_write_access"); /* index root */ - err = ext3_journal_get_write_access(handle, - frames[0].bh); - if (err) - goto journal_error; - - memcpy ((char *) entries2, (char *) (entries + icount1), - icount2 * sizeof(struct dx_entry)); - dx_set_count (entries, icount1); - dx_set_count (entries2, icount2); - dx_set_limit (entries2, dx_node_limit(dir)); - - /* Which index block gets the new entry? */ - if (at - entries >= icount1) { - frame->at = at = at - entries - icount1 + entries2; - frame->entries = entries = entries2; - swap(frame->bh, bh2); - } - dx_insert_block (frames + 0, hash2, newblock); - dxtrace(dx_show_index ("node", frames[1].entries)); - dxtrace(dx_show_index ("node", - ((struct dx_node *) bh2->b_data)->entries)); - err = ext3_journal_dirty_metadata(handle, bh2); - if (err) - goto journal_error; - brelse (bh2); - } else { - dxtrace(printk("Creating second level index...\n")); - memcpy((char *) entries2, (char *) entries, - icount * sizeof(struct dx_entry)); - dx_set_limit(entries2, dx_node_limit(dir)); - - /* Set up root */ - dx_set_count(entries, 1); - dx_set_block(entries + 0, newblock); - ((struct dx_root *) frames[0].bh->b_data)->info.indirect_levels = 1; - - /* Add new access path frame */ - frame = frames + 1; - frame->at = at = at - entries + entries2; - frame->entries = entries = entries2; - frame->bh = bh2; - err = ext3_journal_get_write_access(handle, - frame->bh); - if (err) - goto journal_error; - } - err = ext3_journal_dirty_metadata(handle, frames[0].bh); - if (err) - goto journal_error; - } - de = do_split(handle, dir, &bh, frame, &hinfo, &err); - if (!de) - goto cleanup; - err = add_dirent_to_buf(handle, dentry, inode, de, bh); - bh = NULL; - goto cleanup; - -journal_error: - ext3_std_error(dir->i_sb, err); -cleanup: - if (bh) - brelse(bh); - dx_release(frames); - return err; -} - -/* - * ext3_delete_entry deletes a directory entry by merging it with the - * previous entry - */ -static int ext3_delete_entry (handle_t *handle, - struct inode * dir, - struct ext3_dir_entry_2 * de_del, - struct buffer_head * bh) -{ - struct ext3_dir_entry_2 * de, * pde; - int i; - - i = 0; - pde = NULL; - de = (struct ext3_dir_entry_2 *) bh->b_data; - while (i < bh->b_size) { - if (!ext3_check_dir_entry("ext3_delete_entry", dir, de, bh, i)) - return -EIO; - if (de == de_del) { - int err; - - BUFFER_TRACE(bh, "get_write_access"); - err = ext3_journal_get_write_access(handle, bh); - if (err) - goto journal_error; - - if (pde) - pde->rec_len = ext3_rec_len_to_disk( - ext3_rec_len_from_disk(pde->rec_len) + - ext3_rec_len_from_disk(de->rec_len)); - else - de->inode = 0; - dir->i_version++; - BUFFER_TRACE(bh, "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, bh); - if (err) { -journal_error: - ext3_std_error(dir->i_sb, err); - return err; - } - return 0; - } - i += ext3_rec_len_from_disk(de->rec_len); - pde = de; - de = ext3_next_entry(de); - } - return -ENOENT; -} - -static int ext3_add_nondir(handle_t *handle, - struct dentry *dentry, struct inode *inode) -{ - int err = ext3_add_entry(handle, dentry, inode); - if (!err) { - ext3_mark_inode_dirty(handle, inode); - unlock_new_inode(inode); - d_instantiate(dentry, inode); - return 0; - } - drop_nlink(inode); - unlock_new_inode(inode); - iput(inode); - return err; -} - -/* - * By the time this is called, we already have created - * the directory cache entry for the new file, but it - * is so far negative - it has no inode. - * - * If the create succeeds, we fill in the inode information - * with d_instantiate(). - */ -static int ext3_create (struct inode * dir, struct dentry * dentry, umode_t mode, - bool excl) -{ - handle_t *handle; - struct inode * inode; - int err, retries = 0; - - dquot_initialize(dir); - -retry: - handle = ext3_journal_start(dir, EXT3_DATA_TRANS_BLOCKS(dir->i_sb) + - EXT3_INDEX_EXTRA_TRANS_BLOCKS + 3 + - EXT3_MAXQUOTAS_INIT_BLOCKS(dir->i_sb)); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - if (IS_DIRSYNC(dir)) - handle->h_sync = 1; - - inode = ext3_new_inode (handle, dir, &dentry->d_name, mode); - err = PTR_ERR(inode); - if (!IS_ERR(inode)) { - inode->i_op = &ext3_file_inode_operations; - inode->i_fop = &ext3_file_operations; - ext3_set_aops(inode); - err = ext3_add_nondir(handle, dentry, inode); - } - ext3_journal_stop(handle); - if (err == -ENOSPC && ext3_should_retry_alloc(dir->i_sb, &retries)) - goto retry; - return err; -} - -static int ext3_mknod (struct inode * dir, struct dentry *dentry, - umode_t mode, dev_t rdev) -{ - handle_t *handle; - struct inode *inode; - int err, retries = 0; - - if (!new_valid_dev(rdev)) - return -EINVAL; - - dquot_initialize(dir); - -retry: - handle = ext3_journal_start(dir, EXT3_DATA_TRANS_BLOCKS(dir->i_sb) + - EXT3_INDEX_EXTRA_TRANS_BLOCKS + 3 + - EXT3_MAXQUOTAS_INIT_BLOCKS(dir->i_sb)); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - if (IS_DIRSYNC(dir)) - handle->h_sync = 1; - - inode = ext3_new_inode (handle, dir, &dentry->d_name, mode); - err = PTR_ERR(inode); - if (!IS_ERR(inode)) { - init_special_inode(inode, inode->i_mode, rdev); -#ifdef CONFIG_EXT3_FS_XATTR - inode->i_op = &ext3_special_inode_operations; -#endif - err = ext3_add_nondir(handle, dentry, inode); - } - ext3_journal_stop(handle); - if (err == -ENOSPC && ext3_should_retry_alloc(dir->i_sb, &retries)) - goto retry; - return err; -} - -static int ext3_tmpfile(struct inode *dir, struct dentry *dentry, umode_t mode) -{ - handle_t *handle; - struct inode *inode; - int err, retries = 0; - - dquot_initialize(dir); - -retry: - handle = ext3_journal_start(dir, EXT3_MAXQUOTAS_INIT_BLOCKS(dir->i_sb) + - 4 + EXT3_XATTR_TRANS_BLOCKS); - - if (IS_ERR(handle)) - return PTR_ERR(handle); - - inode = ext3_new_inode (handle, dir, NULL, mode); - err = PTR_ERR(inode); - if (!IS_ERR(inode)) { - inode->i_op = &ext3_file_inode_operations; - inode->i_fop = &ext3_file_operations; - ext3_set_aops(inode); - d_tmpfile(dentry, inode); - err = ext3_orphan_add(handle, inode); - if (err) - goto err_unlock_inode; - mark_inode_dirty(inode); - unlock_new_inode(inode); - } - ext3_journal_stop(handle); - if (err == -ENOSPC && ext3_should_retry_alloc(dir->i_sb, &retries)) - goto retry; - return err; -err_unlock_inode: - ext3_journal_stop(handle); - unlock_new_inode(inode); - return err; -} - -static int ext3_mkdir(struct inode * dir, struct dentry * dentry, umode_t mode) -{ - handle_t *handle; - struct inode * inode; - struct buffer_head * dir_block = NULL; - struct ext3_dir_entry_2 * de; - int err, retries = 0; - - if (dir->i_nlink >= EXT3_LINK_MAX) - return -EMLINK; - - dquot_initialize(dir); - -retry: - handle = ext3_journal_start(dir, EXT3_DATA_TRANS_BLOCKS(dir->i_sb) + - EXT3_INDEX_EXTRA_TRANS_BLOCKS + 3 + - EXT3_MAXQUOTAS_INIT_BLOCKS(dir->i_sb)); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - if (IS_DIRSYNC(dir)) - handle->h_sync = 1; - - inode = ext3_new_inode (handle, dir, &dentry->d_name, S_IFDIR | mode); - err = PTR_ERR(inode); - if (IS_ERR(inode)) - goto out_stop; - - inode->i_op = &ext3_dir_inode_operations; - inode->i_fop = &ext3_dir_operations; - inode->i_size = EXT3_I(inode)->i_disksize = inode->i_sb->s_blocksize; - if (!(dir_block = ext3_dir_bread(handle, inode, 0, 1, &err))) - goto out_clear_inode; - - BUFFER_TRACE(dir_block, "get_write_access"); - err = ext3_journal_get_write_access(handle, dir_block); - if (err) - goto out_clear_inode; - - de = (struct ext3_dir_entry_2 *) dir_block->b_data; - de->inode = cpu_to_le32(inode->i_ino); - de->name_len = 1; - de->rec_len = ext3_rec_len_to_disk(EXT3_DIR_REC_LEN(de->name_len)); - strcpy (de->name, "."); - ext3_set_de_type(dir->i_sb, de, S_IFDIR); - de = ext3_next_entry(de); - de->inode = cpu_to_le32(dir->i_ino); - de->rec_len = ext3_rec_len_to_disk(inode->i_sb->s_blocksize - - EXT3_DIR_REC_LEN(1)); - de->name_len = 2; - strcpy (de->name, ".."); - ext3_set_de_type(dir->i_sb, de, S_IFDIR); - set_nlink(inode, 2); - BUFFER_TRACE(dir_block, "call ext3_journal_dirty_metadata"); - err = ext3_journal_dirty_metadata(handle, dir_block); - if (err) - goto out_clear_inode; - - err = ext3_mark_inode_dirty(handle, inode); - if (!err) - err = ext3_add_entry (handle, dentry, inode); - - if (err) { -out_clear_inode: - clear_nlink(inode); - unlock_new_inode(inode); - ext3_mark_inode_dirty(handle, inode); - iput (inode); - goto out_stop; - } - inc_nlink(dir); - ext3_update_dx_flag(dir); - err = ext3_mark_inode_dirty(handle, dir); - if (err) - goto out_clear_inode; - - unlock_new_inode(inode); - d_instantiate(dentry, inode); -out_stop: - brelse(dir_block); - ext3_journal_stop(handle); - if (err == -ENOSPC && ext3_should_retry_alloc(dir->i_sb, &retries)) - goto retry; - return err; -} - -/* - * routine to check that the specified directory is empty (for rmdir) - */ -static int empty_dir (struct inode * inode) -{ - unsigned long offset; - struct buffer_head * bh; - struct ext3_dir_entry_2 * de, * de1; - struct super_block * sb; - int err = 0; - - sb = inode->i_sb; - if (inode->i_size < EXT3_DIR_REC_LEN(1) + EXT3_DIR_REC_LEN(2) || - !(bh = ext3_dir_bread(NULL, inode, 0, 0, &err))) { - if (err) - ext3_error(inode->i_sb, __func__, - "error %d reading directory #%lu offset 0", - err, inode->i_ino); - else - ext3_warning(inode->i_sb, __func__, - "bad directory (dir #%lu) - no data block", - inode->i_ino); - return 1; - } - de = (struct ext3_dir_entry_2 *) bh->b_data; - de1 = ext3_next_entry(de); - if (le32_to_cpu(de->inode) != inode->i_ino || - !le32_to_cpu(de1->inode) || - strcmp (".", de->name) || - strcmp ("..", de1->name)) { - ext3_warning (inode->i_sb, "empty_dir", - "bad directory (dir #%lu) - no `.' or `..'", - inode->i_ino); - brelse (bh); - return 1; - } - offset = ext3_rec_len_from_disk(de->rec_len) + - ext3_rec_len_from_disk(de1->rec_len); - de = ext3_next_entry(de1); - while (offset < inode->i_size ) { - if (!bh || - (void *) de >= (void *) (bh->b_data+sb->s_blocksize)) { - err = 0; - brelse (bh); - if (!(bh = ext3_dir_bread (NULL, inode, - offset >> EXT3_BLOCK_SIZE_BITS(sb), 0, &err))) { - if (err) - ext3_error(sb, __func__, - "error %d reading directory" - " #%lu offset %lu", - err, inode->i_ino, offset); - offset += sb->s_blocksize; - continue; - } - de = (struct ext3_dir_entry_2 *) bh->b_data; - } - if (!ext3_check_dir_entry("empty_dir", inode, de, bh, offset)) { - de = (struct ext3_dir_entry_2 *)(bh->b_data + - sb->s_blocksize); - offset = (offset | (sb->s_blocksize - 1)) + 1; - continue; - } - if (le32_to_cpu(de->inode)) { - brelse (bh); - return 0; - } - offset += ext3_rec_len_from_disk(de->rec_len); - de = ext3_next_entry(de); - } - brelse (bh); - return 1; -} - -/* ext3_orphan_add() links an unlinked or truncated inode into a list of - * such inodes, starting at the superblock, in case we crash before the - * file is closed/deleted, or in case the inode truncate spans multiple - * transactions and the last transaction is not recovered after a crash. - * - * At filesystem recovery time, we walk this list deleting unlinked - * inodes and truncating linked inodes in ext3_orphan_cleanup(). - */ -int ext3_orphan_add(handle_t *handle, struct inode *inode) -{ - struct super_block *sb = inode->i_sb; - struct ext3_iloc iloc; - int err = 0, rc; - - mutex_lock(&EXT3_SB(sb)->s_orphan_lock); - if (!list_empty(&EXT3_I(inode)->i_orphan)) - goto out_unlock; - - /* Orphan handling is only valid for files with data blocks - * being truncated, or files being unlinked. */ - - /* @@@ FIXME: Observation from aviro: - * I think I can trigger J_ASSERT in ext3_orphan_add(). We block - * here (on s_orphan_lock), so race with ext3_link() which might bump - * ->i_nlink. For, say it, character device. Not a regular file, - * not a directory, not a symlink and ->i_nlink > 0. - * - * tytso, 4/25/2009: I'm not sure how that could happen; - * shouldn't the fs core protect us from these sort of - * unlink()/link() races? - */ - J_ASSERT ((S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) || - S_ISLNK(inode->i_mode)) || inode->i_nlink == 0); - - BUFFER_TRACE(EXT3_SB(sb)->s_sbh, "get_write_access"); - err = ext3_journal_get_write_access(handle, EXT3_SB(sb)->s_sbh); - if (err) - goto out_unlock; - - err = ext3_reserve_inode_write(handle, inode, &iloc); - if (err) - goto out_unlock; - - /* Insert this inode at the head of the on-disk orphan list... */ - NEXT_ORPHAN(inode) = le32_to_cpu(EXT3_SB(sb)->s_es->s_last_orphan); - EXT3_SB(sb)->s_es->s_last_orphan = cpu_to_le32(inode->i_ino); - err = ext3_journal_dirty_metadata(handle, EXT3_SB(sb)->s_sbh); - rc = ext3_mark_iloc_dirty(handle, inode, &iloc); - if (!err) - err = rc; - - /* Only add to the head of the in-memory list if all the - * previous operations succeeded. If the orphan_add is going to - * fail (possibly taking the journal offline), we can't risk - * leaving the inode on the orphan list: stray orphan-list - * entries can cause panics at unmount time. - * - * This is safe: on error we're going to ignore the orphan list - * anyway on the next recovery. */ - if (!err) - list_add(&EXT3_I(inode)->i_orphan, &EXT3_SB(sb)->s_orphan); - - jbd_debug(4, "superblock will point to %lu\n", inode->i_ino); - jbd_debug(4, "orphan inode %lu will point to %d\n", - inode->i_ino, NEXT_ORPHAN(inode)); -out_unlock: - mutex_unlock(&EXT3_SB(sb)->s_orphan_lock); - ext3_std_error(inode->i_sb, err); - return err; -} - -/* - * ext3_orphan_del() removes an unlinked or truncated inode from the list - * of such inodes stored on disk, because it is finally being cleaned up. - */ -int ext3_orphan_del(handle_t *handle, struct inode *inode) -{ - struct list_head *prev; - struct ext3_inode_info *ei = EXT3_I(inode); - struct ext3_sb_info *sbi; - unsigned long ino_next; - struct ext3_iloc iloc; - int err = 0; - - mutex_lock(&EXT3_SB(inode->i_sb)->s_orphan_lock); - if (list_empty(&ei->i_orphan)) - goto out; - - ino_next = NEXT_ORPHAN(inode); - prev = ei->i_orphan.prev; - sbi = EXT3_SB(inode->i_sb); - - jbd_debug(4, "remove inode %lu from orphan list\n", inode->i_ino); - - list_del_init(&ei->i_orphan); - - /* If we're on an error path, we may not have a valid - * transaction handle with which to update the orphan list on - * disk, but we still need to remove the inode from the linked - * list in memory. */ - if (!handle) - goto out; - - err = ext3_reserve_inode_write(handle, inode, &iloc); - if (err) - goto out_err; - - if (prev == &sbi->s_orphan) { - jbd_debug(4, "superblock will point to %lu\n", ino_next); - BUFFER_TRACE(sbi->s_sbh, "get_write_access"); - err = ext3_journal_get_write_access(handle, sbi->s_sbh); - if (err) - goto out_brelse; - sbi->s_es->s_last_orphan = cpu_to_le32(ino_next); - err = ext3_journal_dirty_metadata(handle, sbi->s_sbh); - } else { - struct ext3_iloc iloc2; - struct inode *i_prev = - &list_entry(prev, struct ext3_inode_info, i_orphan)->vfs_inode; - - jbd_debug(4, "orphan inode %lu will point to %lu\n", - i_prev->i_ino, ino_next); - err = ext3_reserve_inode_write(handle, i_prev, &iloc2); - if (err) - goto out_brelse; - NEXT_ORPHAN(i_prev) = ino_next; - err = ext3_mark_iloc_dirty(handle, i_prev, &iloc2); - } - if (err) - goto out_brelse; - NEXT_ORPHAN(inode) = 0; - err = ext3_mark_iloc_dirty(handle, inode, &iloc); - -out_err: - ext3_std_error(inode->i_sb, err); -out: - mutex_unlock(&EXT3_SB(inode->i_sb)->s_orphan_lock); - return err; - -out_brelse: - brelse(iloc.bh); - goto out_err; -} - -static int ext3_rmdir (struct inode * dir, struct dentry *dentry) -{ - int retval; - struct inode * inode; - struct buffer_head * bh; - struct ext3_dir_entry_2 * de; - handle_t *handle; - - /* Initialize quotas before so that eventual writes go in - * separate transaction */ - dquot_initialize(dir); - dquot_initialize(d_inode(dentry)); - - handle = ext3_journal_start(dir, EXT3_DELETE_TRANS_BLOCKS(dir->i_sb)); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - retval = -ENOENT; - bh = ext3_find_entry(dir, &dentry->d_name, &de); - if (!bh) - goto end_rmdir; - - if (IS_DIRSYNC(dir)) - handle->h_sync = 1; - - inode = d_inode(dentry); - - retval = -EIO; - if (le32_to_cpu(de->inode) != inode->i_ino) - goto end_rmdir; - - retval = -ENOTEMPTY; - if (!empty_dir (inode)) - goto end_rmdir; - - retval = ext3_delete_entry(handle, dir, de, bh); - if (retval) - goto end_rmdir; - if (inode->i_nlink != 2) - ext3_warning (inode->i_sb, "ext3_rmdir", - "empty directory has nlink!=2 (%d)", - inode->i_nlink); - inode->i_version++; - clear_nlink(inode); - /* There's no need to set i_disksize: the fact that i_nlink is - * zero will ensure that the right thing happens during any - * recovery. */ - inode->i_size = 0; - ext3_orphan_add(handle, inode); - inode->i_ctime = dir->i_ctime = dir->i_mtime = CURRENT_TIME_SEC; - ext3_mark_inode_dirty(handle, inode); - drop_nlink(dir); - ext3_update_dx_flag(dir); - ext3_mark_inode_dirty(handle, dir); - -end_rmdir: - ext3_journal_stop(handle); - brelse (bh); - return retval; -} - -static int ext3_unlink(struct inode * dir, struct dentry *dentry) -{ - int retval; - struct inode * inode; - struct buffer_head * bh; - struct ext3_dir_entry_2 * de; - handle_t *handle; - - trace_ext3_unlink_enter(dir, dentry); - /* Initialize quotas before so that eventual writes go - * in separate transaction */ - dquot_initialize(dir); - dquot_initialize(d_inode(dentry)); - - handle = ext3_journal_start(dir, EXT3_DELETE_TRANS_BLOCKS(dir->i_sb)); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - if (IS_DIRSYNC(dir)) - handle->h_sync = 1; - - retval = -ENOENT; - bh = ext3_find_entry(dir, &dentry->d_name, &de); - if (!bh) - goto end_unlink; - - inode = d_inode(dentry); - - retval = -EIO; - if (le32_to_cpu(de->inode) != inode->i_ino) - goto end_unlink; - - if (!inode->i_nlink) { - ext3_warning (inode->i_sb, "ext3_unlink", - "Deleting nonexistent file (%lu), %d", - inode->i_ino, inode->i_nlink); - set_nlink(inode, 1); - } - retval = ext3_delete_entry(handle, dir, de, bh); - if (retval) - goto end_unlink; - dir->i_ctime = dir->i_mtime = CURRENT_TIME_SEC; - ext3_update_dx_flag(dir); - ext3_mark_inode_dirty(handle, dir); - drop_nlink(inode); - if (!inode->i_nlink) - ext3_orphan_add(handle, inode); - inode->i_ctime = dir->i_ctime; - ext3_mark_inode_dirty(handle, inode); - retval = 0; - -end_unlink: - ext3_journal_stop(handle); - brelse (bh); - trace_ext3_unlink_exit(dentry, retval); - return retval; -} - -static int ext3_symlink (struct inode * dir, - struct dentry *dentry, const char * symname) -{ - handle_t *handle; - struct inode * inode; - int l, err, retries = 0; - int credits; - - l = strlen(symname)+1; - if (l > dir->i_sb->s_blocksize) - return -ENAMETOOLONG; - - dquot_initialize(dir); - - if (l > EXT3_N_BLOCKS * 4) { - /* - * For non-fast symlinks, we just allocate inode and put it on - * orphan list in the first transaction => we need bitmap, - * group descriptor, sb, inode block, quota blocks, and - * possibly selinux xattr blocks. - */ - credits = 4 + EXT3_MAXQUOTAS_INIT_BLOCKS(dir->i_sb) + - EXT3_XATTR_TRANS_BLOCKS; - } else { - /* - * Fast symlink. We have to add entry to directory - * (EXT3_DATA_TRANS_BLOCKS + EXT3_INDEX_EXTRA_TRANS_BLOCKS), - * allocate new inode (bitmap, group descriptor, inode block, - * quota blocks, sb is already counted in previous macros). - */ - credits = EXT3_DATA_TRANS_BLOCKS(dir->i_sb) + - EXT3_INDEX_EXTRA_TRANS_BLOCKS + 3 + - EXT3_MAXQUOTAS_INIT_BLOCKS(dir->i_sb); - } -retry: - handle = ext3_journal_start(dir, credits); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - if (IS_DIRSYNC(dir)) - handle->h_sync = 1; - - inode = ext3_new_inode (handle, dir, &dentry->d_name, S_IFLNK|S_IRWXUGO); - err = PTR_ERR(inode); - if (IS_ERR(inode)) - goto out_stop; - - if (l > EXT3_N_BLOCKS * 4) { - inode->i_op = &ext3_symlink_inode_operations; - ext3_set_aops(inode); - /* - * We cannot call page_symlink() with transaction started - * because it calls into ext3_write_begin() which acquires page - * lock which ranks below transaction start (and it can also - * wait for journal commit if we are running out of space). So - * we have to stop transaction now and restart it when symlink - * contents is written. - * - * To keep fs consistent in case of crash, we have to put inode - * to orphan list in the mean time. - */ - drop_nlink(inode); - err = ext3_orphan_add(handle, inode); - ext3_journal_stop(handle); - if (err) - goto err_drop_inode; - err = __page_symlink(inode, symname, l, 1); - if (err) - goto err_drop_inode; - /* - * Now inode is being linked into dir (EXT3_DATA_TRANS_BLOCKS - * + EXT3_INDEX_EXTRA_TRANS_BLOCKS), inode is also modified - */ - handle = ext3_journal_start(dir, - EXT3_DATA_TRANS_BLOCKS(dir->i_sb) + - EXT3_INDEX_EXTRA_TRANS_BLOCKS + 1); - if (IS_ERR(handle)) { - err = PTR_ERR(handle); - goto err_drop_inode; - } - set_nlink(inode, 1); - err = ext3_orphan_del(handle, inode); - if (err) { - ext3_journal_stop(handle); - drop_nlink(inode); - goto err_drop_inode; - } - } else { - inode->i_op = &ext3_fast_symlink_inode_operations; - inode->i_link = (char*)&EXT3_I(inode)->i_data; - memcpy(inode->i_link, symname, l); - inode->i_size = l-1; - } - EXT3_I(inode)->i_disksize = inode->i_size; - err = ext3_add_nondir(handle, dentry, inode); -out_stop: - ext3_journal_stop(handle); - if (err == -ENOSPC && ext3_should_retry_alloc(dir->i_sb, &retries)) - goto retry; - return err; -err_drop_inode: - unlock_new_inode(inode); - iput(inode); - return err; -} - -static int ext3_link (struct dentry * old_dentry, - struct inode * dir, struct dentry *dentry) -{ - handle_t *handle; - struct inode *inode = d_inode(old_dentry); - int err, retries = 0; - - if (inode->i_nlink >= EXT3_LINK_MAX) - return -EMLINK; - - dquot_initialize(dir); - -retry: - handle = ext3_journal_start(dir, EXT3_DATA_TRANS_BLOCKS(dir->i_sb) + - EXT3_INDEX_EXTRA_TRANS_BLOCKS + 1); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - if (IS_DIRSYNC(dir)) - handle->h_sync = 1; - - inode->i_ctime = CURRENT_TIME_SEC; - inc_nlink(inode); - ihold(inode); - - err = ext3_add_entry(handle, dentry, inode); - if (!err) { - ext3_mark_inode_dirty(handle, inode); - /* this can happen only for tmpfile being - * linked the first time - */ - if (inode->i_nlink == 1) - ext3_orphan_del(handle, inode); - d_instantiate(dentry, inode); - } else { - drop_nlink(inode); - iput(inode); - } - ext3_journal_stop(handle); - if (err == -ENOSPC && ext3_should_retry_alloc(dir->i_sb, &retries)) - goto retry; - return err; -} - -#define PARENT_INO(buffer) \ - (ext3_next_entry((struct ext3_dir_entry_2 *)(buffer))->inode) - -/* - * Anybody can rename anything with this: the permission checks are left to the - * higher-level routines. - */ -static int ext3_rename (struct inode * old_dir, struct dentry *old_dentry, - struct inode * new_dir,struct dentry *new_dentry) -{ - handle_t *handle; - struct inode * old_inode, * new_inode; - struct buffer_head * old_bh, * new_bh, * dir_bh; - struct ext3_dir_entry_2 * old_de, * new_de; - int retval, flush_file = 0; - - dquot_initialize(old_dir); - dquot_initialize(new_dir); - - old_bh = new_bh = dir_bh = NULL; - - /* Initialize quotas before so that eventual writes go - * in separate transaction */ - if (d_really_is_positive(new_dentry)) - dquot_initialize(d_inode(new_dentry)); - handle = ext3_journal_start(old_dir, 2 * - EXT3_DATA_TRANS_BLOCKS(old_dir->i_sb) + - EXT3_INDEX_EXTRA_TRANS_BLOCKS + 2); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - if (IS_DIRSYNC(old_dir) || IS_DIRSYNC(new_dir)) - handle->h_sync = 1; - - old_bh = ext3_find_entry(old_dir, &old_dentry->d_name, &old_de); - /* - * Check for inode number is _not_ due to possible IO errors. - * We might rmdir the source, keep it as pwd of some process - * and merrily kill the link to whatever was created under the - * same name. Goodbye sticky bit ;-< - */ - old_inode = d_inode(old_dentry); - retval = -ENOENT; - if (!old_bh || le32_to_cpu(old_de->inode) != old_inode->i_ino) - goto end_rename; - - new_inode = d_inode(new_dentry); - new_bh = ext3_find_entry(new_dir, &new_dentry->d_name, &new_de); - if (new_bh) { - if (!new_inode) { - brelse (new_bh); - new_bh = NULL; - } - } - if (S_ISDIR(old_inode->i_mode)) { - if (new_inode) { - retval = -ENOTEMPTY; - if (!empty_dir (new_inode)) - goto end_rename; - } - retval = -EIO; - dir_bh = ext3_dir_bread(handle, old_inode, 0, 0, &retval); - if (!dir_bh) - goto end_rename; - if (le32_to_cpu(PARENT_INO(dir_bh->b_data)) != old_dir->i_ino) - goto end_rename; - retval = -EMLINK; - if (!new_inode && new_dir!=old_dir && - new_dir->i_nlink >= EXT3_LINK_MAX) - goto end_rename; - } - if (!new_bh) { - retval = ext3_add_entry (handle, new_dentry, old_inode); - if (retval) - goto end_rename; - } else { - BUFFER_TRACE(new_bh, "get write access"); - retval = ext3_journal_get_write_access(handle, new_bh); - if (retval) - goto journal_error; - new_de->inode = cpu_to_le32(old_inode->i_ino); - if (EXT3_HAS_INCOMPAT_FEATURE(new_dir->i_sb, - EXT3_FEATURE_INCOMPAT_FILETYPE)) - new_de->file_type = old_de->file_type; - new_dir->i_version++; - new_dir->i_ctime = new_dir->i_mtime = CURRENT_TIME_SEC; - ext3_mark_inode_dirty(handle, new_dir); - BUFFER_TRACE(new_bh, "call ext3_journal_dirty_metadata"); - retval = ext3_journal_dirty_metadata(handle, new_bh); - if (retval) - goto journal_error; - brelse(new_bh); - new_bh = NULL; - } - - /* - * Like most other Unix systems, set the ctime for inodes on a - * rename. - */ - old_inode->i_ctime = CURRENT_TIME_SEC; - ext3_mark_inode_dirty(handle, old_inode); - - /* - * ok, that's it - */ - if (le32_to_cpu(old_de->inode) != old_inode->i_ino || - old_de->name_len != old_dentry->d_name.len || - strncmp(old_de->name, old_dentry->d_name.name, old_de->name_len) || - (retval = ext3_delete_entry(handle, old_dir, - old_de, old_bh)) == -ENOENT) { - /* old_de could have moved from under us during htree split, so - * make sure that we are deleting the right entry. We might - * also be pointing to a stale entry in the unused part of - * old_bh so just checking inum and the name isn't enough. */ - struct buffer_head *old_bh2; - struct ext3_dir_entry_2 *old_de2; - - old_bh2 = ext3_find_entry(old_dir, &old_dentry->d_name, - &old_de2); - if (old_bh2) { - retval = ext3_delete_entry(handle, old_dir, - old_de2, old_bh2); - brelse(old_bh2); - } - } - if (retval) { - ext3_warning(old_dir->i_sb, "ext3_rename", - "Deleting old file (%lu), %d, error=%d", - old_dir->i_ino, old_dir->i_nlink, retval); - } - - if (new_inode) { - drop_nlink(new_inode); - new_inode->i_ctime = CURRENT_TIME_SEC; - } - old_dir->i_ctime = old_dir->i_mtime = CURRENT_TIME_SEC; - ext3_update_dx_flag(old_dir); - if (dir_bh) { - BUFFER_TRACE(dir_bh, "get_write_access"); - retval = ext3_journal_get_write_access(handle, dir_bh); - if (retval) - goto journal_error; - PARENT_INO(dir_bh->b_data) = cpu_to_le32(new_dir->i_ino); - BUFFER_TRACE(dir_bh, "call ext3_journal_dirty_metadata"); - retval = ext3_journal_dirty_metadata(handle, dir_bh); - if (retval) { -journal_error: - ext3_std_error(new_dir->i_sb, retval); - goto end_rename; - } - drop_nlink(old_dir); - if (new_inode) { - drop_nlink(new_inode); - } else { - inc_nlink(new_dir); - ext3_update_dx_flag(new_dir); - ext3_mark_inode_dirty(handle, new_dir); - } - } - ext3_mark_inode_dirty(handle, old_dir); - if (new_inode) { - ext3_mark_inode_dirty(handle, new_inode); - if (!new_inode->i_nlink) - ext3_orphan_add(handle, new_inode); - if (ext3_should_writeback_data(new_inode)) - flush_file = 1; - } - retval = 0; - -end_rename: - brelse (dir_bh); - brelse (old_bh); - brelse (new_bh); - ext3_journal_stop(handle); - if (retval == 0 && flush_file) - filemap_flush(old_inode->i_mapping); - return retval; -} - -/* - * directories can handle most operations... - */ -const struct inode_operations ext3_dir_inode_operations = { - .create = ext3_create, - .lookup = ext3_lookup, - .link = ext3_link, - .unlink = ext3_unlink, - .symlink = ext3_symlink, - .mkdir = ext3_mkdir, - .rmdir = ext3_rmdir, - .mknod = ext3_mknod, - .tmpfile = ext3_tmpfile, - .rename = ext3_rename, - .setattr = ext3_setattr, -#ifdef CONFIG_EXT3_FS_XATTR - .setxattr = generic_setxattr, - .getxattr = generic_getxattr, - .listxattr = ext3_listxattr, - .removexattr = generic_removexattr, -#endif - .get_acl = ext3_get_acl, - .set_acl = ext3_set_acl, -}; - -const struct inode_operations ext3_special_inode_operations = { - .setattr = ext3_setattr, -#ifdef CONFIG_EXT3_FS_XATTR - .setxattr = generic_setxattr, - .getxattr = generic_getxattr, - .listxattr = ext3_listxattr, - .removexattr = generic_removexattr, -#endif - .get_acl = ext3_get_acl, - .set_acl = ext3_set_acl, -}; diff --git a/fs/ext3/namei.h b/fs/ext3/namei.h deleted file mode 100644 index 46304d8c9f0a05..00000000000000 --- a/fs/ext3/namei.h +++ /dev/null @@ -1,27 +0,0 @@ -/* linux/fs/ext3/namei.h - * - * Copyright (C) 2005 Simtec Electronics - * Ben Dooks - * -*/ - -extern struct dentry *ext3_get_parent(struct dentry *child); - -static inline struct buffer_head *ext3_dir_bread(handle_t *handle, - struct inode *inode, - int block, int create, - int *err) -{ - struct buffer_head *bh; - - bh = ext3_bread(handle, inode, block, create, err); - - if (!bh && !(*err)) { - *err = -EIO; - ext3_error(inode->i_sb, __func__, - "Directory hole detected on inode %lu\n", - inode->i_ino); - return NULL; - } - return bh; -} diff --git a/fs/ext3/resize.c b/fs/ext3/resize.c deleted file mode 100644 index 27105655502cf9..00000000000000 --- a/fs/ext3/resize.c +++ /dev/null @@ -1,1117 +0,0 @@ -/* - * linux/fs/ext3/resize.c - * - * Support for resizing an ext3 filesystem while it is mounted. - * - * Copyright (C) 2001, 2002 Andreas Dilger - * - * This could probably be made into a module, because it is not often in use. - */ - - -#define EXT3FS_DEBUG - -#include "ext3.h" - - -#define outside(b, first, last) ((b) < (first) || (b) >= (last)) -#define inside(b, first, last) ((b) >= (first) && (b) < (last)) - -static int verify_group_input(struct super_block *sb, - struct ext3_new_group_data *input) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - struct ext3_super_block *es = sbi->s_es; - ext3_fsblk_t start = le32_to_cpu(es->s_blocks_count); - ext3_fsblk_t end = start + input->blocks_count; - unsigned group = input->group; - ext3_fsblk_t itend = input->inode_table + sbi->s_itb_per_group; - unsigned overhead = ext3_bg_has_super(sb, group) ? - (1 + ext3_bg_num_gdb(sb, group) + - le16_to_cpu(es->s_reserved_gdt_blocks)) : 0; - ext3_fsblk_t metaend = start + overhead; - struct buffer_head *bh = NULL; - ext3_grpblk_t free_blocks_count; - int err = -EINVAL; - - input->free_blocks_count = free_blocks_count = - input->blocks_count - 2 - overhead - sbi->s_itb_per_group; - - if (test_opt(sb, DEBUG)) - printk(KERN_DEBUG "EXT3-fs: adding %s group %u: %u blocks " - "(%d free, %u reserved)\n", - ext3_bg_has_super(sb, input->group) ? "normal" : - "no-super", input->group, input->blocks_count, - free_blocks_count, input->reserved_blocks); - - if (group != sbi->s_groups_count) - ext3_warning(sb, __func__, - "Cannot add at group %u (only %lu groups)", - input->group, sbi->s_groups_count); - else if ((start - le32_to_cpu(es->s_first_data_block)) % - EXT3_BLOCKS_PER_GROUP(sb)) - ext3_warning(sb, __func__, "Last group not full"); - else if (input->reserved_blocks > input->blocks_count / 5) - ext3_warning(sb, __func__, "Reserved blocks too high (%u)", - input->reserved_blocks); - else if (free_blocks_count < 0) - ext3_warning(sb, __func__, "Bad blocks count %u", - input->blocks_count); - else if (!(bh = sb_bread(sb, end - 1))) - ext3_warning(sb, __func__, - "Cannot read last block ("E3FSBLK")", - end - 1); - else if (outside(input->block_bitmap, start, end)) - ext3_warning(sb, __func__, - "Block bitmap not in group (block %u)", - input->block_bitmap); - else if (outside(input->inode_bitmap, start, end)) - ext3_warning(sb, __func__, - "Inode bitmap not in group (block %u)", - input->inode_bitmap); - else if (outside(input->inode_table, start, end) || - outside(itend - 1, start, end)) - ext3_warning(sb, __func__, - "Inode table not in group (blocks %u-"E3FSBLK")", - input->inode_table, itend - 1); - else if (input->inode_bitmap == input->block_bitmap) - ext3_warning(sb, __func__, - "Block bitmap same as inode bitmap (%u)", - input->block_bitmap); - else if (inside(input->block_bitmap, input->inode_table, itend)) - ext3_warning(sb, __func__, - "Block bitmap (%u) in inode table (%u-"E3FSBLK")", - input->block_bitmap, input->inode_table, itend-1); - else if (inside(input->inode_bitmap, input->inode_table, itend)) - ext3_warning(sb, __func__, - "Inode bitmap (%u) in inode table (%u-"E3FSBLK")", - input->inode_bitmap, input->inode_table, itend-1); - else if (inside(input->block_bitmap, start, metaend)) - ext3_warning(sb, __func__, - "Block bitmap (%u) in GDT table" - " ("E3FSBLK"-"E3FSBLK")", - input->block_bitmap, start, metaend - 1); - else if (inside(input->inode_bitmap, start, metaend)) - ext3_warning(sb, __func__, - "Inode bitmap (%u) in GDT table" - " ("E3FSBLK"-"E3FSBLK")", - input->inode_bitmap, start, metaend - 1); - else if (inside(input->inode_table, start, metaend) || - inside(itend - 1, start, metaend)) - ext3_warning(sb, __func__, - "Inode table (%u-"E3FSBLK") overlaps" - "GDT table ("E3FSBLK"-"E3FSBLK")", - input->inode_table, itend - 1, start, metaend - 1); - else - err = 0; - brelse(bh); - - return err; -} - -static struct buffer_head *bclean(handle_t *handle, struct super_block *sb, - ext3_fsblk_t blk) -{ - struct buffer_head *bh; - int err; - - bh = sb_getblk(sb, blk); - if (unlikely(!bh)) - return ERR_PTR(-ENOMEM); - if ((err = ext3_journal_get_write_access(handle, bh))) { - brelse(bh); - bh = ERR_PTR(err); - } else { - lock_buffer(bh); - memset(bh->b_data, 0, sb->s_blocksize); - set_buffer_uptodate(bh); - unlock_buffer(bh); - } - - return bh; -} - -/* - * To avoid calling the atomic setbit hundreds or thousands of times, we only - * need to use it within a single byte (to ensure we get endianness right). - * We can use memset for the rest of the bitmap as there are no other users. - */ -static void mark_bitmap_end(int start_bit, int end_bit, char *bitmap) -{ - int i; - - if (start_bit >= end_bit) - return; - - ext3_debug("mark end bits +%d through +%d used\n", start_bit, end_bit); - for (i = start_bit; i < ((start_bit + 7) & ~7UL); i++) - ext3_set_bit(i, bitmap); - if (i < end_bit) - memset(bitmap + (i >> 3), 0xff, (end_bit - i) >> 3); -} - -/* - * If we have fewer than thresh credits, extend by EXT3_MAX_TRANS_DATA. - * If that fails, restart the transaction & regain write access for the - * buffer head which is used for block_bitmap modifications. - */ -static int extend_or_restart_transaction(handle_t *handle, int thresh, - struct buffer_head *bh) -{ - int err; - - if (handle->h_buffer_credits >= thresh) - return 0; - - err = ext3_journal_extend(handle, EXT3_MAX_TRANS_DATA); - if (err < 0) - return err; - if (err) { - err = ext3_journal_restart(handle, EXT3_MAX_TRANS_DATA); - if (err) - return err; - err = ext3_journal_get_write_access(handle, bh); - if (err) - return err; - } - - return 0; -} - -/* - * Set up the block and inode bitmaps, and the inode table for the new group. - * This doesn't need to be part of the main transaction, since we are only - * changing blocks outside the actual filesystem. We still do journaling to - * ensure the recovery is correct in case of a failure just after resize. - * If any part of this fails, we simply abort the resize. - */ -static int setup_new_group_blocks(struct super_block *sb, - struct ext3_new_group_data *input) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - ext3_fsblk_t start = ext3_group_first_block_no(sb, input->group); - int reserved_gdb = ext3_bg_has_super(sb, input->group) ? - le16_to_cpu(sbi->s_es->s_reserved_gdt_blocks) : 0; - unsigned long gdblocks = ext3_bg_num_gdb(sb, input->group); - struct buffer_head *bh; - handle_t *handle; - ext3_fsblk_t block; - ext3_grpblk_t bit; - int i; - int err = 0, err2; - - /* This transaction may be extended/restarted along the way */ - handle = ext3_journal_start_sb(sb, EXT3_MAX_TRANS_DATA); - - if (IS_ERR(handle)) - return PTR_ERR(handle); - - mutex_lock(&sbi->s_resize_lock); - if (input->group != sbi->s_groups_count) { - err = -EBUSY; - goto exit_journal; - } - - if (IS_ERR(bh = bclean(handle, sb, input->block_bitmap))) { - err = PTR_ERR(bh); - goto exit_journal; - } - - if (ext3_bg_has_super(sb, input->group)) { - ext3_debug("mark backup superblock %#04lx (+0)\n", start); - ext3_set_bit(0, bh->b_data); - } - - /* Copy all of the GDT blocks into the backup in this group */ - for (i = 0, bit = 1, block = start + 1; - i < gdblocks; i++, block++, bit++) { - struct buffer_head *gdb; - - ext3_debug("update backup group %#04lx (+%d)\n", block, bit); - - err = extend_or_restart_transaction(handle, 1, bh); - if (err) - goto exit_bh; - - gdb = sb_getblk(sb, block); - if (unlikely(!gdb)) { - err = -ENOMEM; - goto exit_bh; - } - if ((err = ext3_journal_get_write_access(handle, gdb))) { - brelse(gdb); - goto exit_bh; - } - lock_buffer(gdb); - memcpy(gdb->b_data, sbi->s_group_desc[i]->b_data, gdb->b_size); - set_buffer_uptodate(gdb); - unlock_buffer(gdb); - err = ext3_journal_dirty_metadata(handle, gdb); - if (err) { - brelse(gdb); - goto exit_bh; - } - ext3_set_bit(bit, bh->b_data); - brelse(gdb); - } - - /* Zero out all of the reserved backup group descriptor table blocks */ - for (i = 0, bit = gdblocks + 1, block = start + bit; - i < reserved_gdb; i++, block++, bit++) { - struct buffer_head *gdb; - - ext3_debug("clear reserved block %#04lx (+%d)\n", block, bit); - - err = extend_or_restart_transaction(handle, 1, bh); - if (err) - goto exit_bh; - - if (IS_ERR(gdb = bclean(handle, sb, block))) { - err = PTR_ERR(gdb); - goto exit_bh; - } - err = ext3_journal_dirty_metadata(handle, gdb); - if (err) { - brelse(gdb); - goto exit_bh; - } - ext3_set_bit(bit, bh->b_data); - brelse(gdb); - } - ext3_debug("mark block bitmap %#04x (+%ld)\n", input->block_bitmap, - input->block_bitmap - start); - ext3_set_bit(input->block_bitmap - start, bh->b_data); - ext3_debug("mark inode bitmap %#04x (+%ld)\n", input->inode_bitmap, - input->inode_bitmap - start); - ext3_set_bit(input->inode_bitmap - start, bh->b_data); - - /* Zero out all of the inode table blocks */ - for (i = 0, block = input->inode_table, bit = block - start; - i < sbi->s_itb_per_group; i++, bit++, block++) { - struct buffer_head *it; - - ext3_debug("clear inode block %#04lx (+%d)\n", block, bit); - - err = extend_or_restart_transaction(handle, 1, bh); - if (err) - goto exit_bh; - - if (IS_ERR(it = bclean(handle, sb, block))) { - err = PTR_ERR(it); - goto exit_bh; - } - err = ext3_journal_dirty_metadata(handle, it); - if (err) { - brelse(it); - goto exit_bh; - } - brelse(it); - ext3_set_bit(bit, bh->b_data); - } - - err = extend_or_restart_transaction(handle, 2, bh); - if (err) - goto exit_bh; - - mark_bitmap_end(input->blocks_count, EXT3_BLOCKS_PER_GROUP(sb), - bh->b_data); - err = ext3_journal_dirty_metadata(handle, bh); - if (err) - goto exit_bh; - brelse(bh); - - /* Mark unused entries in inode bitmap used */ - ext3_debug("clear inode bitmap %#04x (+%ld)\n", - input->inode_bitmap, input->inode_bitmap - start); - if (IS_ERR(bh = bclean(handle, sb, input->inode_bitmap))) { - err = PTR_ERR(bh); - goto exit_journal; - } - - mark_bitmap_end(EXT3_INODES_PER_GROUP(sb), EXT3_BLOCKS_PER_GROUP(sb), - bh->b_data); - err = ext3_journal_dirty_metadata(handle, bh); -exit_bh: - brelse(bh); - -exit_journal: - mutex_unlock(&sbi->s_resize_lock); - if ((err2 = ext3_journal_stop(handle)) && !err) - err = err2; - - return err; -} - -/* - * Iterate through the groups which hold BACKUP superblock/GDT copies in an - * ext3 filesystem. The counters should be initialized to 1, 5, and 7 before - * calling this for the first time. In a sparse filesystem it will be the - * sequence of powers of 3, 5, and 7: 1, 3, 5, 7, 9, 25, 27, 49, 81, ... - * For a non-sparse filesystem it will be every group: 1, 2, 3, 4, ... - */ -static unsigned ext3_list_backups(struct super_block *sb, unsigned *three, - unsigned *five, unsigned *seven) -{ - unsigned *min = three; - int mult = 3; - unsigned ret; - - if (!EXT3_HAS_RO_COMPAT_FEATURE(sb, - EXT3_FEATURE_RO_COMPAT_SPARSE_SUPER)) { - ret = *min; - *min += 1; - return ret; - } - - if (*five < *min) { - min = five; - mult = 5; - } - if (*seven < *min) { - min = seven; - mult = 7; - } - - ret = *min; - *min *= mult; - - return ret; -} - -/* - * Check that all of the backup GDT blocks are held in the primary GDT block. - * It is assumed that they are stored in group order. Returns the number of - * groups in current filesystem that have BACKUPS, or -ve error code. - */ -static int verify_reserved_gdb(struct super_block *sb, - struct buffer_head *primary) -{ - const ext3_fsblk_t blk = primary->b_blocknr; - const unsigned long end = EXT3_SB(sb)->s_groups_count; - unsigned three = 1; - unsigned five = 5; - unsigned seven = 7; - unsigned grp; - __le32 *p = (__le32 *)primary->b_data; - int gdbackups = 0; - - while ((grp = ext3_list_backups(sb, &three, &five, &seven)) < end) { - if (le32_to_cpu(*p++) != grp * EXT3_BLOCKS_PER_GROUP(sb) + blk){ - ext3_warning(sb, __func__, - "reserved GDT "E3FSBLK - " missing grp %d ("E3FSBLK")", - blk, grp, - grp * EXT3_BLOCKS_PER_GROUP(sb) + blk); - return -EINVAL; - } - if (++gdbackups > EXT3_ADDR_PER_BLOCK(sb)) - return -EFBIG; - } - - return gdbackups; -} - -/* - * Called when we need to bring a reserved group descriptor table block into - * use from the resize inode. The primary copy of the new GDT block currently - * is an indirect block (under the double indirect block in the resize inode). - * The new backup GDT blocks will be stored as leaf blocks in this indirect - * block, in group order. Even though we know all the block numbers we need, - * we check to ensure that the resize inode has actually reserved these blocks. - * - * Don't need to update the block bitmaps because the blocks are still in use. - * - * We get all of the error cases out of the way, so that we are sure to not - * fail once we start modifying the data on disk, because JBD has no rollback. - */ -static int add_new_gdb(handle_t *handle, struct inode *inode, - struct ext3_new_group_data *input, - struct buffer_head **primary) -{ - struct super_block *sb = inode->i_sb; - struct ext3_super_block *es = EXT3_SB(sb)->s_es; - unsigned long gdb_num = input->group / EXT3_DESC_PER_BLOCK(sb); - ext3_fsblk_t gdblock = EXT3_SB(sb)->s_sbh->b_blocknr + 1 + gdb_num; - struct buffer_head **o_group_desc, **n_group_desc; - struct buffer_head *dind; - int gdbackups; - struct ext3_iloc iloc; - __le32 *data; - int err; - - if (test_opt(sb, DEBUG)) - printk(KERN_DEBUG - "EXT3-fs: ext3_add_new_gdb: adding group block %lu\n", - gdb_num); - - /* - * If we are not using the primary superblock/GDT copy don't resize, - * because the user tools have no way of handling this. Probably a - * bad time to do it anyways. - */ - if (EXT3_SB(sb)->s_sbh->b_blocknr != - le32_to_cpu(EXT3_SB(sb)->s_es->s_first_data_block)) { - ext3_warning(sb, __func__, - "won't resize using backup superblock at %llu", - (unsigned long long)EXT3_SB(sb)->s_sbh->b_blocknr); - return -EPERM; - } - - *primary = sb_bread(sb, gdblock); - if (!*primary) - return -EIO; - - if ((gdbackups = verify_reserved_gdb(sb, *primary)) < 0) { - err = gdbackups; - goto exit_bh; - } - - data = EXT3_I(inode)->i_data + EXT3_DIND_BLOCK; - dind = sb_bread(sb, le32_to_cpu(*data)); - if (!dind) { - err = -EIO; - goto exit_bh; - } - - data = (__le32 *)dind->b_data; - if (le32_to_cpu(data[gdb_num % EXT3_ADDR_PER_BLOCK(sb)]) != gdblock) { - ext3_warning(sb, __func__, - "new group %u GDT block "E3FSBLK" not reserved", - input->group, gdblock); - err = -EINVAL; - goto exit_dind; - } - - if ((err = ext3_journal_get_write_access(handle, EXT3_SB(sb)->s_sbh))) - goto exit_dind; - - if ((err = ext3_journal_get_write_access(handle, *primary))) - goto exit_sbh; - - if ((err = ext3_journal_get_write_access(handle, dind))) - goto exit_primary; - - /* ext3_reserve_inode_write() gets a reference on the iloc */ - if ((err = ext3_reserve_inode_write(handle, inode, &iloc))) - goto exit_dindj; - - n_group_desc = kmalloc((gdb_num + 1) * sizeof(struct buffer_head *), - GFP_NOFS); - if (!n_group_desc) { - err = -ENOMEM; - ext3_warning (sb, __func__, - "not enough memory for %lu groups", gdb_num + 1); - goto exit_inode; - } - - /* - * Finally, we have all of the possible failures behind us... - * - * Remove new GDT block from inode double-indirect block and clear out - * the new GDT block for use (which also "frees" the backup GDT blocks - * from the reserved inode). We don't need to change the bitmaps for - * these blocks, because they are marked as in-use from being in the - * reserved inode, and will become GDT blocks (primary and backup). - */ - data[gdb_num % EXT3_ADDR_PER_BLOCK(sb)] = 0; - err = ext3_journal_dirty_metadata(handle, dind); - if (err) - goto exit_group_desc; - brelse(dind); - dind = NULL; - inode->i_blocks -= (gdbackups + 1) * sb->s_blocksize >> 9; - err = ext3_mark_iloc_dirty(handle, inode, &iloc); - if (err) - goto exit_group_desc; - memset((*primary)->b_data, 0, sb->s_blocksize); - err = ext3_journal_dirty_metadata(handle, *primary); - if (err) - goto exit_group_desc; - - o_group_desc = EXT3_SB(sb)->s_group_desc; - memcpy(n_group_desc, o_group_desc, - EXT3_SB(sb)->s_gdb_count * sizeof(struct buffer_head *)); - n_group_desc[gdb_num] = *primary; - EXT3_SB(sb)->s_group_desc = n_group_desc; - EXT3_SB(sb)->s_gdb_count++; - kfree(o_group_desc); - - le16_add_cpu(&es->s_reserved_gdt_blocks, -1); - err = ext3_journal_dirty_metadata(handle, EXT3_SB(sb)->s_sbh); - if (err) - goto exit_inode; - - return 0; - -exit_group_desc: - kfree(n_group_desc); -exit_inode: - //ext3_journal_release_buffer(handle, iloc.bh); - brelse(iloc.bh); -exit_dindj: - //ext3_journal_release_buffer(handle, dind); -exit_primary: - //ext3_journal_release_buffer(handle, *primary); -exit_sbh: - //ext3_journal_release_buffer(handle, *primary); -exit_dind: - brelse(dind); -exit_bh: - brelse(*primary); - - ext3_debug("leaving with error %d\n", err); - return err; -} - -/* - * Called when we are adding a new group which has a backup copy of each of - * the GDT blocks (i.e. sparse group) and there are reserved GDT blocks. - * We need to add these reserved backup GDT blocks to the resize inode, so - * that they are kept for future resizing and not allocated to files. - * - * Each reserved backup GDT block will go into a different indirect block. - * The indirect blocks are actually the primary reserved GDT blocks, - * so we know in advance what their block numbers are. We only get the - * double-indirect block to verify it is pointing to the primary reserved - * GDT blocks so we don't overwrite a data block by accident. The reserved - * backup GDT blocks are stored in their reserved primary GDT block. - */ -static int reserve_backup_gdb(handle_t *handle, struct inode *inode, - struct ext3_new_group_data *input) -{ - struct super_block *sb = inode->i_sb; - int reserved_gdb =le16_to_cpu(EXT3_SB(sb)->s_es->s_reserved_gdt_blocks); - struct buffer_head **primary; - struct buffer_head *dind; - struct ext3_iloc iloc; - ext3_fsblk_t blk; - __le32 *data, *end; - int gdbackups = 0; - int res, i; - int err; - - primary = kmalloc(reserved_gdb * sizeof(*primary), GFP_NOFS); - if (!primary) - return -ENOMEM; - - data = EXT3_I(inode)->i_data + EXT3_DIND_BLOCK; - dind = sb_bread(sb, le32_to_cpu(*data)); - if (!dind) { - err = -EIO; - goto exit_free; - } - - blk = EXT3_SB(sb)->s_sbh->b_blocknr + 1 + EXT3_SB(sb)->s_gdb_count; - data = (__le32 *)dind->b_data + (EXT3_SB(sb)->s_gdb_count % - EXT3_ADDR_PER_BLOCK(sb)); - end = (__le32 *)dind->b_data + EXT3_ADDR_PER_BLOCK(sb); - - /* Get each reserved primary GDT block and verify it holds backups */ - for (res = 0; res < reserved_gdb; res++, blk++) { - if (le32_to_cpu(*data) != blk) { - ext3_warning(sb, __func__, - "reserved block "E3FSBLK - " not at offset %ld", - blk, - (long)(data - (__le32 *)dind->b_data)); - err = -EINVAL; - goto exit_bh; - } - primary[res] = sb_bread(sb, blk); - if (!primary[res]) { - err = -EIO; - goto exit_bh; - } - if ((gdbackups = verify_reserved_gdb(sb, primary[res])) < 0) { - brelse(primary[res]); - err = gdbackups; - goto exit_bh; - } - if (++data >= end) - data = (__le32 *)dind->b_data; - } - - for (i = 0; i < reserved_gdb; i++) { - if ((err = ext3_journal_get_write_access(handle, primary[i]))) { - /* - int j; - for (j = 0; j < i; j++) - ext3_journal_release_buffer(handle, primary[j]); - */ - goto exit_bh; - } - } - - if ((err = ext3_reserve_inode_write(handle, inode, &iloc))) - goto exit_bh; - - /* - * Finally we can add each of the reserved backup GDT blocks from - * the new group to its reserved primary GDT block. - */ - blk = input->group * EXT3_BLOCKS_PER_GROUP(sb); - for (i = 0; i < reserved_gdb; i++) { - int err2; - data = (__le32 *)primary[i]->b_data; - /* printk("reserving backup %lu[%u] = %lu\n", - primary[i]->b_blocknr, gdbackups, - blk + primary[i]->b_blocknr); */ - data[gdbackups] = cpu_to_le32(blk + primary[i]->b_blocknr); - err2 = ext3_journal_dirty_metadata(handle, primary[i]); - if (!err) - err = err2; - } - inode->i_blocks += reserved_gdb * sb->s_blocksize >> 9; - ext3_mark_iloc_dirty(handle, inode, &iloc); - -exit_bh: - while (--res >= 0) - brelse(primary[res]); - brelse(dind); - -exit_free: - kfree(primary); - - return err; -} - -/* - * Update the backup copies of the ext3 metadata. These don't need to be part - * of the main resize transaction, because e2fsck will re-write them if there - * is a problem (basically only OOM will cause a problem). However, we - * _should_ update the backups if possible, in case the primary gets trashed - * for some reason and we need to run e2fsck from a backup superblock. The - * important part is that the new block and inode counts are in the backup - * superblocks, and the location of the new group metadata in the GDT backups. - * - * We do not need take the s_resize_lock for this, because these - * blocks are not otherwise touched by the filesystem code when it is - * mounted. We don't need to worry about last changing from - * sbi->s_groups_count, because the worst that can happen is that we - * do not copy the full number of backups at this time. The resize - * which changed s_groups_count will backup again. - */ -static void update_backups(struct super_block *sb, - int blk_off, char *data, int size) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - const unsigned long last = sbi->s_groups_count; - const int bpg = EXT3_BLOCKS_PER_GROUP(sb); - unsigned three = 1; - unsigned five = 5; - unsigned seven = 7; - unsigned group; - int rest = sb->s_blocksize - size; - handle_t *handle; - int err = 0, err2; - - handle = ext3_journal_start_sb(sb, EXT3_MAX_TRANS_DATA); - if (IS_ERR(handle)) { - group = 1; - err = PTR_ERR(handle); - goto exit_err; - } - - while ((group = ext3_list_backups(sb, &three, &five, &seven)) < last) { - struct buffer_head *bh; - - /* Out of journal space, and can't get more - abort - so sad */ - if (handle->h_buffer_credits == 0 && - ext3_journal_extend(handle, EXT3_MAX_TRANS_DATA) && - (err = ext3_journal_restart(handle, EXT3_MAX_TRANS_DATA))) - break; - - bh = sb_getblk(sb, group * bpg + blk_off); - if (unlikely(!bh)) { - err = -ENOMEM; - break; - } - ext3_debug("update metadata backup %#04lx\n", - (unsigned long)bh->b_blocknr); - if ((err = ext3_journal_get_write_access(handle, bh))) { - brelse(bh); - break; - } - lock_buffer(bh); - memcpy(bh->b_data, data, size); - if (rest) - memset(bh->b_data + size, 0, rest); - set_buffer_uptodate(bh); - unlock_buffer(bh); - err = ext3_journal_dirty_metadata(handle, bh); - brelse(bh); - if (err) - break; - } - if ((err2 = ext3_journal_stop(handle)) && !err) - err = err2; - - /* - * Ugh! Need to have e2fsck write the backup copies. It is too - * late to revert the resize, we shouldn't fail just because of - * the backup copies (they are only needed in case of corruption). - * - * However, if we got here we have a journal problem too, so we - * can't really start a transaction to mark the superblock. - * Chicken out and just set the flag on the hope it will be written - * to disk, and if not - we will simply wait until next fsck. - */ -exit_err: - if (err) { - ext3_warning(sb, __func__, - "can't update backup for group %d (err %d), " - "forcing fsck on next reboot", group, err); - sbi->s_mount_state &= ~EXT3_VALID_FS; - sbi->s_es->s_state &= cpu_to_le16(~EXT3_VALID_FS); - mark_buffer_dirty(sbi->s_sbh); - } -} - -/* Add group descriptor data to an existing or new group descriptor block. - * Ensure we handle all possible error conditions _before_ we start modifying - * the filesystem, because we cannot abort the transaction and not have it - * write the data to disk. - * - * If we are on a GDT block boundary, we need to get the reserved GDT block. - * Otherwise, we may need to add backup GDT blocks for a sparse group. - * - * We only need to hold the superblock lock while we are actually adding - * in the new group's counts to the superblock. Prior to that we have - * not really "added" the group at all. We re-check that we are still - * adding in the last group in case things have changed since verifying. - */ -int ext3_group_add(struct super_block *sb, struct ext3_new_group_data *input) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - struct ext3_super_block *es = sbi->s_es; - int reserved_gdb = ext3_bg_has_super(sb, input->group) ? - le16_to_cpu(es->s_reserved_gdt_blocks) : 0; - struct buffer_head *primary = NULL; - struct ext3_group_desc *gdp; - struct inode *inode = NULL; - handle_t *handle; - int gdb_off, gdb_num; - int err, err2; - - gdb_num = input->group / EXT3_DESC_PER_BLOCK(sb); - gdb_off = input->group % EXT3_DESC_PER_BLOCK(sb); - - if (gdb_off == 0 && !EXT3_HAS_RO_COMPAT_FEATURE(sb, - EXT3_FEATURE_RO_COMPAT_SPARSE_SUPER)) { - ext3_warning(sb, __func__, - "Can't resize non-sparse filesystem further"); - return -EPERM; - } - - if (le32_to_cpu(es->s_blocks_count) + input->blocks_count < - le32_to_cpu(es->s_blocks_count)) { - ext3_warning(sb, __func__, "blocks_count overflow\n"); - return -EINVAL; - } - - if (le32_to_cpu(es->s_inodes_count) + EXT3_INODES_PER_GROUP(sb) < - le32_to_cpu(es->s_inodes_count)) { - ext3_warning(sb, __func__, "inodes_count overflow\n"); - return -EINVAL; - } - - if (reserved_gdb || gdb_off == 0) { - if (!EXT3_HAS_COMPAT_FEATURE(sb, - EXT3_FEATURE_COMPAT_RESIZE_INODE) - || !le16_to_cpu(es->s_reserved_gdt_blocks)) { - ext3_warning(sb, __func__, - "No reserved GDT blocks, can't resize"); - return -EPERM; - } - inode = ext3_iget(sb, EXT3_RESIZE_INO); - if (IS_ERR(inode)) { - ext3_warning(sb, __func__, - "Error opening resize inode"); - return PTR_ERR(inode); - } - } - - if ((err = verify_group_input(sb, input))) - goto exit_put; - - if ((err = setup_new_group_blocks(sb, input))) - goto exit_put; - - /* - * We will always be modifying at least the superblock and a GDT - * block. If we are adding a group past the last current GDT block, - * we will also modify the inode and the dindirect block. If we - * are adding a group with superblock/GDT backups we will also - * modify each of the reserved GDT dindirect blocks. - */ - handle = ext3_journal_start_sb(sb, - ext3_bg_has_super(sb, input->group) ? - 3 + reserved_gdb : 4); - if (IS_ERR(handle)) { - err = PTR_ERR(handle); - goto exit_put; - } - - mutex_lock(&sbi->s_resize_lock); - if (input->group != sbi->s_groups_count) { - ext3_warning(sb, __func__, - "multiple resizers run on filesystem!"); - err = -EBUSY; - goto exit_journal; - } - - if ((err = ext3_journal_get_write_access(handle, sbi->s_sbh))) - goto exit_journal; - - /* - * We will only either add reserved group blocks to a backup group - * or remove reserved blocks for the first group in a new group block. - * Doing both would be mean more complex code, and sane people don't - * use non-sparse filesystems anymore. This is already checked above. - */ - if (gdb_off) { - primary = sbi->s_group_desc[gdb_num]; - if ((err = ext3_journal_get_write_access(handle, primary))) - goto exit_journal; - - if (reserved_gdb && ext3_bg_num_gdb(sb, input->group) && - (err = reserve_backup_gdb(handle, inode, input))) - goto exit_journal; - } else if ((err = add_new_gdb(handle, inode, input, &primary))) - goto exit_journal; - - /* - * OK, now we've set up the new group. Time to make it active. - * - * We do not lock all allocations via s_resize_lock - * so we have to be safe wrt. concurrent accesses the group - * data. So we need to be careful to set all of the relevant - * group descriptor data etc. *before* we enable the group. - * - * The key field here is sbi->s_groups_count: as long as - * that retains its old value, nobody is going to access the new - * group. - * - * So first we update all the descriptor metadata for the new - * group; then we update the total disk blocks count; then we - * update the groups count to enable the group; then finally we - * update the free space counts so that the system can start - * using the new disk blocks. - */ - - /* Update group descriptor block for new group */ - gdp = (struct ext3_group_desc *)primary->b_data + gdb_off; - - gdp->bg_block_bitmap = cpu_to_le32(input->block_bitmap); - gdp->bg_inode_bitmap = cpu_to_le32(input->inode_bitmap); - gdp->bg_inode_table = cpu_to_le32(input->inode_table); - gdp->bg_free_blocks_count = cpu_to_le16(input->free_blocks_count); - gdp->bg_free_inodes_count = cpu_to_le16(EXT3_INODES_PER_GROUP(sb)); - - /* - * Make the new blocks and inodes valid next. We do this before - * increasing the group count so that once the group is enabled, - * all of its blocks and inodes are already valid. - * - * We always allocate group-by-group, then block-by-block or - * inode-by-inode within a group, so enabling these - * blocks/inodes before the group is live won't actually let us - * allocate the new space yet. - */ - le32_add_cpu(&es->s_blocks_count, input->blocks_count); - le32_add_cpu(&es->s_inodes_count, EXT3_INODES_PER_GROUP(sb)); - - /* - * We need to protect s_groups_count against other CPUs seeing - * inconsistent state in the superblock. - * - * The precise rules we use are: - * - * * Writers of s_groups_count *must* hold s_resize_lock - * AND - * * Writers must perform a smp_wmb() after updating all dependent - * data and before modifying the groups count - * - * * Readers must hold s_resize_lock over the access - * OR - * * Readers must perform an smp_rmb() after reading the groups count - * and before reading any dependent data. - * - * NB. These rules can be relaxed when checking the group count - * while freeing data, as we can only allocate from a block - * group after serialising against the group count, and we can - * only then free after serialising in turn against that - * allocation. - */ - smp_wmb(); - - /* Update the global fs size fields */ - sbi->s_groups_count++; - - err = ext3_journal_dirty_metadata(handle, primary); - if (err) - goto exit_journal; - - /* Update the reserved block counts only once the new group is - * active. */ - le32_add_cpu(&es->s_r_blocks_count, input->reserved_blocks); - - /* Update the free space counts */ - percpu_counter_add(&sbi->s_freeblocks_counter, - input->free_blocks_count); - percpu_counter_add(&sbi->s_freeinodes_counter, - EXT3_INODES_PER_GROUP(sb)); - - err = ext3_journal_dirty_metadata(handle, sbi->s_sbh); - -exit_journal: - mutex_unlock(&sbi->s_resize_lock); - if ((err2 = ext3_journal_stop(handle)) && !err) - err = err2; - if (!err) { - update_backups(sb, sbi->s_sbh->b_blocknr, (char *)es, - sizeof(struct ext3_super_block)); - update_backups(sb, primary->b_blocknr, primary->b_data, - primary->b_size); - } -exit_put: - iput(inode); - return err; -} /* ext3_group_add */ - -/* Extend the filesystem to the new number of blocks specified. This entry - * point is only used to extend the current filesystem to the end of the last - * existing group. It can be accessed via ioctl, or by "remount,resize=" - * for emergencies (because it has no dependencies on reserved blocks). - * - * If we _really_ wanted, we could use default values to call ext3_group_add() - * allow the "remount" trick to work for arbitrary resizing, assuming enough - * GDT blocks are reserved to grow to the desired size. - */ -int ext3_group_extend(struct super_block *sb, struct ext3_super_block *es, - ext3_fsblk_t n_blocks_count) -{ - ext3_fsblk_t o_blocks_count; - ext3_grpblk_t last; - ext3_grpblk_t add; - struct buffer_head * bh; - handle_t *handle; - int err; - unsigned long freed_blocks; - - /* We don't need to worry about locking wrt other resizers just - * yet: we're going to revalidate es->s_blocks_count after - * taking the s_resize_lock below. */ - o_blocks_count = le32_to_cpu(es->s_blocks_count); - - if (test_opt(sb, DEBUG)) - printk(KERN_DEBUG "EXT3-fs: extending last group from "E3FSBLK - " up to "E3FSBLK" blocks\n", - o_blocks_count, n_blocks_count); - - if (n_blocks_count == 0 || n_blocks_count == o_blocks_count) - return 0; - - if (n_blocks_count > (sector_t)(~0ULL) >> (sb->s_blocksize_bits - 9)) { - printk(KERN_ERR "EXT3-fs: filesystem on %s:" - " too large to resize to "E3FSBLK" blocks safely\n", - sb->s_id, n_blocks_count); - if (sizeof(sector_t) < 8) - ext3_warning(sb, __func__, - "CONFIG_LBDAF not enabled\n"); - return -EINVAL; - } - - if (n_blocks_count < o_blocks_count) { - ext3_warning(sb, __func__, - "can't shrink FS - resize aborted"); - return -EBUSY; - } - - /* Handle the remaining blocks in the last group only. */ - last = (o_blocks_count - le32_to_cpu(es->s_first_data_block)) % - EXT3_BLOCKS_PER_GROUP(sb); - - if (last == 0) { - ext3_warning(sb, __func__, - "need to use ext2online to resize further"); - return -EPERM; - } - - add = EXT3_BLOCKS_PER_GROUP(sb) - last; - - if (o_blocks_count + add < o_blocks_count) { - ext3_warning(sb, __func__, "blocks_count overflow"); - return -EINVAL; - } - - if (o_blocks_count + add > n_blocks_count) - add = n_blocks_count - o_blocks_count; - - if (o_blocks_count + add < n_blocks_count) - ext3_warning(sb, __func__, - "will only finish group ("E3FSBLK - " blocks, %u new)", - o_blocks_count + add, add); - - /* See if the device is actually as big as what was requested */ - bh = sb_bread(sb, o_blocks_count + add -1); - if (!bh) { - ext3_warning(sb, __func__, - "can't read last block, resize aborted"); - return -ENOSPC; - } - brelse(bh); - - /* We will update the superblock, one block bitmap, and - * one group descriptor via ext3_free_blocks(). - */ - handle = ext3_journal_start_sb(sb, 3); - if (IS_ERR(handle)) { - err = PTR_ERR(handle); - ext3_warning(sb, __func__, "error %d on journal start",err); - goto exit_put; - } - - mutex_lock(&EXT3_SB(sb)->s_resize_lock); - if (o_blocks_count != le32_to_cpu(es->s_blocks_count)) { - ext3_warning(sb, __func__, - "multiple resizers run on filesystem!"); - mutex_unlock(&EXT3_SB(sb)->s_resize_lock); - ext3_journal_stop(handle); - err = -EBUSY; - goto exit_put; - } - - if ((err = ext3_journal_get_write_access(handle, - EXT3_SB(sb)->s_sbh))) { - ext3_warning(sb, __func__, - "error %d on journal write access", err); - mutex_unlock(&EXT3_SB(sb)->s_resize_lock); - ext3_journal_stop(handle); - goto exit_put; - } - es->s_blocks_count = cpu_to_le32(o_blocks_count + add); - err = ext3_journal_dirty_metadata(handle, EXT3_SB(sb)->s_sbh); - mutex_unlock(&EXT3_SB(sb)->s_resize_lock); - if (err) { - ext3_warning(sb, __func__, - "error %d on journal dirty metadata", err); - ext3_journal_stop(handle); - goto exit_put; - } - ext3_debug("freeing blocks "E3FSBLK" through "E3FSBLK"\n", - o_blocks_count, o_blocks_count + add); - ext3_free_blocks_sb(handle, sb, o_blocks_count, add, &freed_blocks); - ext3_debug("freed blocks "E3FSBLK" through "E3FSBLK"\n", - o_blocks_count, o_blocks_count + add); - if ((err = ext3_journal_stop(handle))) - goto exit_put; - if (test_opt(sb, DEBUG)) - printk(KERN_DEBUG "EXT3-fs: extended group to %u blocks\n", - le32_to_cpu(es->s_blocks_count)); - update_backups(sb, EXT3_SB(sb)->s_sbh->b_blocknr, (char *)es, - sizeof(struct ext3_super_block)); -exit_put: - return err; -} /* ext3_group_extend */ diff --git a/fs/ext3/super.c b/fs/ext3/super.c deleted file mode 100644 index 5ed0044fbb3778..00000000000000 --- a/fs/ext3/super.c +++ /dev/null @@ -1,3165 +0,0 @@ -/* - * linux/fs/ext3/super.c - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * - * from - * - * linux/fs/minix/inode.c - * - * Copyright (C) 1991, 1992 Linus Torvalds - * - * Big-endian to little-endian byte-swapping/bitmaps by - * David S. Miller (davem@caip.rutgers.edu), 1995 - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#define CREATE_TRACE_POINTS - -#include "ext3.h" -#include "xattr.h" -#include "acl.h" -#include "namei.h" - -#ifdef CONFIG_EXT3_DEFAULTS_TO_ORDERED - #define EXT3_MOUNT_DEFAULT_DATA_MODE EXT3_MOUNT_ORDERED_DATA -#else - #define EXT3_MOUNT_DEFAULT_DATA_MODE EXT3_MOUNT_WRITEBACK_DATA -#endif - -static int ext3_load_journal(struct super_block *, struct ext3_super_block *, - unsigned long journal_devnum); -static int ext3_create_journal(struct super_block *, struct ext3_super_block *, - unsigned int); -static int ext3_commit_super(struct super_block *sb, - struct ext3_super_block *es, - int sync); -static void ext3_mark_recovery_complete(struct super_block * sb, - struct ext3_super_block * es); -static void ext3_clear_journal_err(struct super_block * sb, - struct ext3_super_block * es); -static int ext3_sync_fs(struct super_block *sb, int wait); -static const char *ext3_decode_error(struct super_block * sb, int errno, - char nbuf[16]); -static int ext3_remount (struct super_block * sb, int * flags, char * data); -static int ext3_statfs (struct dentry * dentry, struct kstatfs * buf); -static int ext3_unfreeze(struct super_block *sb); -static int ext3_freeze(struct super_block *sb); - -/* - * Wrappers for journal_start/end. - */ -handle_t *ext3_journal_start_sb(struct super_block *sb, int nblocks) -{ - journal_t *journal; - - if (sb->s_flags & MS_RDONLY) - return ERR_PTR(-EROFS); - - /* Special case here: if the journal has aborted behind our - * backs (eg. EIO in the commit thread), then we still need to - * take the FS itself readonly cleanly. */ - journal = EXT3_SB(sb)->s_journal; - if (is_journal_aborted(journal)) { - ext3_abort(sb, __func__, - "Detected aborted journal"); - return ERR_PTR(-EROFS); - } - - return journal_start(journal, nblocks); -} - -int __ext3_journal_stop(const char *where, handle_t *handle) -{ - struct super_block *sb; - int err; - int rc; - - sb = handle->h_transaction->t_journal->j_private; - err = handle->h_err; - rc = journal_stop(handle); - - if (!err) - err = rc; - if (err) - __ext3_std_error(sb, where, err); - return err; -} - -void ext3_journal_abort_handle(const char *caller, const char *err_fn, - struct buffer_head *bh, handle_t *handle, int err) -{ - char nbuf[16]; - const char *errstr = ext3_decode_error(NULL, err, nbuf); - - if (bh) - BUFFER_TRACE(bh, "abort"); - - if (!handle->h_err) - handle->h_err = err; - - if (is_handle_aborted(handle)) - return; - - printk(KERN_ERR "EXT3-fs: %s: aborting transaction: %s in %s\n", - caller, errstr, err_fn); - - journal_abort_handle(handle); -} - -void ext3_msg(struct super_block *sb, const char *prefix, - const char *fmt, ...) -{ - struct va_format vaf; - va_list args; - - va_start(args, fmt); - - vaf.fmt = fmt; - vaf.va = &args; - - printk("%sEXT3-fs (%s): %pV\n", prefix, sb->s_id, &vaf); - - va_end(args); -} - -/* Deal with the reporting of failure conditions on a filesystem such as - * inconsistencies detected or read IO failures. - * - * On ext2, we can store the error state of the filesystem in the - * superblock. That is not possible on ext3, because we may have other - * write ordering constraints on the superblock which prevent us from - * writing it out straight away; and given that the journal is about to - * be aborted, we can't rely on the current, or future, transactions to - * write out the superblock safely. - * - * We'll just use the journal_abort() error code to record an error in - * the journal instead. On recovery, the journal will complain about - * that error until we've noted it down and cleared it. - */ - -static void ext3_handle_error(struct super_block *sb) -{ - struct ext3_super_block *es = EXT3_SB(sb)->s_es; - - EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS; - es->s_state |= cpu_to_le16(EXT3_ERROR_FS); - - if (sb->s_flags & MS_RDONLY) - return; - - if (!test_opt (sb, ERRORS_CONT)) { - journal_t *journal = EXT3_SB(sb)->s_journal; - - set_opt(EXT3_SB(sb)->s_mount_opt, ABORT); - if (journal) - journal_abort(journal, -EIO); - } - if (test_opt (sb, ERRORS_RO)) { - ext3_msg(sb, KERN_CRIT, - "error: remounting filesystem read-only"); - /* - * Make sure updated value of ->s_mount_state will be visible - * before ->s_flags update. - */ - smp_wmb(); - sb->s_flags |= MS_RDONLY; - } - ext3_commit_super(sb, es, 1); - if (test_opt(sb, ERRORS_PANIC)) - panic("EXT3-fs (%s): panic forced after error\n", - sb->s_id); -} - -void ext3_error(struct super_block *sb, const char *function, - const char *fmt, ...) -{ - struct va_format vaf; - va_list args; - - va_start(args, fmt); - - vaf.fmt = fmt; - vaf.va = &args; - - printk(KERN_CRIT "EXT3-fs error (device %s): %s: %pV\n", - sb->s_id, function, &vaf); - - va_end(args); - - ext3_handle_error(sb); -} - -static const char *ext3_decode_error(struct super_block * sb, int errno, - char nbuf[16]) -{ - char *errstr = NULL; - - switch (errno) { - case -EIO: - errstr = "IO failure"; - break; - case -ENOMEM: - errstr = "Out of memory"; - break; - case -EROFS: - if (!sb || EXT3_SB(sb)->s_journal->j_flags & JFS_ABORT) - errstr = "Journal has aborted"; - else - errstr = "Readonly filesystem"; - break; - default: - /* If the caller passed in an extra buffer for unknown - * errors, textualise them now. Else we just return - * NULL. */ - if (nbuf) { - /* Check for truncated error codes... */ - if (snprintf(nbuf, 16, "error %d", -errno) >= 0) - errstr = nbuf; - } - break; - } - - return errstr; -} - -/* __ext3_std_error decodes expected errors from journaling functions - * automatically and invokes the appropriate error response. */ - -void __ext3_std_error (struct super_block * sb, const char * function, - int errno) -{ - char nbuf[16]; - const char *errstr; - - /* Special case: if the error is EROFS, and we're not already - * inside a transaction, then there's really no point in logging - * an error. */ - if (errno == -EROFS && journal_current_handle() == NULL && - (sb->s_flags & MS_RDONLY)) - return; - - errstr = ext3_decode_error(sb, errno, nbuf); - ext3_msg(sb, KERN_CRIT, "error in %s: %s", function, errstr); - - ext3_handle_error(sb); -} - -/* - * ext3_abort is a much stronger failure handler than ext3_error. The - * abort function may be used to deal with unrecoverable failures such - * as journal IO errors or ENOMEM at a critical moment in log management. - * - * We unconditionally force the filesystem into an ABORT|READONLY state, - * unless the error response on the fs has been set to panic in which - * case we take the easy way out and panic immediately. - */ - -void ext3_abort(struct super_block *sb, const char *function, - const char *fmt, ...) -{ - struct va_format vaf; - va_list args; - - va_start(args, fmt); - - vaf.fmt = fmt; - vaf.va = &args; - - printk(KERN_CRIT "EXT3-fs (%s): error: %s: %pV\n", - sb->s_id, function, &vaf); - - va_end(args); - - if (test_opt(sb, ERRORS_PANIC)) - panic("EXT3-fs: panic from previous error\n"); - - if (sb->s_flags & MS_RDONLY) - return; - - ext3_msg(sb, KERN_CRIT, - "error: remounting filesystem read-only"); - EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS; - set_opt(EXT3_SB(sb)->s_mount_opt, ABORT); - /* - * Make sure updated value of ->s_mount_state will be visible - * before ->s_flags update. - */ - smp_wmb(); - sb->s_flags |= MS_RDONLY; - - if (EXT3_SB(sb)->s_journal) - journal_abort(EXT3_SB(sb)->s_journal, -EIO); -} - -void ext3_warning(struct super_block *sb, const char *function, - const char *fmt, ...) -{ - struct va_format vaf; - va_list args; - - va_start(args, fmt); - - vaf.fmt = fmt; - vaf.va = &args; - - printk(KERN_WARNING "EXT3-fs (%s): warning: %s: %pV\n", - sb->s_id, function, &vaf); - - va_end(args); -} - -void ext3_update_dynamic_rev(struct super_block *sb) -{ - struct ext3_super_block *es = EXT3_SB(sb)->s_es; - - if (le32_to_cpu(es->s_rev_level) > EXT3_GOOD_OLD_REV) - return; - - ext3_msg(sb, KERN_WARNING, - "warning: updating to rev %d because of " - "new feature flag, running e2fsck is recommended", - EXT3_DYNAMIC_REV); - - es->s_first_ino = cpu_to_le32(EXT3_GOOD_OLD_FIRST_INO); - es->s_inode_size = cpu_to_le16(EXT3_GOOD_OLD_INODE_SIZE); - es->s_rev_level = cpu_to_le32(EXT3_DYNAMIC_REV); - /* leave es->s_feature_*compat flags alone */ - /* es->s_uuid will be set by e2fsck if empty */ - - /* - * The rest of the superblock fields should be zero, and if not it - * means they are likely already in use, so leave them alone. We - * can leave it up to e2fsck to clean up any inconsistencies there. - */ -} - -/* - * Open the external journal device - */ -static struct block_device *ext3_blkdev_get(dev_t dev, struct super_block *sb) -{ - struct block_device *bdev; - char b[BDEVNAME_SIZE]; - - bdev = blkdev_get_by_dev(dev, FMODE_READ|FMODE_WRITE|FMODE_EXCL, sb); - if (IS_ERR(bdev)) - goto fail; - return bdev; - -fail: - ext3_msg(sb, KERN_ERR, "error: failed to open journal device %s: %ld", - __bdevname(dev, b), PTR_ERR(bdev)); - - return NULL; -} - -/* - * Release the journal device - */ -static void ext3_blkdev_put(struct block_device *bdev) -{ - blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL); -} - -static void ext3_blkdev_remove(struct ext3_sb_info *sbi) -{ - struct block_device *bdev; - bdev = sbi->journal_bdev; - if (bdev) { - ext3_blkdev_put(bdev); - sbi->journal_bdev = NULL; - } -} - -static inline struct inode *orphan_list_entry(struct list_head *l) -{ - return &list_entry(l, struct ext3_inode_info, i_orphan)->vfs_inode; -} - -static void dump_orphan_list(struct super_block *sb, struct ext3_sb_info *sbi) -{ - struct list_head *l; - - ext3_msg(sb, KERN_ERR, "error: sb orphan head is %d", - le32_to_cpu(sbi->s_es->s_last_orphan)); - - ext3_msg(sb, KERN_ERR, "sb_info orphan list:"); - list_for_each(l, &sbi->s_orphan) { - struct inode *inode = orphan_list_entry(l); - ext3_msg(sb, KERN_ERR, " " - "inode %s:%lu at %p: mode %o, nlink %d, next %d\n", - inode->i_sb->s_id, inode->i_ino, inode, - inode->i_mode, inode->i_nlink, - NEXT_ORPHAN(inode)); - } -} - -static void ext3_put_super (struct super_block * sb) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - struct ext3_super_block *es = sbi->s_es; - int i, err; - - dquot_disable(sb, -1, DQUOT_USAGE_ENABLED | DQUOT_LIMITS_ENABLED); - ext3_xattr_put_super(sb); - err = journal_destroy(sbi->s_journal); - sbi->s_journal = NULL; - if (err < 0) - ext3_abort(sb, __func__, "Couldn't clean up the journal"); - - if (!(sb->s_flags & MS_RDONLY)) { - EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER); - es->s_state = cpu_to_le16(sbi->s_mount_state); - BUFFER_TRACE(sbi->s_sbh, "marking dirty"); - mark_buffer_dirty(sbi->s_sbh); - ext3_commit_super(sb, es, 1); - } - - for (i = 0; i < sbi->s_gdb_count; i++) - brelse(sbi->s_group_desc[i]); - kfree(sbi->s_group_desc); - percpu_counter_destroy(&sbi->s_freeblocks_counter); - percpu_counter_destroy(&sbi->s_freeinodes_counter); - percpu_counter_destroy(&sbi->s_dirs_counter); - brelse(sbi->s_sbh); -#ifdef CONFIG_QUOTA - for (i = 0; i < EXT3_MAXQUOTAS; i++) - kfree(sbi->s_qf_names[i]); -#endif - - /* Debugging code just in case the in-memory inode orphan list - * isn't empty. The on-disk one can be non-empty if we've - * detected an error and taken the fs readonly, but the - * in-memory list had better be clean by this point. */ - if (!list_empty(&sbi->s_orphan)) - dump_orphan_list(sb, sbi); - J_ASSERT(list_empty(&sbi->s_orphan)); - - invalidate_bdev(sb->s_bdev); - if (sbi->journal_bdev && sbi->journal_bdev != sb->s_bdev) { - /* - * Invalidate the journal device's buffers. We don't want them - * floating about in memory - the physical journal device may - * hotswapped, and it breaks the `ro-after' testing code. - */ - sync_blockdev(sbi->journal_bdev); - invalidate_bdev(sbi->journal_bdev); - ext3_blkdev_remove(sbi); - } - sb->s_fs_info = NULL; - kfree(sbi->s_blockgroup_lock); - mutex_destroy(&sbi->s_orphan_lock); - mutex_destroy(&sbi->s_resize_lock); - kfree(sbi); -} - -static struct kmem_cache *ext3_inode_cachep; - -/* - * Called inside transaction, so use GFP_NOFS - */ -static struct inode *ext3_alloc_inode(struct super_block *sb) -{ - struct ext3_inode_info *ei; - - ei = kmem_cache_alloc(ext3_inode_cachep, GFP_NOFS); - if (!ei) - return NULL; - ei->i_block_alloc_info = NULL; - ei->vfs_inode.i_version = 1; - atomic_set(&ei->i_datasync_tid, 0); - atomic_set(&ei->i_sync_tid, 0); -#ifdef CONFIG_QUOTA - memset(&ei->i_dquot, 0, sizeof(ei->i_dquot)); -#endif - - return &ei->vfs_inode; -} - -static int ext3_drop_inode(struct inode *inode) -{ - int drop = generic_drop_inode(inode); - - trace_ext3_drop_inode(inode, drop); - return drop; -} - -static void ext3_i_callback(struct rcu_head *head) -{ - struct inode *inode = container_of(head, struct inode, i_rcu); - kmem_cache_free(ext3_inode_cachep, EXT3_I(inode)); -} - -static void ext3_destroy_inode(struct inode *inode) -{ - if (!list_empty(&(EXT3_I(inode)->i_orphan))) { - printk("EXT3 Inode %p: orphan list check failed!\n", - EXT3_I(inode)); - print_hex_dump(KERN_INFO, "", DUMP_PREFIX_ADDRESS, 16, 4, - EXT3_I(inode), sizeof(struct ext3_inode_info), - false); - dump_stack(); - } - call_rcu(&inode->i_rcu, ext3_i_callback); -} - -static void init_once(void *foo) -{ - struct ext3_inode_info *ei = (struct ext3_inode_info *) foo; - - INIT_LIST_HEAD(&ei->i_orphan); -#ifdef CONFIG_EXT3_FS_XATTR - init_rwsem(&ei->xattr_sem); -#endif - mutex_init(&ei->truncate_mutex); - inode_init_once(&ei->vfs_inode); -} - -static int __init init_inodecache(void) -{ - ext3_inode_cachep = kmem_cache_create("ext3_inode_cache", - sizeof(struct ext3_inode_info), - 0, (SLAB_RECLAIM_ACCOUNT| - SLAB_MEM_SPREAD), - init_once); - if (ext3_inode_cachep == NULL) - return -ENOMEM; - return 0; -} - -static void destroy_inodecache(void) -{ - /* - * Make sure all delayed rcu free inodes are flushed before we - * destroy cache. - */ - rcu_barrier(); - kmem_cache_destroy(ext3_inode_cachep); -} - -static inline void ext3_show_quota_options(struct seq_file *seq, struct super_block *sb) -{ -#if defined(CONFIG_QUOTA) - struct ext3_sb_info *sbi = EXT3_SB(sb); - - if (sbi->s_jquota_fmt) { - char *fmtname = ""; - - switch (sbi->s_jquota_fmt) { - case QFMT_VFS_OLD: - fmtname = "vfsold"; - break; - case QFMT_VFS_V0: - fmtname = "vfsv0"; - break; - case QFMT_VFS_V1: - fmtname = "vfsv1"; - break; - } - seq_printf(seq, ",jqfmt=%s", fmtname); - } - - if (sbi->s_qf_names[USRQUOTA]) - seq_printf(seq, ",usrjquota=%s", sbi->s_qf_names[USRQUOTA]); - - if (sbi->s_qf_names[GRPQUOTA]) - seq_printf(seq, ",grpjquota=%s", sbi->s_qf_names[GRPQUOTA]); - - if (test_opt(sb, USRQUOTA)) - seq_puts(seq, ",usrquota"); - - if (test_opt(sb, GRPQUOTA)) - seq_puts(seq, ",grpquota"); -#endif -} - -static char *data_mode_string(unsigned long mode) -{ - switch (mode) { - case EXT3_MOUNT_JOURNAL_DATA: - return "journal"; - case EXT3_MOUNT_ORDERED_DATA: - return "ordered"; - case EXT3_MOUNT_WRITEBACK_DATA: - return "writeback"; - } - return "unknown"; -} - -/* - * Show an option if - * - it's set to a non-default value OR - * - if the per-sb default is different from the global default - */ -static int ext3_show_options(struct seq_file *seq, struct dentry *root) -{ - struct super_block *sb = root->d_sb; - struct ext3_sb_info *sbi = EXT3_SB(sb); - struct ext3_super_block *es = sbi->s_es; - unsigned long def_mount_opts; - - def_mount_opts = le32_to_cpu(es->s_default_mount_opts); - - if (sbi->s_sb_block != 1) - seq_printf(seq, ",sb=%lu", sbi->s_sb_block); - if (test_opt(sb, MINIX_DF)) - seq_puts(seq, ",minixdf"); - if (test_opt(sb, GRPID)) - seq_puts(seq, ",grpid"); - if (!test_opt(sb, GRPID) && (def_mount_opts & EXT3_DEFM_BSDGROUPS)) - seq_puts(seq, ",nogrpid"); - if (!uid_eq(sbi->s_resuid, make_kuid(&init_user_ns, EXT3_DEF_RESUID)) || - le16_to_cpu(es->s_def_resuid) != EXT3_DEF_RESUID) { - seq_printf(seq, ",resuid=%u", - from_kuid_munged(&init_user_ns, sbi->s_resuid)); - } - if (!gid_eq(sbi->s_resgid, make_kgid(&init_user_ns, EXT3_DEF_RESGID)) || - le16_to_cpu(es->s_def_resgid) != EXT3_DEF_RESGID) { - seq_printf(seq, ",resgid=%u", - from_kgid_munged(&init_user_ns, sbi->s_resgid)); - } - if (test_opt(sb, ERRORS_RO)) { - int def_errors = le16_to_cpu(es->s_errors); - - if (def_errors == EXT3_ERRORS_PANIC || - def_errors == EXT3_ERRORS_CONTINUE) { - seq_puts(seq, ",errors=remount-ro"); - } - } - if (test_opt(sb, ERRORS_CONT)) - seq_puts(seq, ",errors=continue"); - if (test_opt(sb, ERRORS_PANIC)) - seq_puts(seq, ",errors=panic"); - if (test_opt(sb, NO_UID32)) - seq_puts(seq, ",nouid32"); - if (test_opt(sb, DEBUG)) - seq_puts(seq, ",debug"); -#ifdef CONFIG_EXT3_FS_XATTR - if (test_opt(sb, XATTR_USER)) - seq_puts(seq, ",user_xattr"); - if (!test_opt(sb, XATTR_USER) && - (def_mount_opts & EXT3_DEFM_XATTR_USER)) { - seq_puts(seq, ",nouser_xattr"); - } -#endif -#ifdef CONFIG_EXT3_FS_POSIX_ACL - if (test_opt(sb, POSIX_ACL)) - seq_puts(seq, ",acl"); - if (!test_opt(sb, POSIX_ACL) && (def_mount_opts & EXT3_DEFM_ACL)) - seq_puts(seq, ",noacl"); -#endif - if (!test_opt(sb, RESERVATION)) - seq_puts(seq, ",noreservation"); - if (sbi->s_commit_interval) { - seq_printf(seq, ",commit=%u", - (unsigned) (sbi->s_commit_interval / HZ)); - } - - /* - * Always display barrier state so it's clear what the status is. - */ - seq_puts(seq, ",barrier="); - seq_puts(seq, test_opt(sb, BARRIER) ? "1" : "0"); - seq_printf(seq, ",data=%s", data_mode_string(test_opt(sb, DATA_FLAGS))); - if (test_opt(sb, DATA_ERR_ABORT)) - seq_puts(seq, ",data_err=abort"); - - if (test_opt(sb, NOLOAD)) - seq_puts(seq, ",norecovery"); - - ext3_show_quota_options(seq, sb); - - return 0; -} - - -static struct inode *ext3_nfs_get_inode(struct super_block *sb, - u64 ino, u32 generation) -{ - struct inode *inode; - - if (ino < EXT3_FIRST_INO(sb) && ino != EXT3_ROOT_INO) - return ERR_PTR(-ESTALE); - if (ino > le32_to_cpu(EXT3_SB(sb)->s_es->s_inodes_count)) - return ERR_PTR(-ESTALE); - - /* iget isn't really right if the inode is currently unallocated!! - * - * ext3_read_inode will return a bad_inode if the inode had been - * deleted, so we should be safe. - * - * Currently we don't know the generation for parent directory, so - * a generation of 0 means "accept any" - */ - inode = ext3_iget(sb, ino); - if (IS_ERR(inode)) - return ERR_CAST(inode); - if (generation && inode->i_generation != generation) { - iput(inode); - return ERR_PTR(-ESTALE); - } - - return inode; -} - -static struct dentry *ext3_fh_to_dentry(struct super_block *sb, struct fid *fid, - int fh_len, int fh_type) -{ - return generic_fh_to_dentry(sb, fid, fh_len, fh_type, - ext3_nfs_get_inode); -} - -static struct dentry *ext3_fh_to_parent(struct super_block *sb, struct fid *fid, - int fh_len, int fh_type) -{ - return generic_fh_to_parent(sb, fid, fh_len, fh_type, - ext3_nfs_get_inode); -} - -/* - * Try to release metadata pages (indirect blocks, directories) which are - * mapped via the block device. Since these pages could have journal heads - * which would prevent try_to_free_buffers() from freeing them, we must use - * jbd layer's try_to_free_buffers() function to release them. - */ -static int bdev_try_to_free_page(struct super_block *sb, struct page *page, - gfp_t wait) -{ - journal_t *journal = EXT3_SB(sb)->s_journal; - - WARN_ON(PageChecked(page)); - if (!page_has_buffers(page)) - return 0; - if (journal) - return journal_try_to_free_buffers(journal, page, - wait & ~__GFP_WAIT); - return try_to_free_buffers(page); -} - -#ifdef CONFIG_QUOTA -#define QTYPE2NAME(t) ((t)==USRQUOTA?"user":"group") -#define QTYPE2MOPT(on, t) ((t)==USRQUOTA?((on)##USRJQUOTA):((on)##GRPJQUOTA)) - -static int ext3_write_dquot(struct dquot *dquot); -static int ext3_acquire_dquot(struct dquot *dquot); -static int ext3_release_dquot(struct dquot *dquot); -static int ext3_mark_dquot_dirty(struct dquot *dquot); -static int ext3_write_info(struct super_block *sb, int type); -static int ext3_quota_on(struct super_block *sb, int type, int format_id, - struct path *path); -static int ext3_quota_on_mount(struct super_block *sb, int type); -static ssize_t ext3_quota_read(struct super_block *sb, int type, char *data, - size_t len, loff_t off); -static ssize_t ext3_quota_write(struct super_block *sb, int type, - const char *data, size_t len, loff_t off); -static struct dquot **ext3_get_dquots(struct inode *inode) -{ - return EXT3_I(inode)->i_dquot; -} - -static const struct dquot_operations ext3_quota_operations = { - .write_dquot = ext3_write_dquot, - .acquire_dquot = ext3_acquire_dquot, - .release_dquot = ext3_release_dquot, - .mark_dirty = ext3_mark_dquot_dirty, - .write_info = ext3_write_info, - .alloc_dquot = dquot_alloc, - .destroy_dquot = dquot_destroy, -}; - -static const struct quotactl_ops ext3_qctl_operations = { - .quota_on = ext3_quota_on, - .quota_off = dquot_quota_off, - .quota_sync = dquot_quota_sync, - .get_state = dquot_get_state, - .set_info = dquot_set_dqinfo, - .get_dqblk = dquot_get_dqblk, - .set_dqblk = dquot_set_dqblk -}; -#endif - -static const struct super_operations ext3_sops = { - .alloc_inode = ext3_alloc_inode, - .destroy_inode = ext3_destroy_inode, - .write_inode = ext3_write_inode, - .dirty_inode = ext3_dirty_inode, - .drop_inode = ext3_drop_inode, - .evict_inode = ext3_evict_inode, - .put_super = ext3_put_super, - .sync_fs = ext3_sync_fs, - .freeze_fs = ext3_freeze, - .unfreeze_fs = ext3_unfreeze, - .statfs = ext3_statfs, - .remount_fs = ext3_remount, - .show_options = ext3_show_options, -#ifdef CONFIG_QUOTA - .quota_read = ext3_quota_read, - .quota_write = ext3_quota_write, - .get_dquots = ext3_get_dquots, -#endif - .bdev_try_to_free_page = bdev_try_to_free_page, -}; - -static const struct export_operations ext3_export_ops = { - .fh_to_dentry = ext3_fh_to_dentry, - .fh_to_parent = ext3_fh_to_parent, - .get_parent = ext3_get_parent, -}; - -enum { - Opt_bsd_df, Opt_minix_df, Opt_grpid, Opt_nogrpid, - Opt_resgid, Opt_resuid, Opt_sb, Opt_err_cont, Opt_err_panic, Opt_err_ro, - Opt_nouid32, Opt_nocheck, Opt_debug, Opt_oldalloc, Opt_orlov, - Opt_user_xattr, Opt_nouser_xattr, Opt_acl, Opt_noacl, - Opt_reservation, Opt_noreservation, Opt_noload, Opt_nobh, Opt_bh, - Opt_commit, Opt_journal_update, Opt_journal_inum, Opt_journal_dev, - Opt_journal_path, - Opt_abort, Opt_data_journal, Opt_data_ordered, Opt_data_writeback, - Opt_data_err_abort, Opt_data_err_ignore, - Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota, - Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_jqfmt_vfsv1, Opt_quota, - Opt_noquota, Opt_ignore, Opt_barrier, Opt_nobarrier, Opt_err, - Opt_resize, Opt_usrquota, Opt_grpquota -}; - -static const match_table_t tokens = { - {Opt_bsd_df, "bsddf"}, - {Opt_minix_df, "minixdf"}, - {Opt_grpid, "grpid"}, - {Opt_grpid, "bsdgroups"}, - {Opt_nogrpid, "nogrpid"}, - {Opt_nogrpid, "sysvgroups"}, - {Opt_resgid, "resgid=%u"}, - {Opt_resuid, "resuid=%u"}, - {Opt_sb, "sb=%u"}, - {Opt_err_cont, "errors=continue"}, - {Opt_err_panic, "errors=panic"}, - {Opt_err_ro, "errors=remount-ro"}, - {Opt_nouid32, "nouid32"}, - {Opt_nocheck, "nocheck"}, - {Opt_nocheck, "check=none"}, - {Opt_debug, "debug"}, - {Opt_oldalloc, "oldalloc"}, - {Opt_orlov, "orlov"}, - {Opt_user_xattr, "user_xattr"}, - {Opt_nouser_xattr, "nouser_xattr"}, - {Opt_acl, "acl"}, - {Opt_noacl, "noacl"}, - {Opt_reservation, "reservation"}, - {Opt_noreservation, "noreservation"}, - {Opt_noload, "noload"}, - {Opt_noload, "norecovery"}, - {Opt_nobh, "nobh"}, - {Opt_bh, "bh"}, - {Opt_commit, "commit=%u"}, - {Opt_journal_update, "journal=update"}, - {Opt_journal_inum, "journal=%u"}, - {Opt_journal_dev, "journal_dev=%u"}, - {Opt_journal_path, "journal_path=%s"}, - {Opt_abort, "abort"}, - {Opt_data_journal, "data=journal"}, - {Opt_data_ordered, "data=ordered"}, - {Opt_data_writeback, "data=writeback"}, - {Opt_data_err_abort, "data_err=abort"}, - {Opt_data_err_ignore, "data_err=ignore"}, - {Opt_offusrjquota, "usrjquota="}, - {Opt_usrjquota, "usrjquota=%s"}, - {Opt_offgrpjquota, "grpjquota="}, - {Opt_grpjquota, "grpjquota=%s"}, - {Opt_jqfmt_vfsold, "jqfmt=vfsold"}, - {Opt_jqfmt_vfsv0, "jqfmt=vfsv0"}, - {Opt_jqfmt_vfsv1, "jqfmt=vfsv1"}, - {Opt_grpquota, "grpquota"}, - {Opt_noquota, "noquota"}, - {Opt_quota, "quota"}, - {Opt_usrquota, "usrquota"}, - {Opt_barrier, "barrier=%u"}, - {Opt_barrier, "barrier"}, - {Opt_nobarrier, "nobarrier"}, - {Opt_resize, "resize"}, - {Opt_err, NULL}, -}; - -static ext3_fsblk_t get_sb_block(void **data, struct super_block *sb) -{ - ext3_fsblk_t sb_block; - char *options = (char *) *data; - - if (!options || strncmp(options, "sb=", 3) != 0) - return 1; /* Default location */ - options += 3; - /*todo: use simple_strtoll with >32bit ext3 */ - sb_block = simple_strtoul(options, &options, 0); - if (*options && *options != ',') { - ext3_msg(sb, KERN_ERR, "error: invalid sb specification: %s", - (char *) *data); - return 1; - } - if (*options == ',') - options++; - *data = (void *) options; - return sb_block; -} - -#ifdef CONFIG_QUOTA -static int set_qf_name(struct super_block *sb, int qtype, substring_t *args) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - char *qname; - - if (sb_any_quota_loaded(sb) && - !sbi->s_qf_names[qtype]) { - ext3_msg(sb, KERN_ERR, - "Cannot change journaled " - "quota options when quota turned on"); - return 0; - } - qname = match_strdup(args); - if (!qname) { - ext3_msg(sb, KERN_ERR, - "Not enough memory for storing quotafile name"); - return 0; - } - if (sbi->s_qf_names[qtype]) { - int same = !strcmp(sbi->s_qf_names[qtype], qname); - - kfree(qname); - if (!same) { - ext3_msg(sb, KERN_ERR, - "%s quota file already specified", - QTYPE2NAME(qtype)); - } - return same; - } - if (strchr(qname, '/')) { - ext3_msg(sb, KERN_ERR, - "quotafile must be on filesystem root"); - kfree(qname); - return 0; - } - sbi->s_qf_names[qtype] = qname; - set_opt(sbi->s_mount_opt, QUOTA); - return 1; -} - -static int clear_qf_name(struct super_block *sb, int qtype) { - - struct ext3_sb_info *sbi = EXT3_SB(sb); - - if (sb_any_quota_loaded(sb) && - sbi->s_qf_names[qtype]) { - ext3_msg(sb, KERN_ERR, "Cannot change journaled quota options" - " when quota turned on"); - return 0; - } - if (sbi->s_qf_names[qtype]) { - kfree(sbi->s_qf_names[qtype]); - sbi->s_qf_names[qtype] = NULL; - } - return 1; -} -#endif - -static int parse_options (char *options, struct super_block *sb, - unsigned int *inum, unsigned long *journal_devnum, - ext3_fsblk_t *n_blocks_count, int is_remount) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - char * p; - substring_t args[MAX_OPT_ARGS]; - int data_opt = 0; - int option; - kuid_t uid; - kgid_t gid; - char *journal_path; - struct inode *journal_inode; - struct path path; - int error; - -#ifdef CONFIG_QUOTA - int qfmt; -#endif - - if (!options) - return 1; - - while ((p = strsep (&options, ",")) != NULL) { - int token; - if (!*p) - continue; - /* - * Initialize args struct so we know whether arg was - * found; some options take optional arguments. - */ - args[0].to = args[0].from = NULL; - token = match_token(p, tokens, args); - switch (token) { - case Opt_bsd_df: - clear_opt (sbi->s_mount_opt, MINIX_DF); - break; - case Opt_minix_df: - set_opt (sbi->s_mount_opt, MINIX_DF); - break; - case Opt_grpid: - set_opt (sbi->s_mount_opt, GRPID); - break; - case Opt_nogrpid: - clear_opt (sbi->s_mount_opt, GRPID); - break; - case Opt_resuid: - if (match_int(&args[0], &option)) - return 0; - uid = make_kuid(current_user_ns(), option); - if (!uid_valid(uid)) { - ext3_msg(sb, KERN_ERR, "Invalid uid value %d", option); - return 0; - - } - sbi->s_resuid = uid; - break; - case Opt_resgid: - if (match_int(&args[0], &option)) - return 0; - gid = make_kgid(current_user_ns(), option); - if (!gid_valid(gid)) { - ext3_msg(sb, KERN_ERR, "Invalid gid value %d", option); - return 0; - } - sbi->s_resgid = gid; - break; - case Opt_sb: - /* handled by get_sb_block() instead of here */ - /* *sb_block = match_int(&args[0]); */ - break; - case Opt_err_panic: - clear_opt (sbi->s_mount_opt, ERRORS_CONT); - clear_opt (sbi->s_mount_opt, ERRORS_RO); - set_opt (sbi->s_mount_opt, ERRORS_PANIC); - break; - case Opt_err_ro: - clear_opt (sbi->s_mount_opt, ERRORS_CONT); - clear_opt (sbi->s_mount_opt, ERRORS_PANIC); - set_opt (sbi->s_mount_opt, ERRORS_RO); - break; - case Opt_err_cont: - clear_opt (sbi->s_mount_opt, ERRORS_RO); - clear_opt (sbi->s_mount_opt, ERRORS_PANIC); - set_opt (sbi->s_mount_opt, ERRORS_CONT); - break; - case Opt_nouid32: - set_opt (sbi->s_mount_opt, NO_UID32); - break; - case Opt_nocheck: - clear_opt (sbi->s_mount_opt, CHECK); - break; - case Opt_debug: - set_opt (sbi->s_mount_opt, DEBUG); - break; - case Opt_oldalloc: - ext3_msg(sb, KERN_WARNING, - "Ignoring deprecated oldalloc option"); - break; - case Opt_orlov: - ext3_msg(sb, KERN_WARNING, - "Ignoring deprecated orlov option"); - break; -#ifdef CONFIG_EXT3_FS_XATTR - case Opt_user_xattr: - set_opt (sbi->s_mount_opt, XATTR_USER); - break; - case Opt_nouser_xattr: - clear_opt (sbi->s_mount_opt, XATTR_USER); - break; -#else - case Opt_user_xattr: - case Opt_nouser_xattr: - ext3_msg(sb, KERN_INFO, - "(no)user_xattr options not supported"); - break; -#endif -#ifdef CONFIG_EXT3_FS_POSIX_ACL - case Opt_acl: - set_opt(sbi->s_mount_opt, POSIX_ACL); - break; - case Opt_noacl: - clear_opt(sbi->s_mount_opt, POSIX_ACL); - break; -#else - case Opt_acl: - case Opt_noacl: - ext3_msg(sb, KERN_INFO, - "(no)acl options not supported"); - break; -#endif - case Opt_reservation: - set_opt(sbi->s_mount_opt, RESERVATION); - break; - case Opt_noreservation: - clear_opt(sbi->s_mount_opt, RESERVATION); - break; - case Opt_journal_update: - /* @@@ FIXME */ - /* Eventually we will want to be able to create - a journal file here. For now, only allow the - user to specify an existing inode to be the - journal file. */ - if (is_remount) { - ext3_msg(sb, KERN_ERR, "error: cannot specify " - "journal on remount"); - return 0; - } - set_opt (sbi->s_mount_opt, UPDATE_JOURNAL); - break; - case Opt_journal_inum: - if (is_remount) { - ext3_msg(sb, KERN_ERR, "error: cannot specify " - "journal on remount"); - return 0; - } - if (match_int(&args[0], &option)) - return 0; - *inum = option; - break; - case Opt_journal_dev: - if (is_remount) { - ext3_msg(sb, KERN_ERR, "error: cannot specify " - "journal on remount"); - return 0; - } - if (match_int(&args[0], &option)) - return 0; - *journal_devnum = option; - break; - case Opt_journal_path: - if (is_remount) { - ext3_msg(sb, KERN_ERR, "error: cannot specify " - "journal on remount"); - return 0; - } - - journal_path = match_strdup(&args[0]); - if (!journal_path) { - ext3_msg(sb, KERN_ERR, "error: could not dup " - "journal device string"); - return 0; - } - - error = kern_path(journal_path, LOOKUP_FOLLOW, &path); - if (error) { - ext3_msg(sb, KERN_ERR, "error: could not find " - "journal device path: error %d", error); - kfree(journal_path); - return 0; - } - - journal_inode = d_inode(path.dentry); - if (!S_ISBLK(journal_inode->i_mode)) { - ext3_msg(sb, KERN_ERR, "error: journal path %s " - "is not a block device", journal_path); - path_put(&path); - kfree(journal_path); - return 0; - } - - *journal_devnum = new_encode_dev(journal_inode->i_rdev); - path_put(&path); - kfree(journal_path); - break; - case Opt_noload: - set_opt (sbi->s_mount_opt, NOLOAD); - break; - case Opt_commit: - if (match_int(&args[0], &option)) - return 0; - if (option < 0) - return 0; - if (option == 0) - option = JBD_DEFAULT_MAX_COMMIT_AGE; - sbi->s_commit_interval = HZ * option; - break; - case Opt_data_journal: - data_opt = EXT3_MOUNT_JOURNAL_DATA; - goto datacheck; - case Opt_data_ordered: - data_opt = EXT3_MOUNT_ORDERED_DATA; - goto datacheck; - case Opt_data_writeback: - data_opt = EXT3_MOUNT_WRITEBACK_DATA; - datacheck: - if (is_remount) { - if (test_opt(sb, DATA_FLAGS) == data_opt) - break; - ext3_msg(sb, KERN_ERR, - "error: cannot change " - "data mode on remount. The filesystem " - "is mounted in data=%s mode and you " - "try to remount it in data=%s mode.", - data_mode_string(test_opt(sb, - DATA_FLAGS)), - data_mode_string(data_opt)); - return 0; - } else { - clear_opt(sbi->s_mount_opt, DATA_FLAGS); - sbi->s_mount_opt |= data_opt; - } - break; - case Opt_data_err_abort: - set_opt(sbi->s_mount_opt, DATA_ERR_ABORT); - break; - case Opt_data_err_ignore: - clear_opt(sbi->s_mount_opt, DATA_ERR_ABORT); - break; -#ifdef CONFIG_QUOTA - case Opt_usrjquota: - if (!set_qf_name(sb, USRQUOTA, &args[0])) - return 0; - break; - case Opt_grpjquota: - if (!set_qf_name(sb, GRPQUOTA, &args[0])) - return 0; - break; - case Opt_offusrjquota: - if (!clear_qf_name(sb, USRQUOTA)) - return 0; - break; - case Opt_offgrpjquota: - if (!clear_qf_name(sb, GRPQUOTA)) - return 0; - break; - case Opt_jqfmt_vfsold: - qfmt = QFMT_VFS_OLD; - goto set_qf_format; - case Opt_jqfmt_vfsv0: - qfmt = QFMT_VFS_V0; - goto set_qf_format; - case Opt_jqfmt_vfsv1: - qfmt = QFMT_VFS_V1; -set_qf_format: - if (sb_any_quota_loaded(sb) && - sbi->s_jquota_fmt != qfmt) { - ext3_msg(sb, KERN_ERR, "error: cannot change " - "journaled quota options when " - "quota turned on."); - return 0; - } - sbi->s_jquota_fmt = qfmt; - break; - case Opt_quota: - case Opt_usrquota: - set_opt(sbi->s_mount_opt, QUOTA); - set_opt(sbi->s_mount_opt, USRQUOTA); - break; - case Opt_grpquota: - set_opt(sbi->s_mount_opt, QUOTA); - set_opt(sbi->s_mount_opt, GRPQUOTA); - break; - case Opt_noquota: - if (sb_any_quota_loaded(sb)) { - ext3_msg(sb, KERN_ERR, "error: cannot change " - "quota options when quota turned on."); - return 0; - } - clear_opt(sbi->s_mount_opt, QUOTA); - clear_opt(sbi->s_mount_opt, USRQUOTA); - clear_opt(sbi->s_mount_opt, GRPQUOTA); - break; -#else - case Opt_quota: - case Opt_usrquota: - case Opt_grpquota: - ext3_msg(sb, KERN_ERR, - "error: quota options not supported."); - break; - case Opt_usrjquota: - case Opt_grpjquota: - case Opt_offusrjquota: - case Opt_offgrpjquota: - case Opt_jqfmt_vfsold: - case Opt_jqfmt_vfsv0: - case Opt_jqfmt_vfsv1: - ext3_msg(sb, KERN_ERR, - "error: journaled quota options not " - "supported."); - break; - case Opt_noquota: - break; -#endif - case Opt_abort: - set_opt(sbi->s_mount_opt, ABORT); - break; - case Opt_nobarrier: - clear_opt(sbi->s_mount_opt, BARRIER); - break; - case Opt_barrier: - if (args[0].from) { - if (match_int(&args[0], &option)) - return 0; - } else - option = 1; /* No argument, default to 1 */ - if (option) - set_opt(sbi->s_mount_opt, BARRIER); - else - clear_opt(sbi->s_mount_opt, BARRIER); - break; - case Opt_ignore: - break; - case Opt_resize: - if (!is_remount) { - ext3_msg(sb, KERN_ERR, - "error: resize option only available " - "for remount"); - return 0; - } - if (match_int(&args[0], &option) != 0) - return 0; - *n_blocks_count = option; - break; - case Opt_nobh: - ext3_msg(sb, KERN_WARNING, - "warning: ignoring deprecated nobh option"); - break; - case Opt_bh: - ext3_msg(sb, KERN_WARNING, - "warning: ignoring deprecated bh option"); - break; - default: - ext3_msg(sb, KERN_ERR, - "error: unrecognized mount option \"%s\" " - "or missing value", p); - return 0; - } - } -#ifdef CONFIG_QUOTA - if (sbi->s_qf_names[USRQUOTA] || sbi->s_qf_names[GRPQUOTA]) { - if (test_opt(sb, USRQUOTA) && sbi->s_qf_names[USRQUOTA]) - clear_opt(sbi->s_mount_opt, USRQUOTA); - if (test_opt(sb, GRPQUOTA) && sbi->s_qf_names[GRPQUOTA]) - clear_opt(sbi->s_mount_opt, GRPQUOTA); - - if (test_opt(sb, GRPQUOTA) || test_opt(sb, USRQUOTA)) { - ext3_msg(sb, KERN_ERR, "error: old and new quota " - "format mixing."); - return 0; - } - - if (!sbi->s_jquota_fmt) { - ext3_msg(sb, KERN_ERR, "error: journaled quota format " - "not specified."); - return 0; - } - } -#endif - return 1; -} - -static int ext3_setup_super(struct super_block *sb, struct ext3_super_block *es, - int read_only) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - int res = 0; - - if (le32_to_cpu(es->s_rev_level) > EXT3_MAX_SUPP_REV) { - ext3_msg(sb, KERN_ERR, - "error: revision level too high, " - "forcing read-only mode"); - res = MS_RDONLY; - } - if (read_only) - return res; - if (!(sbi->s_mount_state & EXT3_VALID_FS)) - ext3_msg(sb, KERN_WARNING, - "warning: mounting unchecked fs, " - "running e2fsck is recommended"); - else if ((sbi->s_mount_state & EXT3_ERROR_FS)) - ext3_msg(sb, KERN_WARNING, - "warning: mounting fs with errors, " - "running e2fsck is recommended"); - else if ((__s16) le16_to_cpu(es->s_max_mnt_count) > 0 && - le16_to_cpu(es->s_mnt_count) >= - le16_to_cpu(es->s_max_mnt_count)) - ext3_msg(sb, KERN_WARNING, - "warning: maximal mount count reached, " - "running e2fsck is recommended"); - else if (le32_to_cpu(es->s_checkinterval) && - (le32_to_cpu(es->s_lastcheck) + - le32_to_cpu(es->s_checkinterval) <= get_seconds())) - ext3_msg(sb, KERN_WARNING, - "warning: checktime reached, " - "running e2fsck is recommended"); -#if 0 - /* @@@ We _will_ want to clear the valid bit if we find - inconsistencies, to force a fsck at reboot. But for - a plain journaled filesystem we can keep it set as - valid forever! :) */ - es->s_state &= cpu_to_le16(~EXT3_VALID_FS); -#endif - if (!le16_to_cpu(es->s_max_mnt_count)) - es->s_max_mnt_count = cpu_to_le16(EXT3_DFL_MAX_MNT_COUNT); - le16_add_cpu(&es->s_mnt_count, 1); - es->s_mtime = cpu_to_le32(get_seconds()); - ext3_update_dynamic_rev(sb); - EXT3_SET_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER); - - ext3_commit_super(sb, es, 1); - if (test_opt(sb, DEBUG)) - ext3_msg(sb, KERN_INFO, "[bs=%lu, gc=%lu, " - "bpg=%lu, ipg=%lu, mo=%04lx]", - sb->s_blocksize, - sbi->s_groups_count, - EXT3_BLOCKS_PER_GROUP(sb), - EXT3_INODES_PER_GROUP(sb), - sbi->s_mount_opt); - - if (EXT3_SB(sb)->s_journal->j_inode == NULL) { - char b[BDEVNAME_SIZE]; - ext3_msg(sb, KERN_INFO, "using external journal on %s", - bdevname(EXT3_SB(sb)->s_journal->j_dev, b)); - } else { - ext3_msg(sb, KERN_INFO, "using internal journal"); - } - cleancache_init_fs(sb); - return res; -} - -/* Called at mount-time, super-block is locked */ -static int ext3_check_descriptors(struct super_block *sb) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - int i; - - ext3_debug ("Checking group descriptors"); - - for (i = 0; i < sbi->s_groups_count; i++) { - struct ext3_group_desc *gdp = ext3_get_group_desc(sb, i, NULL); - ext3_fsblk_t first_block = ext3_group_first_block_no(sb, i); - ext3_fsblk_t last_block; - - if (i == sbi->s_groups_count - 1) - last_block = le32_to_cpu(sbi->s_es->s_blocks_count) - 1; - else - last_block = first_block + - (EXT3_BLOCKS_PER_GROUP(sb) - 1); - - if (le32_to_cpu(gdp->bg_block_bitmap) < first_block || - le32_to_cpu(gdp->bg_block_bitmap) > last_block) - { - ext3_error (sb, "ext3_check_descriptors", - "Block bitmap for group %d" - " not in group (block %lu)!", - i, (unsigned long) - le32_to_cpu(gdp->bg_block_bitmap)); - return 0; - } - if (le32_to_cpu(gdp->bg_inode_bitmap) < first_block || - le32_to_cpu(gdp->bg_inode_bitmap) > last_block) - { - ext3_error (sb, "ext3_check_descriptors", - "Inode bitmap for group %d" - " not in group (block %lu)!", - i, (unsigned long) - le32_to_cpu(gdp->bg_inode_bitmap)); - return 0; - } - if (le32_to_cpu(gdp->bg_inode_table) < first_block || - le32_to_cpu(gdp->bg_inode_table) + sbi->s_itb_per_group - 1 > - last_block) - { - ext3_error (sb, "ext3_check_descriptors", - "Inode table for group %d" - " not in group (block %lu)!", - i, (unsigned long) - le32_to_cpu(gdp->bg_inode_table)); - return 0; - } - } - - sbi->s_es->s_free_blocks_count=cpu_to_le32(ext3_count_free_blocks(sb)); - sbi->s_es->s_free_inodes_count=cpu_to_le32(ext3_count_free_inodes(sb)); - return 1; -} - - -/* ext3_orphan_cleanup() walks a singly-linked list of inodes (starting at - * the superblock) which were deleted from all directories, but held open by - * a process at the time of a crash. We walk the list and try to delete these - * inodes at recovery time (only with a read-write filesystem). - * - * In order to keep the orphan inode chain consistent during traversal (in - * case of crash during recovery), we link each inode into the superblock - * orphan list_head and handle it the same way as an inode deletion during - * normal operation (which journals the operations for us). - * - * We only do an iget() and an iput() on each inode, which is very safe if we - * accidentally point at an in-use or already deleted inode. The worst that - * can happen in this case is that we get a "bit already cleared" message from - * ext3_free_inode(). The only reason we would point at a wrong inode is if - * e2fsck was run on this filesystem, and it must have already done the orphan - * inode cleanup for us, so we can safely abort without any further action. - */ -static void ext3_orphan_cleanup (struct super_block * sb, - struct ext3_super_block * es) -{ - unsigned int s_flags = sb->s_flags; - int nr_orphans = 0, nr_truncates = 0; -#ifdef CONFIG_QUOTA - int i; -#endif - if (!es->s_last_orphan) { - jbd_debug(4, "no orphan inodes to clean up\n"); - return; - } - - if (bdev_read_only(sb->s_bdev)) { - ext3_msg(sb, KERN_ERR, "error: write access " - "unavailable, skipping orphan cleanup."); - return; - } - - /* Check if feature set allows readwrite operations */ - if (EXT3_HAS_RO_COMPAT_FEATURE(sb, ~EXT3_FEATURE_RO_COMPAT_SUPP)) { - ext3_msg(sb, KERN_INFO, "Skipping orphan cleanup due to " - "unknown ROCOMPAT features"); - return; - } - - if (EXT3_SB(sb)->s_mount_state & EXT3_ERROR_FS) { - /* don't clear list on RO mount w/ errors */ - if (es->s_last_orphan && !(s_flags & MS_RDONLY)) { - jbd_debug(1, "Errors on filesystem, " - "clearing orphan list.\n"); - es->s_last_orphan = 0; - } - jbd_debug(1, "Skipping orphan recovery on fs with errors.\n"); - return; - } - - if (s_flags & MS_RDONLY) { - ext3_msg(sb, KERN_INFO, "orphan cleanup on readonly fs"); - sb->s_flags &= ~MS_RDONLY; - } -#ifdef CONFIG_QUOTA - /* Needed for iput() to work correctly and not trash data */ - sb->s_flags |= MS_ACTIVE; - /* Turn on quotas so that they are updated correctly */ - for (i = 0; i < EXT3_MAXQUOTAS; i++) { - if (EXT3_SB(sb)->s_qf_names[i]) { - int ret = ext3_quota_on_mount(sb, i); - if (ret < 0) - ext3_msg(sb, KERN_ERR, - "error: cannot turn on journaled " - "quota: %d", ret); - } - } -#endif - - while (es->s_last_orphan) { - struct inode *inode; - - inode = ext3_orphan_get(sb, le32_to_cpu(es->s_last_orphan)); - if (IS_ERR(inode)) { - es->s_last_orphan = 0; - break; - } - - list_add(&EXT3_I(inode)->i_orphan, &EXT3_SB(sb)->s_orphan); - dquot_initialize(inode); - if (inode->i_nlink) { - printk(KERN_DEBUG - "%s: truncating inode %lu to %Ld bytes\n", - __func__, inode->i_ino, inode->i_size); - jbd_debug(2, "truncating inode %lu to %Ld bytes\n", - inode->i_ino, inode->i_size); - ext3_truncate(inode); - nr_truncates++; - } else { - printk(KERN_DEBUG - "%s: deleting unreferenced inode %lu\n", - __func__, inode->i_ino); - jbd_debug(2, "deleting unreferenced inode %lu\n", - inode->i_ino); - nr_orphans++; - } - iput(inode); /* The delete magic happens here! */ - } - -#define PLURAL(x) (x), ((x)==1) ? "" : "s" - - if (nr_orphans) - ext3_msg(sb, KERN_INFO, "%d orphan inode%s deleted", - PLURAL(nr_orphans)); - if (nr_truncates) - ext3_msg(sb, KERN_INFO, "%d truncate%s cleaned up", - PLURAL(nr_truncates)); -#ifdef CONFIG_QUOTA - /* Turn quotas off */ - for (i = 0; i < EXT3_MAXQUOTAS; i++) { - if (sb_dqopt(sb)->files[i]) - dquot_quota_off(sb, i); - } -#endif - sb->s_flags = s_flags; /* Restore MS_RDONLY status */ -} - -/* - * Maximal file size. There is a direct, and {,double-,triple-}indirect - * block limit, and also a limit of (2^32 - 1) 512-byte sectors in i_blocks. - * We need to be 1 filesystem block less than the 2^32 sector limit. - */ -static loff_t ext3_max_size(int bits) -{ - loff_t res = EXT3_NDIR_BLOCKS; - int meta_blocks; - loff_t upper_limit; - - /* This is calculated to be the largest file size for a - * dense, file such that the total number of - * sectors in the file, including data and all indirect blocks, - * does not exceed 2^32 -1 - * __u32 i_blocks representing the total number of - * 512 bytes blocks of the file - */ - upper_limit = (1LL << 32) - 1; - - /* total blocks in file system block size */ - upper_limit >>= (bits - 9); - - - /* indirect blocks */ - meta_blocks = 1; - /* double indirect blocks */ - meta_blocks += 1 + (1LL << (bits-2)); - /* tripple indirect blocks */ - meta_blocks += 1 + (1LL << (bits-2)) + (1LL << (2*(bits-2))); - - upper_limit -= meta_blocks; - upper_limit <<= bits; - - res += 1LL << (bits-2); - res += 1LL << (2*(bits-2)); - res += 1LL << (3*(bits-2)); - res <<= bits; - if (res > upper_limit) - res = upper_limit; - - if (res > MAX_LFS_FILESIZE) - res = MAX_LFS_FILESIZE; - - return res; -} - -static ext3_fsblk_t descriptor_loc(struct super_block *sb, - ext3_fsblk_t logic_sb_block, - int nr) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - unsigned long bg, first_meta_bg; - int has_super = 0; - - first_meta_bg = le32_to_cpu(sbi->s_es->s_first_meta_bg); - - if (!EXT3_HAS_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_META_BG) || - nr < first_meta_bg) - return (logic_sb_block + nr + 1); - bg = sbi->s_desc_per_block * nr; - if (ext3_bg_has_super(sb, bg)) - has_super = 1; - return (has_super + ext3_group_first_block_no(sb, bg)); -} - - -static int ext3_fill_super (struct super_block *sb, void *data, int silent) -{ - struct buffer_head * bh; - struct ext3_super_block *es = NULL; - struct ext3_sb_info *sbi; - ext3_fsblk_t block; - ext3_fsblk_t sb_block = get_sb_block(&data, sb); - ext3_fsblk_t logic_sb_block; - unsigned long offset = 0; - unsigned int journal_inum = 0; - unsigned long journal_devnum = 0; - unsigned long def_mount_opts; - struct inode *root; - int blocksize; - int hblock; - int db_count; - int i; - int needs_recovery; - int ret = -EINVAL; - __le32 features; - int err; - - sbi = kzalloc(sizeof(*sbi), GFP_KERNEL); - if (!sbi) - return -ENOMEM; - - sbi->s_blockgroup_lock = - kzalloc(sizeof(struct blockgroup_lock), GFP_KERNEL); - if (!sbi->s_blockgroup_lock) { - kfree(sbi); - return -ENOMEM; - } - sb->s_fs_info = sbi; - sbi->s_sb_block = sb_block; - - blocksize = sb_min_blocksize(sb, EXT3_MIN_BLOCK_SIZE); - if (!blocksize) { - ext3_msg(sb, KERN_ERR, "error: unable to set blocksize"); - goto out_fail; - } - - /* - * The ext3 superblock will not be buffer aligned for other than 1kB - * block sizes. We need to calculate the offset from buffer start. - */ - if (blocksize != EXT3_MIN_BLOCK_SIZE) { - logic_sb_block = (sb_block * EXT3_MIN_BLOCK_SIZE) / blocksize; - offset = (sb_block * EXT3_MIN_BLOCK_SIZE) % blocksize; - } else { - logic_sb_block = sb_block; - } - - if (!(bh = sb_bread(sb, logic_sb_block))) { - ext3_msg(sb, KERN_ERR, "error: unable to read superblock"); - goto out_fail; - } - /* - * Note: s_es must be initialized as soon as possible because - * some ext3 macro-instructions depend on its value - */ - es = (struct ext3_super_block *) (bh->b_data + offset); - sbi->s_es = es; - sb->s_magic = le16_to_cpu(es->s_magic); - if (sb->s_magic != EXT3_SUPER_MAGIC) - goto cantfind_ext3; - - /* Set defaults before we parse the mount options */ - def_mount_opts = le32_to_cpu(es->s_default_mount_opts); - if (def_mount_opts & EXT3_DEFM_DEBUG) - set_opt(sbi->s_mount_opt, DEBUG); - if (def_mount_opts & EXT3_DEFM_BSDGROUPS) - set_opt(sbi->s_mount_opt, GRPID); - if (def_mount_opts & EXT3_DEFM_UID16) - set_opt(sbi->s_mount_opt, NO_UID32); -#ifdef CONFIG_EXT3_FS_XATTR - if (def_mount_opts & EXT3_DEFM_XATTR_USER) - set_opt(sbi->s_mount_opt, XATTR_USER); -#endif -#ifdef CONFIG_EXT3_FS_POSIX_ACL - if (def_mount_opts & EXT3_DEFM_ACL) - set_opt(sbi->s_mount_opt, POSIX_ACL); -#endif - if ((def_mount_opts & EXT3_DEFM_JMODE) == EXT3_DEFM_JMODE_DATA) - set_opt(sbi->s_mount_opt, JOURNAL_DATA); - else if ((def_mount_opts & EXT3_DEFM_JMODE) == EXT3_DEFM_JMODE_ORDERED) - set_opt(sbi->s_mount_opt, ORDERED_DATA); - else if ((def_mount_opts & EXT3_DEFM_JMODE) == EXT3_DEFM_JMODE_WBACK) - set_opt(sbi->s_mount_opt, WRITEBACK_DATA); - - if (le16_to_cpu(sbi->s_es->s_errors) == EXT3_ERRORS_PANIC) - set_opt(sbi->s_mount_opt, ERRORS_PANIC); - else if (le16_to_cpu(sbi->s_es->s_errors) == EXT3_ERRORS_CONTINUE) - set_opt(sbi->s_mount_opt, ERRORS_CONT); - else - set_opt(sbi->s_mount_opt, ERRORS_RO); - - sbi->s_resuid = make_kuid(&init_user_ns, le16_to_cpu(es->s_def_resuid)); - sbi->s_resgid = make_kgid(&init_user_ns, le16_to_cpu(es->s_def_resgid)); - - /* enable barriers by default */ - set_opt(sbi->s_mount_opt, BARRIER); - set_opt(sbi->s_mount_opt, RESERVATION); - - if (!parse_options ((char *) data, sb, &journal_inum, &journal_devnum, - NULL, 0)) - goto failed_mount; - - sb->s_flags = (sb->s_flags & ~MS_POSIXACL) | - (test_opt(sb, POSIX_ACL) ? MS_POSIXACL : 0); - - if (le32_to_cpu(es->s_rev_level) == EXT3_GOOD_OLD_REV && - (EXT3_HAS_COMPAT_FEATURE(sb, ~0U) || - EXT3_HAS_RO_COMPAT_FEATURE(sb, ~0U) || - EXT3_HAS_INCOMPAT_FEATURE(sb, ~0U))) - ext3_msg(sb, KERN_WARNING, - "warning: feature flags set on rev 0 fs, " - "running e2fsck is recommended"); - /* - * Check feature flags regardless of the revision level, since we - * previously didn't change the revision level when setting the flags, - * so there is a chance incompat flags are set on a rev 0 filesystem. - */ - features = EXT3_HAS_INCOMPAT_FEATURE(sb, ~EXT3_FEATURE_INCOMPAT_SUPP); - if (features) { - ext3_msg(sb, KERN_ERR, - "error: couldn't mount because of unsupported " - "optional features (%x)", le32_to_cpu(features)); - goto failed_mount; - } - features = EXT3_HAS_RO_COMPAT_FEATURE(sb, ~EXT3_FEATURE_RO_COMPAT_SUPP); - if (!(sb->s_flags & MS_RDONLY) && features) { - ext3_msg(sb, KERN_ERR, - "error: couldn't mount RDWR because of unsupported " - "optional features (%x)", le32_to_cpu(features)); - goto failed_mount; - } - blocksize = BLOCK_SIZE << le32_to_cpu(es->s_log_block_size); - - if (blocksize < EXT3_MIN_BLOCK_SIZE || - blocksize > EXT3_MAX_BLOCK_SIZE) { - ext3_msg(sb, KERN_ERR, - "error: couldn't mount because of unsupported " - "filesystem blocksize %d", blocksize); - goto failed_mount; - } - - hblock = bdev_logical_block_size(sb->s_bdev); - if (sb->s_blocksize != blocksize) { - /* - * Make sure the blocksize for the filesystem is larger - * than the hardware sectorsize for the machine. - */ - if (blocksize < hblock) { - ext3_msg(sb, KERN_ERR, - "error: fsblocksize %d too small for " - "hardware sectorsize %d", blocksize, hblock); - goto failed_mount; - } - - brelse (bh); - if (!sb_set_blocksize(sb, blocksize)) { - ext3_msg(sb, KERN_ERR, - "error: bad blocksize %d", blocksize); - goto out_fail; - } - logic_sb_block = (sb_block * EXT3_MIN_BLOCK_SIZE) / blocksize; - offset = (sb_block * EXT3_MIN_BLOCK_SIZE) % blocksize; - bh = sb_bread(sb, logic_sb_block); - if (!bh) { - ext3_msg(sb, KERN_ERR, - "error: can't read superblock on 2nd try"); - goto failed_mount; - } - es = (struct ext3_super_block *)(bh->b_data + offset); - sbi->s_es = es; - if (es->s_magic != cpu_to_le16(EXT3_SUPER_MAGIC)) { - ext3_msg(sb, KERN_ERR, - "error: magic mismatch"); - goto failed_mount; - } - } - - sb->s_maxbytes = ext3_max_size(sb->s_blocksize_bits); - - if (le32_to_cpu(es->s_rev_level) == EXT3_GOOD_OLD_REV) { - sbi->s_inode_size = EXT3_GOOD_OLD_INODE_SIZE; - sbi->s_first_ino = EXT3_GOOD_OLD_FIRST_INO; - } else { - sbi->s_inode_size = le16_to_cpu(es->s_inode_size); - sbi->s_first_ino = le32_to_cpu(es->s_first_ino); - if ((sbi->s_inode_size < EXT3_GOOD_OLD_INODE_SIZE) || - (!is_power_of_2(sbi->s_inode_size)) || - (sbi->s_inode_size > blocksize)) { - ext3_msg(sb, KERN_ERR, - "error: unsupported inode size: %d", - sbi->s_inode_size); - goto failed_mount; - } - } - sbi->s_frag_size = EXT3_MIN_FRAG_SIZE << - le32_to_cpu(es->s_log_frag_size); - if (blocksize != sbi->s_frag_size) { - ext3_msg(sb, KERN_ERR, - "error: fragsize %lu != blocksize %u (unsupported)", - sbi->s_frag_size, blocksize); - goto failed_mount; - } - sbi->s_frags_per_block = 1; - sbi->s_blocks_per_group = le32_to_cpu(es->s_blocks_per_group); - sbi->s_frags_per_group = le32_to_cpu(es->s_frags_per_group); - sbi->s_inodes_per_group = le32_to_cpu(es->s_inodes_per_group); - if (EXT3_INODE_SIZE(sb) == 0 || EXT3_INODES_PER_GROUP(sb) == 0) - goto cantfind_ext3; - sbi->s_inodes_per_block = blocksize / EXT3_INODE_SIZE(sb); - if (sbi->s_inodes_per_block == 0) - goto cantfind_ext3; - sbi->s_itb_per_group = sbi->s_inodes_per_group / - sbi->s_inodes_per_block; - sbi->s_desc_per_block = blocksize / sizeof(struct ext3_group_desc); - sbi->s_sbh = bh; - sbi->s_mount_state = le16_to_cpu(es->s_state); - sbi->s_addr_per_block_bits = ilog2(EXT3_ADDR_PER_BLOCK(sb)); - sbi->s_desc_per_block_bits = ilog2(EXT3_DESC_PER_BLOCK(sb)); - for (i = 0; i < 4; i++) - sbi->s_hash_seed[i] = le32_to_cpu(es->s_hash_seed[i]); - sbi->s_def_hash_version = es->s_def_hash_version; - i = le32_to_cpu(es->s_flags); - if (i & EXT2_FLAGS_UNSIGNED_HASH) - sbi->s_hash_unsigned = 3; - else if ((i & EXT2_FLAGS_SIGNED_HASH) == 0) { -#ifdef __CHAR_UNSIGNED__ - es->s_flags |= cpu_to_le32(EXT2_FLAGS_UNSIGNED_HASH); - sbi->s_hash_unsigned = 3; -#else - es->s_flags |= cpu_to_le32(EXT2_FLAGS_SIGNED_HASH); -#endif - } - - if (sbi->s_blocks_per_group > blocksize * 8) { - ext3_msg(sb, KERN_ERR, - "#blocks per group too big: %lu", - sbi->s_blocks_per_group); - goto failed_mount; - } - if (sbi->s_frags_per_group > blocksize * 8) { - ext3_msg(sb, KERN_ERR, - "error: #fragments per group too big: %lu", - sbi->s_frags_per_group); - goto failed_mount; - } - if (sbi->s_inodes_per_group > blocksize * 8) { - ext3_msg(sb, KERN_ERR, - "error: #inodes per group too big: %lu", - sbi->s_inodes_per_group); - goto failed_mount; - } - - err = generic_check_addressable(sb->s_blocksize_bits, - le32_to_cpu(es->s_blocks_count)); - if (err) { - ext3_msg(sb, KERN_ERR, - "error: filesystem is too large to mount safely"); - if (sizeof(sector_t) < 8) - ext3_msg(sb, KERN_ERR, - "error: CONFIG_LBDAF not enabled"); - ret = err; - goto failed_mount; - } - - if (EXT3_BLOCKS_PER_GROUP(sb) == 0) - goto cantfind_ext3; - sbi->s_groups_count = ((le32_to_cpu(es->s_blocks_count) - - le32_to_cpu(es->s_first_data_block) - 1) - / EXT3_BLOCKS_PER_GROUP(sb)) + 1; - db_count = DIV_ROUND_UP(sbi->s_groups_count, EXT3_DESC_PER_BLOCK(sb)); - sbi->s_group_desc = kmalloc(db_count * sizeof (struct buffer_head *), - GFP_KERNEL); - if (sbi->s_group_desc == NULL) { - ext3_msg(sb, KERN_ERR, - "error: not enough memory"); - ret = -ENOMEM; - goto failed_mount; - } - - bgl_lock_init(sbi->s_blockgroup_lock); - - for (i = 0; i < db_count; i++) { - block = descriptor_loc(sb, logic_sb_block, i); - sbi->s_group_desc[i] = sb_bread(sb, block); - if (!sbi->s_group_desc[i]) { - ext3_msg(sb, KERN_ERR, - "error: can't read group descriptor %d", i); - db_count = i; - goto failed_mount2; - } - } - if (!ext3_check_descriptors (sb)) { - ext3_msg(sb, KERN_ERR, - "error: group descriptors corrupted"); - goto failed_mount2; - } - sbi->s_gdb_count = db_count; - get_random_bytes(&sbi->s_next_generation, sizeof(u32)); - spin_lock_init(&sbi->s_next_gen_lock); - - /* per fileystem reservation list head & lock */ - spin_lock_init(&sbi->s_rsv_window_lock); - sbi->s_rsv_window_root = RB_ROOT; - /* Add a single, static dummy reservation to the start of the - * reservation window list --- it gives us a placeholder for - * append-at-start-of-list which makes the allocation logic - * _much_ simpler. */ - sbi->s_rsv_window_head.rsv_start = EXT3_RESERVE_WINDOW_NOT_ALLOCATED; - sbi->s_rsv_window_head.rsv_end = EXT3_RESERVE_WINDOW_NOT_ALLOCATED; - sbi->s_rsv_window_head.rsv_alloc_hit = 0; - sbi->s_rsv_window_head.rsv_goal_size = 0; - ext3_rsv_window_add(sb, &sbi->s_rsv_window_head); - - /* - * set up enough so that it can read an inode - */ - sb->s_op = &ext3_sops; - sb->s_export_op = &ext3_export_ops; - sb->s_xattr = ext3_xattr_handlers; -#ifdef CONFIG_QUOTA - sb->s_qcop = &ext3_qctl_operations; - sb->dq_op = &ext3_quota_operations; - sb->s_quota_types = QTYPE_MASK_USR | QTYPE_MASK_GRP; -#endif - memcpy(sb->s_uuid, es->s_uuid, sizeof(es->s_uuid)); - INIT_LIST_HEAD(&sbi->s_orphan); /* unlinked but open files */ - mutex_init(&sbi->s_orphan_lock); - mutex_init(&sbi->s_resize_lock); - - sb->s_root = NULL; - - needs_recovery = (es->s_last_orphan != 0 || - EXT3_HAS_INCOMPAT_FEATURE(sb, - EXT3_FEATURE_INCOMPAT_RECOVER)); - - /* - * The first inode we look at is the journal inode. Don't try - * root first: it may be modified in the journal! - */ - if (!test_opt(sb, NOLOAD) && - EXT3_HAS_COMPAT_FEATURE(sb, EXT3_FEATURE_COMPAT_HAS_JOURNAL)) { - if (ext3_load_journal(sb, es, journal_devnum)) - goto failed_mount2; - } else if (journal_inum) { - if (ext3_create_journal(sb, es, journal_inum)) - goto failed_mount2; - } else { - if (!silent) - ext3_msg(sb, KERN_ERR, - "error: no journal found. " - "mounting ext3 over ext2?"); - goto failed_mount2; - } - err = percpu_counter_init(&sbi->s_freeblocks_counter, - ext3_count_free_blocks(sb), GFP_KERNEL); - if (!err) { - err = percpu_counter_init(&sbi->s_freeinodes_counter, - ext3_count_free_inodes(sb), GFP_KERNEL); - } - if (!err) { - err = percpu_counter_init(&sbi->s_dirs_counter, - ext3_count_dirs(sb), GFP_KERNEL); - } - if (err) { - ext3_msg(sb, KERN_ERR, "error: insufficient memory"); - ret = err; - goto failed_mount3; - } - - /* We have now updated the journal if required, so we can - * validate the data journaling mode. */ - switch (test_opt(sb, DATA_FLAGS)) { - case 0: - /* No mode set, assume a default based on the journal - capabilities: ORDERED_DATA if the journal can - cope, else JOURNAL_DATA */ - if (journal_check_available_features - (sbi->s_journal, 0, 0, JFS_FEATURE_INCOMPAT_REVOKE)) - set_opt(sbi->s_mount_opt, DEFAULT_DATA_MODE); - else - set_opt(sbi->s_mount_opt, JOURNAL_DATA); - break; - - case EXT3_MOUNT_ORDERED_DATA: - case EXT3_MOUNT_WRITEBACK_DATA: - if (!journal_check_available_features - (sbi->s_journal, 0, 0, JFS_FEATURE_INCOMPAT_REVOKE)) { - ext3_msg(sb, KERN_ERR, - "error: journal does not support " - "requested data journaling mode"); - goto failed_mount3; - } - default: - break; - } - - /* - * The journal_load will have done any necessary log recovery, - * so we can safely mount the rest of the filesystem now. - */ - - root = ext3_iget(sb, EXT3_ROOT_INO); - if (IS_ERR(root)) { - ext3_msg(sb, KERN_ERR, "error: get root inode failed"); - ret = PTR_ERR(root); - goto failed_mount3; - } - if (!S_ISDIR(root->i_mode) || !root->i_blocks || !root->i_size) { - iput(root); - ext3_msg(sb, KERN_ERR, "error: corrupt root inode, run e2fsck"); - goto failed_mount3; - } - sb->s_root = d_make_root(root); - if (!sb->s_root) { - ext3_msg(sb, KERN_ERR, "error: get root dentry failed"); - ret = -ENOMEM; - goto failed_mount3; - } - - if (ext3_setup_super(sb, es, sb->s_flags & MS_RDONLY)) - sb->s_flags |= MS_RDONLY; - - EXT3_SB(sb)->s_mount_state |= EXT3_ORPHAN_FS; - ext3_orphan_cleanup(sb, es); - EXT3_SB(sb)->s_mount_state &= ~EXT3_ORPHAN_FS; - if (needs_recovery) { - ext3_mark_recovery_complete(sb, es); - ext3_msg(sb, KERN_INFO, "recovery complete"); - } - ext3_msg(sb, KERN_INFO, "mounted filesystem with %s data mode", - test_opt(sb,DATA_FLAGS) == EXT3_MOUNT_JOURNAL_DATA ? "journal": - test_opt(sb,DATA_FLAGS) == EXT3_MOUNT_ORDERED_DATA ? "ordered": - "writeback"); - - return 0; - -cantfind_ext3: - if (!silent) - ext3_msg(sb, KERN_INFO, - "error: can't find ext3 filesystem on dev %s.", - sb->s_id); - goto failed_mount; - -failed_mount3: - percpu_counter_destroy(&sbi->s_freeblocks_counter); - percpu_counter_destroy(&sbi->s_freeinodes_counter); - percpu_counter_destroy(&sbi->s_dirs_counter); - journal_destroy(sbi->s_journal); -failed_mount2: - for (i = 0; i < db_count; i++) - brelse(sbi->s_group_desc[i]); - kfree(sbi->s_group_desc); -failed_mount: -#ifdef CONFIG_QUOTA - for (i = 0; i < EXT3_MAXQUOTAS; i++) - kfree(sbi->s_qf_names[i]); -#endif - ext3_blkdev_remove(sbi); - brelse(bh); -out_fail: - sb->s_fs_info = NULL; - kfree(sbi->s_blockgroup_lock); - kfree(sbi); - return ret; -} - -/* - * Setup any per-fs journal parameters now. We'll do this both on - * initial mount, once the journal has been initialised but before we've - * done any recovery; and again on any subsequent remount. - */ -static void ext3_init_journal_params(struct super_block *sb, journal_t *journal) -{ - struct ext3_sb_info *sbi = EXT3_SB(sb); - - if (sbi->s_commit_interval) - journal->j_commit_interval = sbi->s_commit_interval; - /* We could also set up an ext3-specific default for the commit - * interval here, but for now we'll just fall back to the jbd - * default. */ - - spin_lock(&journal->j_state_lock); - if (test_opt(sb, BARRIER)) - journal->j_flags |= JFS_BARRIER; - else - journal->j_flags &= ~JFS_BARRIER; - if (test_opt(sb, DATA_ERR_ABORT)) - journal->j_flags |= JFS_ABORT_ON_SYNCDATA_ERR; - else - journal->j_flags &= ~JFS_ABORT_ON_SYNCDATA_ERR; - spin_unlock(&journal->j_state_lock); -} - -static journal_t *ext3_get_journal(struct super_block *sb, - unsigned int journal_inum) -{ - struct inode *journal_inode; - journal_t *journal; - - /* First, test for the existence of a valid inode on disk. Bad - * things happen if we iget() an unused inode, as the subsequent - * iput() will try to delete it. */ - - journal_inode = ext3_iget(sb, journal_inum); - if (IS_ERR(journal_inode)) { - ext3_msg(sb, KERN_ERR, "error: no journal found"); - return NULL; - } - if (!journal_inode->i_nlink) { - make_bad_inode(journal_inode); - iput(journal_inode); - ext3_msg(sb, KERN_ERR, "error: journal inode is deleted"); - return NULL; - } - - jbd_debug(2, "Journal inode found at %p: %Ld bytes\n", - journal_inode, journal_inode->i_size); - if (!S_ISREG(journal_inode->i_mode)) { - ext3_msg(sb, KERN_ERR, "error: invalid journal inode"); - iput(journal_inode); - return NULL; - } - - journal = journal_init_inode(journal_inode); - if (!journal) { - ext3_msg(sb, KERN_ERR, "error: could not load journal inode"); - iput(journal_inode); - return NULL; - } - journal->j_private = sb; - ext3_init_journal_params(sb, journal); - return journal; -} - -static journal_t *ext3_get_dev_journal(struct super_block *sb, - dev_t j_dev) -{ - struct buffer_head * bh; - journal_t *journal; - ext3_fsblk_t start; - ext3_fsblk_t len; - int hblock, blocksize; - ext3_fsblk_t sb_block; - unsigned long offset; - struct ext3_super_block * es; - struct block_device *bdev; - - bdev = ext3_blkdev_get(j_dev, sb); - if (bdev == NULL) - return NULL; - - blocksize = sb->s_blocksize; - hblock = bdev_logical_block_size(bdev); - if (blocksize < hblock) { - ext3_msg(sb, KERN_ERR, - "error: blocksize too small for journal device"); - goto out_bdev; - } - - sb_block = EXT3_MIN_BLOCK_SIZE / blocksize; - offset = EXT3_MIN_BLOCK_SIZE % blocksize; - set_blocksize(bdev, blocksize); - if (!(bh = __bread(bdev, sb_block, blocksize))) { - ext3_msg(sb, KERN_ERR, "error: couldn't read superblock of " - "external journal"); - goto out_bdev; - } - - es = (struct ext3_super_block *) (bh->b_data + offset); - if ((le16_to_cpu(es->s_magic) != EXT3_SUPER_MAGIC) || - !(le32_to_cpu(es->s_feature_incompat) & - EXT3_FEATURE_INCOMPAT_JOURNAL_DEV)) { - ext3_msg(sb, KERN_ERR, "error: external journal has " - "bad superblock"); - brelse(bh); - goto out_bdev; - } - - if (memcmp(EXT3_SB(sb)->s_es->s_journal_uuid, es->s_uuid, 16)) { - ext3_msg(sb, KERN_ERR, "error: journal UUID does not match"); - brelse(bh); - goto out_bdev; - } - - len = le32_to_cpu(es->s_blocks_count); - start = sb_block + 1; - brelse(bh); /* we're done with the superblock */ - - journal = journal_init_dev(bdev, sb->s_bdev, - start, len, blocksize); - if (!journal) { - ext3_msg(sb, KERN_ERR, - "error: failed to create device journal"); - goto out_bdev; - } - journal->j_private = sb; - if (!bh_uptodate_or_lock(journal->j_sb_buffer)) { - if (bh_submit_read(journal->j_sb_buffer)) { - ext3_msg(sb, KERN_ERR, "I/O error on journal device"); - goto out_journal; - } - } - if (be32_to_cpu(journal->j_superblock->s_nr_users) != 1) { - ext3_msg(sb, KERN_ERR, - "error: external journal has more than one " - "user (unsupported) - %d", - be32_to_cpu(journal->j_superblock->s_nr_users)); - goto out_journal; - } - EXT3_SB(sb)->journal_bdev = bdev; - ext3_init_journal_params(sb, journal); - return journal; -out_journal: - journal_destroy(journal); -out_bdev: - ext3_blkdev_put(bdev); - return NULL; -} - -static int ext3_load_journal(struct super_block *sb, - struct ext3_super_block *es, - unsigned long journal_devnum) -{ - journal_t *journal; - unsigned int journal_inum = le32_to_cpu(es->s_journal_inum); - dev_t journal_dev; - int err = 0; - int really_read_only; - - if (journal_devnum && - journal_devnum != le32_to_cpu(es->s_journal_dev)) { - ext3_msg(sb, KERN_INFO, "external journal device major/minor " - "numbers have changed"); - journal_dev = new_decode_dev(journal_devnum); - } else - journal_dev = new_decode_dev(le32_to_cpu(es->s_journal_dev)); - - really_read_only = bdev_read_only(sb->s_bdev); - - /* - * Are we loading a blank journal or performing recovery after a - * crash? For recovery, we need to check in advance whether we - * can get read-write access to the device. - */ - - if (EXT3_HAS_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER)) { - if (sb->s_flags & MS_RDONLY) { - ext3_msg(sb, KERN_INFO, - "recovery required on readonly filesystem"); - if (really_read_only) { - ext3_msg(sb, KERN_ERR, "error: write access " - "unavailable, cannot proceed"); - return -EROFS; - } - ext3_msg(sb, KERN_INFO, - "write access will be enabled during recovery"); - } - } - - if (journal_inum && journal_dev) { - ext3_msg(sb, KERN_ERR, "error: filesystem has both journal " - "and inode journals"); - return -EINVAL; - } - - if (journal_inum) { - if (!(journal = ext3_get_journal(sb, journal_inum))) - return -EINVAL; - } else { - if (!(journal = ext3_get_dev_journal(sb, journal_dev))) - return -EINVAL; - } - - if (!(journal->j_flags & JFS_BARRIER)) - printk(KERN_INFO "EXT3-fs: barriers not enabled\n"); - - if (!really_read_only && test_opt(sb, UPDATE_JOURNAL)) { - err = journal_update_format(journal); - if (err) { - ext3_msg(sb, KERN_ERR, "error updating journal"); - journal_destroy(journal); - return err; - } - } - - if (!EXT3_HAS_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER)) - err = journal_wipe(journal, !really_read_only); - if (!err) - err = journal_load(journal); - - if (err) { - ext3_msg(sb, KERN_ERR, "error loading journal"); - journal_destroy(journal); - return err; - } - - EXT3_SB(sb)->s_journal = journal; - ext3_clear_journal_err(sb, es); - - if (!really_read_only && journal_devnum && - journal_devnum != le32_to_cpu(es->s_journal_dev)) { - es->s_journal_dev = cpu_to_le32(journal_devnum); - - /* Make sure we flush the recovery flag to disk. */ - ext3_commit_super(sb, es, 1); - } - - return 0; -} - -static int ext3_create_journal(struct super_block *sb, - struct ext3_super_block *es, - unsigned int journal_inum) -{ - journal_t *journal; - int err; - - if (sb->s_flags & MS_RDONLY) { - ext3_msg(sb, KERN_ERR, - "error: readonly filesystem when trying to " - "create journal"); - return -EROFS; - } - - journal = ext3_get_journal(sb, journal_inum); - if (!journal) - return -EINVAL; - - ext3_msg(sb, KERN_INFO, "creating new journal on inode %u", - journal_inum); - - err = journal_create(journal); - if (err) { - ext3_msg(sb, KERN_ERR, "error creating journal"); - journal_destroy(journal); - return -EIO; - } - - EXT3_SB(sb)->s_journal = journal; - - ext3_update_dynamic_rev(sb); - EXT3_SET_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER); - EXT3_SET_COMPAT_FEATURE(sb, EXT3_FEATURE_COMPAT_HAS_JOURNAL); - - es->s_journal_inum = cpu_to_le32(journal_inum); - - /* Make sure we flush the recovery flag to disk. */ - ext3_commit_super(sb, es, 1); - - return 0; -} - -static int ext3_commit_super(struct super_block *sb, - struct ext3_super_block *es, - int sync) -{ - struct buffer_head *sbh = EXT3_SB(sb)->s_sbh; - int error = 0; - - if (!sbh) - return error; - - if (buffer_write_io_error(sbh)) { - /* - * Oh, dear. A previous attempt to write the - * superblock failed. This could happen because the - * USB device was yanked out. Or it could happen to - * be a transient write error and maybe the block will - * be remapped. Nothing we can do but to retry the - * write and hope for the best. - */ - ext3_msg(sb, KERN_ERR, "previous I/O error to " - "superblock detected"); - clear_buffer_write_io_error(sbh); - set_buffer_uptodate(sbh); - } - /* - * If the file system is mounted read-only, don't update the - * superblock write time. This avoids updating the superblock - * write time when we are mounting the root file system - * read/only but we need to replay the journal; at that point, - * for people who are east of GMT and who make their clock - * tick in localtime for Windows bug-for-bug compatibility, - * the clock is set in the future, and this will cause e2fsck - * to complain and force a full file system check. - */ - if (!(sb->s_flags & MS_RDONLY)) - es->s_wtime = cpu_to_le32(get_seconds()); - es->s_free_blocks_count = cpu_to_le32(ext3_count_free_blocks(sb)); - es->s_free_inodes_count = cpu_to_le32(ext3_count_free_inodes(sb)); - BUFFER_TRACE(sbh, "marking dirty"); - mark_buffer_dirty(sbh); - if (sync) { - error = sync_dirty_buffer(sbh); - if (buffer_write_io_error(sbh)) { - ext3_msg(sb, KERN_ERR, "I/O error while writing " - "superblock"); - clear_buffer_write_io_error(sbh); - set_buffer_uptodate(sbh); - } - } - return error; -} - - -/* - * Have we just finished recovery? If so, and if we are mounting (or - * remounting) the filesystem readonly, then we will end up with a - * consistent fs on disk. Record that fact. - */ -static void ext3_mark_recovery_complete(struct super_block * sb, - struct ext3_super_block * es) -{ - journal_t *journal = EXT3_SB(sb)->s_journal; - - journal_lock_updates(journal); - if (journal_flush(journal) < 0) - goto out; - - if (EXT3_HAS_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER) && - sb->s_flags & MS_RDONLY) { - EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER); - ext3_commit_super(sb, es, 1); - } - -out: - journal_unlock_updates(journal); -} - -/* - * If we are mounting (or read-write remounting) a filesystem whose journal - * has recorded an error from a previous lifetime, move that error to the - * main filesystem now. - */ -static void ext3_clear_journal_err(struct super_block *sb, - struct ext3_super_block *es) -{ - journal_t *journal; - int j_errno; - const char *errstr; - - journal = EXT3_SB(sb)->s_journal; - - /* - * Now check for any error status which may have been recorded in the - * journal by a prior ext3_error() or ext3_abort() - */ - - j_errno = journal_errno(journal); - if (j_errno) { - char nbuf[16]; - - errstr = ext3_decode_error(sb, j_errno, nbuf); - ext3_warning(sb, __func__, "Filesystem error recorded " - "from previous mount: %s", errstr); - ext3_warning(sb, __func__, "Marking fs in need of " - "filesystem check."); - - EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS; - es->s_state |= cpu_to_le16(EXT3_ERROR_FS); - ext3_commit_super (sb, es, 1); - - journal_clear_err(journal); - } -} - -/* - * Force the running and committing transactions to commit, - * and wait on the commit. - */ -int ext3_force_commit(struct super_block *sb) -{ - journal_t *journal; - int ret; - - if (sb->s_flags & MS_RDONLY) - return 0; - - journal = EXT3_SB(sb)->s_journal; - ret = ext3_journal_force_commit(journal); - return ret; -} - -static int ext3_sync_fs(struct super_block *sb, int wait) -{ - tid_t target; - - trace_ext3_sync_fs(sb, wait); - /* - * Writeback quota in non-journalled quota case - journalled quota has - * no dirty dquots - */ - dquot_writeback_dquots(sb, -1); - if (journal_start_commit(EXT3_SB(sb)->s_journal, &target)) { - if (wait) - log_wait_commit(EXT3_SB(sb)->s_journal, target); - } - return 0; -} - -/* - * LVM calls this function before a (read-only) snapshot is created. This - * gives us a chance to flush the journal completely and mark the fs clean. - */ -static int ext3_freeze(struct super_block *sb) -{ - int error = 0; - journal_t *journal; - - if (!(sb->s_flags & MS_RDONLY)) { - journal = EXT3_SB(sb)->s_journal; - - /* Now we set up the journal barrier. */ - journal_lock_updates(journal); - - /* - * We don't want to clear needs_recovery flag when we failed - * to flush the journal. - */ - error = journal_flush(journal); - if (error < 0) - goto out; - - /* Journal blocked and flushed, clear needs_recovery flag. */ - EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER); - error = ext3_commit_super(sb, EXT3_SB(sb)->s_es, 1); - if (error) - goto out; - } - return 0; - -out: - journal_unlock_updates(journal); - return error; -} - -/* - * Called by LVM after the snapshot is done. We need to reset the RECOVER - * flag here, even though the filesystem is not technically dirty yet. - */ -static int ext3_unfreeze(struct super_block *sb) -{ - if (!(sb->s_flags & MS_RDONLY)) { - /* Reser the needs_recovery flag before the fs is unlocked. */ - EXT3_SET_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER); - ext3_commit_super(sb, EXT3_SB(sb)->s_es, 1); - journal_unlock_updates(EXT3_SB(sb)->s_journal); - } - return 0; -} - -static int ext3_remount (struct super_block * sb, int * flags, char * data) -{ - struct ext3_super_block * es; - struct ext3_sb_info *sbi = EXT3_SB(sb); - ext3_fsblk_t n_blocks_count = 0; - unsigned long old_sb_flags; - struct ext3_mount_options old_opts; - int enable_quota = 0; - int err; -#ifdef CONFIG_QUOTA - int i; -#endif - - sync_filesystem(sb); - - /* Store the original options */ - old_sb_flags = sb->s_flags; - old_opts.s_mount_opt = sbi->s_mount_opt; - old_opts.s_resuid = sbi->s_resuid; - old_opts.s_resgid = sbi->s_resgid; - old_opts.s_commit_interval = sbi->s_commit_interval; -#ifdef CONFIG_QUOTA - old_opts.s_jquota_fmt = sbi->s_jquota_fmt; - for (i = 0; i < EXT3_MAXQUOTAS; i++) - if (sbi->s_qf_names[i]) { - old_opts.s_qf_names[i] = kstrdup(sbi->s_qf_names[i], - GFP_KERNEL); - if (!old_opts.s_qf_names[i]) { - int j; - - for (j = 0; j < i; j++) - kfree(old_opts.s_qf_names[j]); - return -ENOMEM; - } - } else - old_opts.s_qf_names[i] = NULL; -#endif - - /* - * Allow the "check" option to be passed as a remount option. - */ - if (!parse_options(data, sb, NULL, NULL, &n_blocks_count, 1)) { - err = -EINVAL; - goto restore_opts; - } - - if (test_opt(sb, ABORT)) - ext3_abort(sb, __func__, "Abort forced by user"); - - sb->s_flags = (sb->s_flags & ~MS_POSIXACL) | - (test_opt(sb, POSIX_ACL) ? MS_POSIXACL : 0); - - es = sbi->s_es; - - ext3_init_journal_params(sb, sbi->s_journal); - - if ((*flags & MS_RDONLY) != (sb->s_flags & MS_RDONLY) || - n_blocks_count > le32_to_cpu(es->s_blocks_count)) { - if (test_opt(sb, ABORT)) { - err = -EROFS; - goto restore_opts; - } - - if (*flags & MS_RDONLY) { - err = dquot_suspend(sb, -1); - if (err < 0) - goto restore_opts; - - /* - * First of all, the unconditional stuff we have to do - * to disable replay of the journal when we next remount - */ - sb->s_flags |= MS_RDONLY; - - /* - * OK, test if we are remounting a valid rw partition - * readonly, and if so set the rdonly flag and then - * mark the partition as valid again. - */ - if (!(es->s_state & cpu_to_le16(EXT3_VALID_FS)) && - (sbi->s_mount_state & EXT3_VALID_FS)) - es->s_state = cpu_to_le16(sbi->s_mount_state); - - ext3_mark_recovery_complete(sb, es); - } else { - __le32 ret; - if ((ret = EXT3_HAS_RO_COMPAT_FEATURE(sb, - ~EXT3_FEATURE_RO_COMPAT_SUPP))) { - ext3_msg(sb, KERN_WARNING, - "warning: couldn't remount RDWR " - "because of unsupported optional " - "features (%x)", le32_to_cpu(ret)); - err = -EROFS; - goto restore_opts; - } - - /* - * If we have an unprocessed orphan list hanging - * around from a previously readonly bdev mount, - * require a full umount & mount for now. - */ - if (es->s_last_orphan) { - ext3_msg(sb, KERN_WARNING, "warning: couldn't " - "remount RDWR because of unprocessed " - "orphan inode list. Please " - "umount & mount instead."); - err = -EINVAL; - goto restore_opts; - } - - /* - * Mounting a RDONLY partition read-write, so reread - * and store the current valid flag. (It may have - * been changed by e2fsck since we originally mounted - * the partition.) - */ - ext3_clear_journal_err(sb, es); - sbi->s_mount_state = le16_to_cpu(es->s_state); - if ((err = ext3_group_extend(sb, es, n_blocks_count))) - goto restore_opts; - if (!ext3_setup_super (sb, es, 0)) - sb->s_flags &= ~MS_RDONLY; - enable_quota = 1; - } - } -#ifdef CONFIG_QUOTA - /* Release old quota file names */ - for (i = 0; i < EXT3_MAXQUOTAS; i++) - kfree(old_opts.s_qf_names[i]); -#endif - if (enable_quota) - dquot_resume(sb, -1); - return 0; -restore_opts: - sb->s_flags = old_sb_flags; - sbi->s_mount_opt = old_opts.s_mount_opt; - sbi->s_resuid = old_opts.s_resuid; - sbi->s_resgid = old_opts.s_resgid; - sbi->s_commit_interval = old_opts.s_commit_interval; -#ifdef CONFIG_QUOTA - sbi->s_jquota_fmt = old_opts.s_jquota_fmt; - for (i = 0; i < EXT3_MAXQUOTAS; i++) { - kfree(sbi->s_qf_names[i]); - sbi->s_qf_names[i] = old_opts.s_qf_names[i]; - } -#endif - return err; -} - -static int ext3_statfs (struct dentry * dentry, struct kstatfs * buf) -{ - struct super_block *sb = dentry->d_sb; - struct ext3_sb_info *sbi = EXT3_SB(sb); - struct ext3_super_block *es = sbi->s_es; - u64 fsid; - - if (test_opt(sb, MINIX_DF)) { - sbi->s_overhead_last = 0; - } else if (sbi->s_blocks_last != le32_to_cpu(es->s_blocks_count)) { - unsigned long ngroups = sbi->s_groups_count, i; - ext3_fsblk_t overhead = 0; - smp_rmb(); - - /* - * Compute the overhead (FS structures). This is constant - * for a given filesystem unless the number of block groups - * changes so we cache the previous value until it does. - */ - - /* - * All of the blocks before first_data_block are - * overhead - */ - overhead = le32_to_cpu(es->s_first_data_block); - - /* - * Add the overhead attributed to the superblock and - * block group descriptors. If the sparse superblocks - * feature is turned on, then not all groups have this. - */ - for (i = 0; i < ngroups; i++) { - overhead += ext3_bg_has_super(sb, i) + - ext3_bg_num_gdb(sb, i); - cond_resched(); - } - - /* - * Every block group has an inode bitmap, a block - * bitmap, and an inode table. - */ - overhead += ngroups * (2 + sbi->s_itb_per_group); - - /* Add the internal journal blocks as well */ - if (sbi->s_journal && !sbi->journal_bdev) - overhead += sbi->s_journal->j_maxlen; - - sbi->s_overhead_last = overhead; - smp_wmb(); - sbi->s_blocks_last = le32_to_cpu(es->s_blocks_count); - } - - buf->f_type = EXT3_SUPER_MAGIC; - buf->f_bsize = sb->s_blocksize; - buf->f_blocks = le32_to_cpu(es->s_blocks_count) - sbi->s_overhead_last; - buf->f_bfree = percpu_counter_sum_positive(&sbi->s_freeblocks_counter); - buf->f_bavail = buf->f_bfree - le32_to_cpu(es->s_r_blocks_count); - if (buf->f_bfree < le32_to_cpu(es->s_r_blocks_count)) - buf->f_bavail = 0; - buf->f_files = le32_to_cpu(es->s_inodes_count); - buf->f_ffree = percpu_counter_sum_positive(&sbi->s_freeinodes_counter); - buf->f_namelen = EXT3_NAME_LEN; - fsid = le64_to_cpup((void *)es->s_uuid) ^ - le64_to_cpup((void *)es->s_uuid + sizeof(u64)); - buf->f_fsid.val[0] = fsid & 0xFFFFFFFFUL; - buf->f_fsid.val[1] = (fsid >> 32) & 0xFFFFFFFFUL; - return 0; -} - -/* Helper function for writing quotas on sync - we need to start transaction before quota file - * is locked for write. Otherwise the are possible deadlocks: - * Process 1 Process 2 - * ext3_create() quota_sync() - * journal_start() write_dquot() - * dquot_initialize() down(dqio_mutex) - * down(dqio_mutex) journal_start() - * - */ - -#ifdef CONFIG_QUOTA - -static inline struct inode *dquot_to_inode(struct dquot *dquot) -{ - return sb_dqopt(dquot->dq_sb)->files[dquot->dq_id.type]; -} - -static int ext3_write_dquot(struct dquot *dquot) -{ - int ret, err; - handle_t *handle; - struct inode *inode; - - inode = dquot_to_inode(dquot); - handle = ext3_journal_start(inode, - EXT3_QUOTA_TRANS_BLOCKS(dquot->dq_sb)); - if (IS_ERR(handle)) - return PTR_ERR(handle); - ret = dquot_commit(dquot); - err = ext3_journal_stop(handle); - if (!ret) - ret = err; - return ret; -} - -static int ext3_acquire_dquot(struct dquot *dquot) -{ - int ret, err; - handle_t *handle; - - handle = ext3_journal_start(dquot_to_inode(dquot), - EXT3_QUOTA_INIT_BLOCKS(dquot->dq_sb)); - if (IS_ERR(handle)) - return PTR_ERR(handle); - ret = dquot_acquire(dquot); - err = ext3_journal_stop(handle); - if (!ret) - ret = err; - return ret; -} - -static int ext3_release_dquot(struct dquot *dquot) -{ - int ret, err; - handle_t *handle; - - handle = ext3_journal_start(dquot_to_inode(dquot), - EXT3_QUOTA_DEL_BLOCKS(dquot->dq_sb)); - if (IS_ERR(handle)) { - /* Release dquot anyway to avoid endless cycle in dqput() */ - dquot_release(dquot); - return PTR_ERR(handle); - } - ret = dquot_release(dquot); - err = ext3_journal_stop(handle); - if (!ret) - ret = err; - return ret; -} - -static int ext3_mark_dquot_dirty(struct dquot *dquot) -{ - /* Are we journaling quotas? */ - if (EXT3_SB(dquot->dq_sb)->s_qf_names[USRQUOTA] || - EXT3_SB(dquot->dq_sb)->s_qf_names[GRPQUOTA]) { - dquot_mark_dquot_dirty(dquot); - return ext3_write_dquot(dquot); - } else { - return dquot_mark_dquot_dirty(dquot); - } -} - -static int ext3_write_info(struct super_block *sb, int type) -{ - int ret, err; - handle_t *handle; - - /* Data block + inode block */ - handle = ext3_journal_start(d_inode(sb->s_root), 2); - if (IS_ERR(handle)) - return PTR_ERR(handle); - ret = dquot_commit_info(sb, type); - err = ext3_journal_stop(handle); - if (!ret) - ret = err; - return ret; -} - -/* - * Turn on quotas during mount time - we need to find - * the quota file and such... - */ -static int ext3_quota_on_mount(struct super_block *sb, int type) -{ - return dquot_quota_on_mount(sb, EXT3_SB(sb)->s_qf_names[type], - EXT3_SB(sb)->s_jquota_fmt, type); -} - -/* - * Standard function to be called on quota_on - */ -static int ext3_quota_on(struct super_block *sb, int type, int format_id, - struct path *path) -{ - int err; - - if (!test_opt(sb, QUOTA)) - return -EINVAL; - - /* Quotafile not on the same filesystem? */ - if (path->dentry->d_sb != sb) - return -EXDEV; - /* Journaling quota? */ - if (EXT3_SB(sb)->s_qf_names[type]) { - /* Quotafile not of fs root? */ - if (path->dentry->d_parent != sb->s_root) - ext3_msg(sb, KERN_WARNING, - "warning: Quota file not on filesystem root. " - "Journaled quota will not work."); - } - - /* - * When we journal data on quota file, we have to flush journal to see - * all updates to the file when we bypass pagecache... - */ - if (ext3_should_journal_data(d_inode(path->dentry))) { - /* - * We don't need to lock updates but journal_flush() could - * otherwise be livelocked... - */ - journal_lock_updates(EXT3_SB(sb)->s_journal); - err = journal_flush(EXT3_SB(sb)->s_journal); - journal_unlock_updates(EXT3_SB(sb)->s_journal); - if (err) - return err; - } - - return dquot_quota_on(sb, type, format_id, path); -} - -/* Read data from quotafile - avoid pagecache and such because we cannot afford - * acquiring the locks... As quota files are never truncated and quota code - * itself serializes the operations (and no one else should touch the files) - * we don't have to be afraid of races */ -static ssize_t ext3_quota_read(struct super_block *sb, int type, char *data, - size_t len, loff_t off) -{ - struct inode *inode = sb_dqopt(sb)->files[type]; - sector_t blk = off >> EXT3_BLOCK_SIZE_BITS(sb); - int err = 0; - int offset = off & (sb->s_blocksize - 1); - int tocopy; - size_t toread; - struct buffer_head *bh; - loff_t i_size = i_size_read(inode); - - if (off > i_size) - return 0; - if (off+len > i_size) - len = i_size-off; - toread = len; - while (toread > 0) { - tocopy = sb->s_blocksize - offset < toread ? - sb->s_blocksize - offset : toread; - bh = ext3_bread(NULL, inode, blk, 0, &err); - if (err) - return err; - if (!bh) /* A hole? */ - memset(data, 0, tocopy); - else - memcpy(data, bh->b_data+offset, tocopy); - brelse(bh); - offset = 0; - toread -= tocopy; - data += tocopy; - blk++; - } - return len; -} - -/* Write to quotafile (we know the transaction is already started and has - * enough credits) */ -static ssize_t ext3_quota_write(struct super_block *sb, int type, - const char *data, size_t len, loff_t off) -{ - struct inode *inode = sb_dqopt(sb)->files[type]; - sector_t blk = off >> EXT3_BLOCK_SIZE_BITS(sb); - int err = 0; - int offset = off & (sb->s_blocksize - 1); - int journal_quota = EXT3_SB(sb)->s_qf_names[type] != NULL; - struct buffer_head *bh; - handle_t *handle = journal_current_handle(); - - if (!handle) { - ext3_msg(sb, KERN_WARNING, - "warning: quota write (off=%llu, len=%llu)" - " cancelled because transaction is not started.", - (unsigned long long)off, (unsigned long long)len); - return -EIO; - } - - /* - * Since we account only one data block in transaction credits, - * then it is impossible to cross a block boundary. - */ - if (sb->s_blocksize - offset < len) { - ext3_msg(sb, KERN_WARNING, "Quota write (off=%llu, len=%llu)" - " cancelled because not block aligned", - (unsigned long long)off, (unsigned long long)len); - return -EIO; - } - bh = ext3_bread(handle, inode, blk, 1, &err); - if (!bh) - goto out; - if (journal_quota) { - err = ext3_journal_get_write_access(handle, bh); - if (err) { - brelse(bh); - goto out; - } - } - lock_buffer(bh); - memcpy(bh->b_data+offset, data, len); - flush_dcache_page(bh->b_page); - unlock_buffer(bh); - if (journal_quota) - err = ext3_journal_dirty_metadata(handle, bh); - else { - /* Always do at least ordered writes for quotas */ - err = ext3_journal_dirty_data(handle, bh); - mark_buffer_dirty(bh); - } - brelse(bh); -out: - if (err) - return err; - if (inode->i_size < off + len) { - i_size_write(inode, off + len); - EXT3_I(inode)->i_disksize = inode->i_size; - } - inode->i_version++; - inode->i_mtime = inode->i_ctime = CURRENT_TIME; - ext3_mark_inode_dirty(handle, inode); - return len; -} - -#endif - -static struct dentry *ext3_mount(struct file_system_type *fs_type, - int flags, const char *dev_name, void *data) -{ - return mount_bdev(fs_type, flags, dev_name, data, ext3_fill_super); -} - -static struct file_system_type ext3_fs_type = { - .owner = THIS_MODULE, - .name = "ext3", - .mount = ext3_mount, - .kill_sb = kill_block_super, - .fs_flags = FS_REQUIRES_DEV, -}; -MODULE_ALIAS_FS("ext3"); - -static int __init init_ext3_fs(void) -{ - int err = init_ext3_xattr(); - if (err) - return err; - err = init_inodecache(); - if (err) - goto out1; - err = register_filesystem(&ext3_fs_type); - if (err) - goto out; - return 0; -out: - destroy_inodecache(); -out1: - exit_ext3_xattr(); - return err; -} - -static void __exit exit_ext3_fs(void) -{ - unregister_filesystem(&ext3_fs_type); - destroy_inodecache(); - exit_ext3_xattr(); -} - -MODULE_AUTHOR("Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, Theodore Ts'o and others"); -MODULE_DESCRIPTION("Second Extended Filesystem with journaling extensions"); -MODULE_LICENSE("GPL"); -module_init(init_ext3_fs) -module_exit(exit_ext3_fs) diff --git a/fs/ext3/symlink.c b/fs/ext3/symlink.c deleted file mode 100644 index c08c59094ae61f..00000000000000 --- a/fs/ext3/symlink.c +++ /dev/null @@ -1,46 +0,0 @@ -/* - * linux/fs/ext3/symlink.c - * - * Only fast symlinks left here - the rest is done by generic code. AV, 1999 - * - * Copyright (C) 1992, 1993, 1994, 1995 - * Remy Card (card@masi.ibp.fr) - * Laboratoire MASI - Institut Blaise Pascal - * Universite Pierre et Marie Curie (Paris VI) - * - * from - * - * linux/fs/minix/symlink.c - * - * Copyright (C) 1991, 1992 Linus Torvalds - * - * ext3 symlink handling code - */ - -#include "ext3.h" -#include "xattr.h" - -const struct inode_operations ext3_symlink_inode_operations = { - .readlink = generic_readlink, - .follow_link = page_follow_link_light, - .put_link = page_put_link, - .setattr = ext3_setattr, -#ifdef CONFIG_EXT3_FS_XATTR - .setxattr = generic_setxattr, - .getxattr = generic_getxattr, - .listxattr = ext3_listxattr, - .removexattr = generic_removexattr, -#endif -}; - -const struct inode_operations ext3_fast_symlink_inode_operations = { - .readlink = generic_readlink, - .follow_link = simple_follow_link, - .setattr = ext3_setattr, -#ifdef CONFIG_EXT3_FS_XATTR - .setxattr = generic_setxattr, - .getxattr = generic_getxattr, - .listxattr = ext3_listxattr, - .removexattr = generic_removexattr, -#endif -}; diff --git a/fs/ext3/xattr.c b/fs/ext3/xattr.c deleted file mode 100644 index 7cf36501ccf46c..00000000000000 --- a/fs/ext3/xattr.c +++ /dev/null @@ -1,1330 +0,0 @@ -/* - * linux/fs/ext3/xattr.c - * - * Copyright (C) 2001-2003 Andreas Gruenbacher, - * - * Fix by Harrison Xing . - * Ext3 code with a lot of help from Eric Jarman . - * Extended attributes for symlinks and special files added per - * suggestion of Luka Renko . - * xattr consolidation Copyright (c) 2004 James Morris , - * Red Hat Inc. - * ea-in-inode support by Alex Tomas aka bzzz - * and Andreas Gruenbacher . - */ - -/* - * Extended attributes are stored directly in inodes (on file systems with - * inodes bigger than 128 bytes) and on additional disk blocks. The i_file_acl - * field contains the block number if an inode uses an additional block. All - * attributes must fit in the inode and one additional block. Blocks that - * contain the identical set of attributes may be shared among several inodes. - * Identical blocks are detected by keeping a cache of blocks that have - * recently been accessed. - * - * The attributes in inodes and on blocks have a different header; the entries - * are stored in the same format: - * - * +------------------+ - * | header | - * | entry 1 | | - * | entry 2 | | growing downwards - * | entry 3 | v - * | four null bytes | - * | . . . | - * | value 1 | ^ - * | value 3 | | growing upwards - * | value 2 | | - * +------------------+ - * - * The header is followed by multiple entry descriptors. In disk blocks, the - * entry descriptors are kept sorted. In inodes, they are unsorted. The - * attribute values are aligned to the end of the block in no specific order. - * - * Locking strategy - * ---------------- - * EXT3_I(inode)->i_file_acl is protected by EXT3_I(inode)->xattr_sem. - * EA blocks are only changed if they are exclusive to an inode, so - * holding xattr_sem also means that nothing but the EA block's reference - * count can change. Multiple writers to the same block are synchronized - * by the buffer lock. - */ - -#include "ext3.h" -#include -#include -#include "xattr.h" -#include "acl.h" - -#define BHDR(bh) ((struct ext3_xattr_header *)((bh)->b_data)) -#define ENTRY(ptr) ((struct ext3_xattr_entry *)(ptr)) -#define BFIRST(bh) ENTRY(BHDR(bh)+1) -#define IS_LAST_ENTRY(entry) (*(__u32 *)(entry) == 0) - -#define IHDR(inode, raw_inode) \ - ((struct ext3_xattr_ibody_header *) \ - ((void *)raw_inode + \ - EXT3_GOOD_OLD_INODE_SIZE + \ - EXT3_I(inode)->i_extra_isize)) -#define IFIRST(hdr) ((struct ext3_xattr_entry *)((hdr)+1)) - -#ifdef EXT3_XATTR_DEBUG -# define ea_idebug(inode, f...) do { \ - printk(KERN_DEBUG "inode %s:%lu: ", \ - inode->i_sb->s_id, inode->i_ino); \ - printk(f); \ - printk("\n"); \ - } while (0) -# define ea_bdebug(bh, f...) do { \ - char b[BDEVNAME_SIZE]; \ - printk(KERN_DEBUG "block %s:%lu: ", \ - bdevname(bh->b_bdev, b), \ - (unsigned long) bh->b_blocknr); \ - printk(f); \ - printk("\n"); \ - } while (0) -#else -# define ea_idebug(f...) -# define ea_bdebug(f...) -#endif - -static void ext3_xattr_cache_insert(struct buffer_head *); -static struct buffer_head *ext3_xattr_cache_find(struct inode *, - struct ext3_xattr_header *, - struct mb_cache_entry **); -static void ext3_xattr_rehash(struct ext3_xattr_header *, - struct ext3_xattr_entry *); -static int ext3_xattr_list(struct dentry *dentry, char *buffer, - size_t buffer_size); - -static struct mb_cache *ext3_xattr_cache; - -static const struct xattr_handler *ext3_xattr_handler_map[] = { - [EXT3_XATTR_INDEX_USER] = &ext3_xattr_user_handler, -#ifdef CONFIG_EXT3_FS_POSIX_ACL - [EXT3_XATTR_INDEX_POSIX_ACL_ACCESS] = &posix_acl_access_xattr_handler, - [EXT3_XATTR_INDEX_POSIX_ACL_DEFAULT] = &posix_acl_default_xattr_handler, -#endif - [EXT3_XATTR_INDEX_TRUSTED] = &ext3_xattr_trusted_handler, -#ifdef CONFIG_EXT3_FS_SECURITY - [EXT3_XATTR_INDEX_SECURITY] = &ext3_xattr_security_handler, -#endif -}; - -const struct xattr_handler *ext3_xattr_handlers[] = { - &ext3_xattr_user_handler, - &ext3_xattr_trusted_handler, -#ifdef CONFIG_EXT3_FS_POSIX_ACL - &posix_acl_access_xattr_handler, - &posix_acl_default_xattr_handler, -#endif -#ifdef CONFIG_EXT3_FS_SECURITY - &ext3_xattr_security_handler, -#endif - NULL -}; - -static inline const struct xattr_handler * -ext3_xattr_handler(int name_index) -{ - const struct xattr_handler *handler = NULL; - - if (name_index > 0 && name_index < ARRAY_SIZE(ext3_xattr_handler_map)) - handler = ext3_xattr_handler_map[name_index]; - return handler; -} - -/* - * Inode operation listxattr() - * - * d_inode(dentry)->i_mutex: don't care - */ -ssize_t -ext3_listxattr(struct dentry *dentry, char *buffer, size_t size) -{ - return ext3_xattr_list(dentry, buffer, size); -} - -static int -ext3_xattr_check_names(struct ext3_xattr_entry *entry, void *end) -{ - while (!IS_LAST_ENTRY(entry)) { - struct ext3_xattr_entry *next = EXT3_XATTR_NEXT(entry); - if ((void *)next >= end) - return -EIO; - entry = next; - } - return 0; -} - -static inline int -ext3_xattr_check_block(struct buffer_head *bh) -{ - int error; - - if (BHDR(bh)->h_magic != cpu_to_le32(EXT3_XATTR_MAGIC) || - BHDR(bh)->h_blocks != cpu_to_le32(1)) - return -EIO; - error = ext3_xattr_check_names(BFIRST(bh), bh->b_data + bh->b_size); - return error; -} - -static inline int -ext3_xattr_check_entry(struct ext3_xattr_entry *entry, size_t size) -{ - size_t value_size = le32_to_cpu(entry->e_value_size); - - if (entry->e_value_block != 0 || value_size > size || - le16_to_cpu(entry->e_value_offs) + value_size > size) - return -EIO; - return 0; -} - -static int -ext3_xattr_find_entry(struct ext3_xattr_entry **pentry, int name_index, - const char *name, size_t size, int sorted) -{ - struct ext3_xattr_entry *entry; - size_t name_len; - int cmp = 1; - - if (name == NULL) - return -EINVAL; - name_len = strlen(name); - entry = *pentry; - for (; !IS_LAST_ENTRY(entry); entry = EXT3_XATTR_NEXT(entry)) { - cmp = name_index - entry->e_name_index; - if (!cmp) - cmp = name_len - entry->e_name_len; - if (!cmp) - cmp = memcmp(name, entry->e_name, name_len); - if (cmp <= 0 && (sorted || cmp == 0)) - break; - } - *pentry = entry; - if (!cmp && ext3_xattr_check_entry(entry, size)) - return -EIO; - return cmp ? -ENODATA : 0; -} - -static int -ext3_xattr_block_get(struct inode *inode, int name_index, const char *name, - void *buffer, size_t buffer_size) -{ - struct buffer_head *bh = NULL; - struct ext3_xattr_entry *entry; - size_t size; - int error; - - ea_idebug(inode, "name=%d.%s, buffer=%p, buffer_size=%ld", - name_index, name, buffer, (long)buffer_size); - - error = -ENODATA; - if (!EXT3_I(inode)->i_file_acl) - goto cleanup; - ea_idebug(inode, "reading block %u", EXT3_I(inode)->i_file_acl); - bh = sb_bread(inode->i_sb, EXT3_I(inode)->i_file_acl); - if (!bh) - goto cleanup; - ea_bdebug(bh, "b_count=%d, refcount=%d", - atomic_read(&(bh->b_count)), le32_to_cpu(BHDR(bh)->h_refcount)); - if (ext3_xattr_check_block(bh)) { -bad_block: ext3_error(inode->i_sb, __func__, - "inode %lu: bad block "E3FSBLK, inode->i_ino, - EXT3_I(inode)->i_file_acl); - error = -EIO; - goto cleanup; - } - ext3_xattr_cache_insert(bh); - entry = BFIRST(bh); - error = ext3_xattr_find_entry(&entry, name_index, name, bh->b_size, 1); - if (error == -EIO) - goto bad_block; - if (error) - goto cleanup; - size = le32_to_cpu(entry->e_value_size); - if (buffer) { - error = -ERANGE; - if (size > buffer_size) - goto cleanup; - memcpy(buffer, bh->b_data + le16_to_cpu(entry->e_value_offs), - size); - } - error = size; - -cleanup: - brelse(bh); - return error; -} - -static int -ext3_xattr_ibody_get(struct inode *inode, int name_index, const char *name, - void *buffer, size_t buffer_size) -{ - struct ext3_xattr_ibody_header *header; - struct ext3_xattr_entry *entry; - struct ext3_inode *raw_inode; - struct ext3_iloc iloc; - size_t size; - void *end; - int error; - - if (!ext3_test_inode_state(inode, EXT3_STATE_XATTR)) - return -ENODATA; - error = ext3_get_inode_loc(inode, &iloc); - if (error) - return error; - raw_inode = ext3_raw_inode(&iloc); - header = IHDR(inode, raw_inode); - entry = IFIRST(header); - end = (void *)raw_inode + EXT3_SB(inode->i_sb)->s_inode_size; - error = ext3_xattr_check_names(entry, end); - if (error) - goto cleanup; - error = ext3_xattr_find_entry(&entry, name_index, name, - end - (void *)entry, 0); - if (error) - goto cleanup; - size = le32_to_cpu(entry->e_value_size); - if (buffer) { - error = -ERANGE; - if (size > buffer_size) - goto cleanup; - memcpy(buffer, (void *)IFIRST(header) + - le16_to_cpu(entry->e_value_offs), size); - } - error = size; - -cleanup: - brelse(iloc.bh); - return error; -} - -/* - * ext3_xattr_get() - * - * Copy an extended attribute into the buffer - * provided, or compute the buffer size required. - * Buffer is NULL to compute the size of the buffer required. - * - * Returns a negative error number on failure, or the number of bytes - * used / required on success. - */ -int -ext3_xattr_get(struct inode *inode, int name_index, const char *name, - void *buffer, size_t buffer_size) -{ - int error; - - down_read(&EXT3_I(inode)->xattr_sem); - error = ext3_xattr_ibody_get(inode, name_index, name, buffer, - buffer_size); - if (error == -ENODATA) - error = ext3_xattr_block_get(inode, name_index, name, buffer, - buffer_size); - up_read(&EXT3_I(inode)->xattr_sem); - return error; -} - -static int -ext3_xattr_list_entries(struct dentry *dentry, struct ext3_xattr_entry *entry, - char *buffer, size_t buffer_size) -{ - size_t rest = buffer_size; - - for (; !IS_LAST_ENTRY(entry); entry = EXT3_XATTR_NEXT(entry)) { - const struct xattr_handler *handler = - ext3_xattr_handler(entry->e_name_index); - - if (handler) { - size_t size = handler->list(dentry, buffer, rest, - entry->e_name, - entry->e_name_len, - handler->flags); - if (buffer) { - if (size > rest) - return -ERANGE; - buffer += size; - } - rest -= size; - } - } - return buffer_size - rest; -} - -static int -ext3_xattr_block_list(struct dentry *dentry, char *buffer, size_t buffer_size) -{ - struct inode *inode = d_inode(dentry); - struct buffer_head *bh = NULL; - int error; - - ea_idebug(inode, "buffer=%p, buffer_size=%ld", - buffer, (long)buffer_size); - - error = 0; - if (!EXT3_I(inode)->i_file_acl) - goto cleanup; - ea_idebug(inode, "reading block %u", EXT3_I(inode)->i_file_acl); - bh = sb_bread(inode->i_sb, EXT3_I(inode)->i_file_acl); - error = -EIO; - if (!bh) - goto cleanup; - ea_bdebug(bh, "b_count=%d, refcount=%d", - atomic_read(&(bh->b_count)), le32_to_cpu(BHDR(bh)->h_refcount)); - if (ext3_xattr_check_block(bh)) { - ext3_error(inode->i_sb, __func__, - "inode %lu: bad block "E3FSBLK, inode->i_ino, - EXT3_I(inode)->i_file_acl); - error = -EIO; - goto cleanup; - } - ext3_xattr_cache_insert(bh); - error = ext3_xattr_list_entries(dentry, BFIRST(bh), buffer, buffer_size); - -cleanup: - brelse(bh); - - return error; -} - -static int -ext3_xattr_ibody_list(struct dentry *dentry, char *buffer, size_t buffer_size) -{ - struct inode *inode = d_inode(dentry); - struct ext3_xattr_ibody_header *header; - struct ext3_inode *raw_inode; - struct ext3_iloc iloc; - void *end; - int error; - - if (!ext3_test_inode_state(inode, EXT3_STATE_XATTR)) - return 0; - error = ext3_get_inode_loc(inode, &iloc); - if (error) - return error; - raw_inode = ext3_raw_inode(&iloc); - header = IHDR(inode, raw_inode); - end = (void *)raw_inode + EXT3_SB(inode->i_sb)->s_inode_size; - error = ext3_xattr_check_names(IFIRST(header), end); - if (error) - goto cleanup; - error = ext3_xattr_list_entries(dentry, IFIRST(header), - buffer, buffer_size); - -cleanup: - brelse(iloc.bh); - return error; -} - -/* - * ext3_xattr_list() - * - * Copy a list of attribute names into the buffer - * provided, or compute the buffer size required. - * Buffer is NULL to compute the size of the buffer required. - * - * Returns a negative error number on failure, or the number of bytes - * used / required on success. - */ -static int -ext3_xattr_list(struct dentry *dentry, char *buffer, size_t buffer_size) -{ - int i_error, b_error; - - down_read(&EXT3_I(d_inode(dentry))->xattr_sem); - i_error = ext3_xattr_ibody_list(dentry, buffer, buffer_size); - if (i_error < 0) { - b_error = 0; - } else { - if (buffer) { - buffer += i_error; - buffer_size -= i_error; - } - b_error = ext3_xattr_block_list(dentry, buffer, buffer_size); - if (b_error < 0) - i_error = 0; - } - up_read(&EXT3_I(d_inode(dentry))->xattr_sem); - return i_error + b_error; -} - -/* - * If the EXT3_FEATURE_COMPAT_EXT_ATTR feature of this file system is - * not set, set it. - */ -static void ext3_xattr_update_super_block(handle_t *handle, - struct super_block *sb) -{ - if (EXT3_HAS_COMPAT_FEATURE(sb, EXT3_FEATURE_COMPAT_EXT_ATTR)) - return; - - if (ext3_journal_get_write_access(handle, EXT3_SB(sb)->s_sbh) == 0) { - EXT3_SET_COMPAT_FEATURE(sb, EXT3_FEATURE_COMPAT_EXT_ATTR); - ext3_journal_dirty_metadata(handle, EXT3_SB(sb)->s_sbh); - } -} - -/* - * Release the xattr block BH: If the reference count is > 1, decrement - * it; otherwise free the block. - */ -static void -ext3_xattr_release_block(handle_t *handle, struct inode *inode, - struct buffer_head *bh) -{ - struct mb_cache_entry *ce = NULL; - int error = 0; - - ce = mb_cache_entry_get(ext3_xattr_cache, bh->b_bdev, bh->b_blocknr); - error = ext3_journal_get_write_access(handle, bh); - if (error) - goto out; - - lock_buffer(bh); - - if (BHDR(bh)->h_refcount == cpu_to_le32(1)) { - ea_bdebug(bh, "refcount now=0; freeing"); - if (ce) - mb_cache_entry_free(ce); - ext3_free_blocks(handle, inode, bh->b_blocknr, 1); - get_bh(bh); - ext3_forget(handle, 1, inode, bh, bh->b_blocknr); - } else { - le32_add_cpu(&BHDR(bh)->h_refcount, -1); - error = ext3_journal_dirty_metadata(handle, bh); - if (IS_SYNC(inode)) - handle->h_sync = 1; - dquot_free_block(inode, 1); - ea_bdebug(bh, "refcount now=%d; releasing", - le32_to_cpu(BHDR(bh)->h_refcount)); - if (ce) - mb_cache_entry_release(ce); - } - unlock_buffer(bh); -out: - ext3_std_error(inode->i_sb, error); - return; -} - -struct ext3_xattr_info { - int name_index; - const char *name; - const void *value; - size_t value_len; -}; - -struct ext3_xattr_search { - struct ext3_xattr_entry *first; - void *base; - void *end; - struct ext3_xattr_entry *here; - int not_found; -}; - -static int -ext3_xattr_set_entry(struct ext3_xattr_info *i, struct ext3_xattr_search *s) -{ - struct ext3_xattr_entry *last; - size_t free, min_offs = s->end - s->base, name_len = strlen(i->name); - - /* Compute min_offs and last. */ - last = s->first; - for (; !IS_LAST_ENTRY(last); last = EXT3_XATTR_NEXT(last)) { - if (!last->e_value_block && last->e_value_size) { - size_t offs = le16_to_cpu(last->e_value_offs); - if (offs < min_offs) - min_offs = offs; - } - } - free = min_offs - ((void *)last - s->base) - sizeof(__u32); - if (!s->not_found) { - if (!s->here->e_value_block && s->here->e_value_size) { - size_t size = le32_to_cpu(s->here->e_value_size); - free += EXT3_XATTR_SIZE(size); - } - free += EXT3_XATTR_LEN(name_len); - } - if (i->value) { - if (free < EXT3_XATTR_LEN(name_len) + - EXT3_XATTR_SIZE(i->value_len)) - return -ENOSPC; - } - - if (i->value && s->not_found) { - /* Insert the new name. */ - size_t size = EXT3_XATTR_LEN(name_len); - size_t rest = (void *)last - (void *)s->here + sizeof(__u32); - memmove((void *)s->here + size, s->here, rest); - memset(s->here, 0, size); - s->here->e_name_index = i->name_index; - s->here->e_name_len = name_len; - memcpy(s->here->e_name, i->name, name_len); - } else { - if (!s->here->e_value_block && s->here->e_value_size) { - void *first_val = s->base + min_offs; - size_t offs = le16_to_cpu(s->here->e_value_offs); - void *val = s->base + offs; - size_t size = EXT3_XATTR_SIZE( - le32_to_cpu(s->here->e_value_size)); - - if (i->value && size == EXT3_XATTR_SIZE(i->value_len)) { - /* The old and the new value have the same - size. Just replace. */ - s->here->e_value_size = - cpu_to_le32(i->value_len); - memset(val + size - EXT3_XATTR_PAD, 0, - EXT3_XATTR_PAD); /* Clear pad bytes. */ - memcpy(val, i->value, i->value_len); - return 0; - } - - /* Remove the old value. */ - memmove(first_val + size, first_val, val - first_val); - memset(first_val, 0, size); - s->here->e_value_size = 0; - s->here->e_value_offs = 0; - min_offs += size; - - /* Adjust all value offsets. */ - last = s->first; - while (!IS_LAST_ENTRY(last)) { - size_t o = le16_to_cpu(last->e_value_offs); - if (!last->e_value_block && - last->e_value_size && o < offs) - last->e_value_offs = - cpu_to_le16(o + size); - last = EXT3_XATTR_NEXT(last); - } - } - if (!i->value) { - /* Remove the old name. */ - size_t size = EXT3_XATTR_LEN(name_len); - last = ENTRY((void *)last - size); - memmove(s->here, (void *)s->here + size, - (void *)last - (void *)s->here + sizeof(__u32)); - memset(last, 0, size); - } - } - - if (i->value) { - /* Insert the new value. */ - s->here->e_value_size = cpu_to_le32(i->value_len); - if (i->value_len) { - size_t size = EXT3_XATTR_SIZE(i->value_len); - void *val = s->base + min_offs - size; - s->here->e_value_offs = cpu_to_le16(min_offs - size); - memset(val + size - EXT3_XATTR_PAD, 0, - EXT3_XATTR_PAD); /* Clear the pad bytes. */ - memcpy(val, i->value, i->value_len); - } - } - return 0; -} - -struct ext3_xattr_block_find { - struct ext3_xattr_search s; - struct buffer_head *bh; -}; - -static int -ext3_xattr_block_find(struct inode *inode, struct ext3_xattr_info *i, - struct ext3_xattr_block_find *bs) -{ - struct super_block *sb = inode->i_sb; - int error; - - ea_idebug(inode, "name=%d.%s, value=%p, value_len=%ld", - i->name_index, i->name, i->value, (long)i->value_len); - - if (EXT3_I(inode)->i_file_acl) { - /* The inode already has an extended attribute block. */ - bs->bh = sb_bread(sb, EXT3_I(inode)->i_file_acl); - error = -EIO; - if (!bs->bh) - goto cleanup; - ea_bdebug(bs->bh, "b_count=%d, refcount=%d", - atomic_read(&(bs->bh->b_count)), - le32_to_cpu(BHDR(bs->bh)->h_refcount)); - if (ext3_xattr_check_block(bs->bh)) { - ext3_error(sb, __func__, - "inode %lu: bad block "E3FSBLK, inode->i_ino, - EXT3_I(inode)->i_file_acl); - error = -EIO; - goto cleanup; - } - /* Find the named attribute. */ - bs->s.base = BHDR(bs->bh); - bs->s.first = BFIRST(bs->bh); - bs->s.end = bs->bh->b_data + bs->bh->b_size; - bs->s.here = bs->s.first; - error = ext3_xattr_find_entry(&bs->s.here, i->name_index, - i->name, bs->bh->b_size, 1); - if (error && error != -ENODATA) - goto cleanup; - bs->s.not_found = error; - } - error = 0; - -cleanup: - return error; -} - -static int -ext3_xattr_block_set(handle_t *handle, struct inode *inode, - struct ext3_xattr_info *i, - struct ext3_xattr_block_find *bs) -{ - struct super_block *sb = inode->i_sb; - struct buffer_head *new_bh = NULL; - struct ext3_xattr_search *s = &bs->s; - struct mb_cache_entry *ce = NULL; - int error = 0; - -#define header(x) ((struct ext3_xattr_header *)(x)) - - if (i->value && i->value_len > sb->s_blocksize) - return -ENOSPC; - if (s->base) { - ce = mb_cache_entry_get(ext3_xattr_cache, bs->bh->b_bdev, - bs->bh->b_blocknr); - error = ext3_journal_get_write_access(handle, bs->bh); - if (error) - goto cleanup; - lock_buffer(bs->bh); - - if (header(s->base)->h_refcount == cpu_to_le32(1)) { - if (ce) { - mb_cache_entry_free(ce); - ce = NULL; - } - ea_bdebug(bs->bh, "modifying in-place"); - error = ext3_xattr_set_entry(i, s); - if (!error) { - if (!IS_LAST_ENTRY(s->first)) - ext3_xattr_rehash(header(s->base), - s->here); - ext3_xattr_cache_insert(bs->bh); - } - unlock_buffer(bs->bh); - if (error == -EIO) - goto bad_block; - if (!error) - error = ext3_journal_dirty_metadata(handle, - bs->bh); - if (error) - goto cleanup; - goto inserted; - } else { - int offset = (char *)s->here - bs->bh->b_data; - - unlock_buffer(bs->bh); - journal_release_buffer(handle, bs->bh); - - if (ce) { - mb_cache_entry_release(ce); - ce = NULL; - } - ea_bdebug(bs->bh, "cloning"); - s->base = kmalloc(bs->bh->b_size, GFP_NOFS); - error = -ENOMEM; - if (s->base == NULL) - goto cleanup; - memcpy(s->base, BHDR(bs->bh), bs->bh->b_size); - s->first = ENTRY(header(s->base)+1); - header(s->base)->h_refcount = cpu_to_le32(1); - s->here = ENTRY(s->base + offset); - s->end = s->base + bs->bh->b_size; - } - } else { - /* Allocate a buffer where we construct the new block. */ - s->base = kzalloc(sb->s_blocksize, GFP_NOFS); - /* assert(header == s->base) */ - error = -ENOMEM; - if (s->base == NULL) - goto cleanup; - header(s->base)->h_magic = cpu_to_le32(EXT3_XATTR_MAGIC); - header(s->base)->h_blocks = cpu_to_le32(1); - header(s->base)->h_refcount = cpu_to_le32(1); - s->first = ENTRY(header(s->base)+1); - s->here = ENTRY(header(s->base)+1); - s->end = s->base + sb->s_blocksize; - } - - error = ext3_xattr_set_entry(i, s); - if (error == -EIO) - goto bad_block; - if (error) - goto cleanup; - if (!IS_LAST_ENTRY(s->first)) - ext3_xattr_rehash(header(s->base), s->here); - -inserted: - if (!IS_LAST_ENTRY(s->first)) { - new_bh = ext3_xattr_cache_find(inode, header(s->base), &ce); - if (new_bh) { - /* We found an identical block in the cache. */ - if (new_bh == bs->bh) - ea_bdebug(new_bh, "keeping"); - else { - /* The old block is released after updating - the inode. */ - error = dquot_alloc_block(inode, 1); - if (error) - goto cleanup; - error = ext3_journal_get_write_access(handle, - new_bh); - if (error) - goto cleanup_dquot; - lock_buffer(new_bh); - le32_add_cpu(&BHDR(new_bh)->h_refcount, 1); - ea_bdebug(new_bh, "reusing; refcount now=%d", - le32_to_cpu(BHDR(new_bh)->h_refcount)); - unlock_buffer(new_bh); - error = ext3_journal_dirty_metadata(handle, - new_bh); - if (error) - goto cleanup_dquot; - } - mb_cache_entry_release(ce); - ce = NULL; - } else if (bs->bh && s->base == bs->bh->b_data) { - /* We were modifying this block in-place. */ - ea_bdebug(bs->bh, "keeping this block"); - new_bh = bs->bh; - get_bh(new_bh); - } else { - /* We need to allocate a new block */ - ext3_fsblk_t goal = ext3_group_first_block_no(sb, - EXT3_I(inode)->i_block_group); - ext3_fsblk_t block; - - /* - * Protect us agaist concurrent allocations to the - * same inode from ext3_..._writepage(). Reservation - * code does not expect racing allocations. - */ - mutex_lock(&EXT3_I(inode)->truncate_mutex); - block = ext3_new_block(handle, inode, goal, &error); - mutex_unlock(&EXT3_I(inode)->truncate_mutex); - if (error) - goto cleanup; - ea_idebug(inode, "creating block %d", block); - - new_bh = sb_getblk(sb, block); - if (unlikely(!new_bh)) { -getblk_failed: - ext3_free_blocks(handle, inode, block, 1); - error = -ENOMEM; - goto cleanup; - } - lock_buffer(new_bh); - error = ext3_journal_get_create_access(handle, new_bh); - if (error) { - unlock_buffer(new_bh); - goto getblk_failed; - } - memcpy(new_bh->b_data, s->base, new_bh->b_size); - set_buffer_uptodate(new_bh); - unlock_buffer(new_bh); - ext3_xattr_cache_insert(new_bh); - error = ext3_journal_dirty_metadata(handle, new_bh); - if (error) - goto cleanup; - } - } - - /* Update the inode. */ - EXT3_I(inode)->i_file_acl = new_bh ? new_bh->b_blocknr : 0; - - /* Drop the previous xattr block. */ - if (bs->bh && bs->bh != new_bh) - ext3_xattr_release_block(handle, inode, bs->bh); - error = 0; - -cleanup: - if (ce) - mb_cache_entry_release(ce); - brelse(new_bh); - if (!(bs->bh && s->base == bs->bh->b_data)) - kfree(s->base); - - return error; - -cleanup_dquot: - dquot_free_block(inode, 1); - goto cleanup; - -bad_block: - ext3_error(inode->i_sb, __func__, - "inode %lu: bad block "E3FSBLK, inode->i_ino, - EXT3_I(inode)->i_file_acl); - goto cleanup; - -#undef header -} - -struct ext3_xattr_ibody_find { - struct ext3_xattr_search s; - struct ext3_iloc iloc; -}; - -static int -ext3_xattr_ibody_find(struct inode *inode, struct ext3_xattr_info *i, - struct ext3_xattr_ibody_find *is) -{ - struct ext3_xattr_ibody_header *header; - struct ext3_inode *raw_inode; - int error; - - if (EXT3_I(inode)->i_extra_isize == 0) - return 0; - raw_inode = ext3_raw_inode(&is->iloc); - header = IHDR(inode, raw_inode); - is->s.base = is->s.first = IFIRST(header); - is->s.here = is->s.first; - is->s.end = (void *)raw_inode + EXT3_SB(inode->i_sb)->s_inode_size; - if (ext3_test_inode_state(inode, EXT3_STATE_XATTR)) { - error = ext3_xattr_check_names(IFIRST(header), is->s.end); - if (error) - return error; - /* Find the named attribute. */ - error = ext3_xattr_find_entry(&is->s.here, i->name_index, - i->name, is->s.end - - (void *)is->s.base, 0); - if (error && error != -ENODATA) - return error; - is->s.not_found = error; - } - return 0; -} - -static int -ext3_xattr_ibody_set(handle_t *handle, struct inode *inode, - struct ext3_xattr_info *i, - struct ext3_xattr_ibody_find *is) -{ - struct ext3_xattr_ibody_header *header; - struct ext3_xattr_search *s = &is->s; - int error; - - if (EXT3_I(inode)->i_extra_isize == 0) - return -ENOSPC; - error = ext3_xattr_set_entry(i, s); - if (error) - return error; - header = IHDR(inode, ext3_raw_inode(&is->iloc)); - if (!IS_LAST_ENTRY(s->first)) { - header->h_magic = cpu_to_le32(EXT3_XATTR_MAGIC); - ext3_set_inode_state(inode, EXT3_STATE_XATTR); - } else { - header->h_magic = cpu_to_le32(0); - ext3_clear_inode_state(inode, EXT3_STATE_XATTR); - } - return 0; -} - -/* - * ext3_xattr_set_handle() - * - * Create, replace or remove an extended attribute for this inode. Value - * is NULL to remove an existing extended attribute, and non-NULL to - * either replace an existing extended attribute, or create a new extended - * attribute. The flags XATTR_REPLACE and XATTR_CREATE - * specify that an extended attribute must exist and must not exist - * previous to the call, respectively. - * - * Returns 0, or a negative error number on failure. - */ -int -ext3_xattr_set_handle(handle_t *handle, struct inode *inode, int name_index, - const char *name, const void *value, size_t value_len, - int flags) -{ - struct ext3_xattr_info i = { - .name_index = name_index, - .name = name, - .value = value, - .value_len = value_len, - - }; - struct ext3_xattr_ibody_find is = { - .s = { .not_found = -ENODATA, }, - }; - struct ext3_xattr_block_find bs = { - .s = { .not_found = -ENODATA, }, - }; - int error; - - if (!name) - return -EINVAL; - if (strlen(name) > 255) - return -ERANGE; - down_write(&EXT3_I(inode)->xattr_sem); - error = ext3_get_inode_loc(inode, &is.iloc); - if (error) - goto cleanup; - - error = ext3_journal_get_write_access(handle, is.iloc.bh); - if (error) - goto cleanup; - - if (ext3_test_inode_state(inode, EXT3_STATE_NEW)) { - struct ext3_inode *raw_inode = ext3_raw_inode(&is.iloc); - memset(raw_inode, 0, EXT3_SB(inode->i_sb)->s_inode_size); - ext3_clear_inode_state(inode, EXT3_STATE_NEW); - } - - error = ext3_xattr_ibody_find(inode, &i, &is); - if (error) - goto cleanup; - if (is.s.not_found) - error = ext3_xattr_block_find(inode, &i, &bs); - if (error) - goto cleanup; - if (is.s.not_found && bs.s.not_found) { - error = -ENODATA; - if (flags & XATTR_REPLACE) - goto cleanup; - error = 0; - if (!value) - goto cleanup; - } else { - error = -EEXIST; - if (flags & XATTR_CREATE) - goto cleanup; - } - if (!value) { - if (!is.s.not_found) - error = ext3_xattr_ibody_set(handle, inode, &i, &is); - else if (!bs.s.not_found) - error = ext3_xattr_block_set(handle, inode, &i, &bs); - } else { - error = ext3_xattr_ibody_set(handle, inode, &i, &is); - if (!error && !bs.s.not_found) { - i.value = NULL; - error = ext3_xattr_block_set(handle, inode, &i, &bs); - } else if (error == -ENOSPC) { - if (EXT3_I(inode)->i_file_acl && !bs.s.base) { - error = ext3_xattr_block_find(inode, &i, &bs); - if (error) - goto cleanup; - } - error = ext3_xattr_block_set(handle, inode, &i, &bs); - if (error) - goto cleanup; - if (!is.s.not_found) { - i.value = NULL; - error = ext3_xattr_ibody_set(handle, inode, &i, - &is); - } - } - } - if (!error) { - ext3_xattr_update_super_block(handle, inode->i_sb); - inode->i_ctime = CURRENT_TIME_SEC; - error = ext3_mark_iloc_dirty(handle, inode, &is.iloc); - /* - * The bh is consumed by ext3_mark_iloc_dirty, even with - * error != 0. - */ - is.iloc.bh = NULL; - if (IS_SYNC(inode)) - handle->h_sync = 1; - } - -cleanup: - brelse(is.iloc.bh); - brelse(bs.bh); - up_write(&EXT3_I(inode)->xattr_sem); - return error; -} - -/* - * ext3_xattr_set() - * - * Like ext3_xattr_set_handle, but start from an inode. This extended - * attribute modification is a filesystem transaction by itself. - * - * Returns 0, or a negative error number on failure. - */ -int -ext3_xattr_set(struct inode *inode, int name_index, const char *name, - const void *value, size_t value_len, int flags) -{ - handle_t *handle; - int error, retries = 0; - -retry: - handle = ext3_journal_start(inode, EXT3_DATA_TRANS_BLOCKS(inode->i_sb)); - if (IS_ERR(handle)) { - error = PTR_ERR(handle); - } else { - int error2; - - error = ext3_xattr_set_handle(handle, inode, name_index, name, - value, value_len, flags); - error2 = ext3_journal_stop(handle); - if (error == -ENOSPC && - ext3_should_retry_alloc(inode->i_sb, &retries)) - goto retry; - if (error == 0) - error = error2; - } - - return error; -} - -/* - * ext3_xattr_delete_inode() - * - * Free extended attribute resources associated with this inode. This - * is called immediately before an inode is freed. We have exclusive - * access to the inode. - */ -void -ext3_xattr_delete_inode(handle_t *handle, struct inode *inode) -{ - struct buffer_head *bh = NULL; - - if (!EXT3_I(inode)->i_file_acl) - goto cleanup; - bh = sb_bread(inode->i_sb, EXT3_I(inode)->i_file_acl); - if (!bh) { - ext3_error(inode->i_sb, __func__, - "inode %lu: block "E3FSBLK" read error", inode->i_ino, - EXT3_I(inode)->i_file_acl); - goto cleanup; - } - if (BHDR(bh)->h_magic != cpu_to_le32(EXT3_XATTR_MAGIC) || - BHDR(bh)->h_blocks != cpu_to_le32(1)) { - ext3_error(inode->i_sb, __func__, - "inode %lu: bad block "E3FSBLK, inode->i_ino, - EXT3_I(inode)->i_file_acl); - goto cleanup; - } - ext3_xattr_release_block(handle, inode, bh); - EXT3_I(inode)->i_file_acl = 0; - -cleanup: - brelse(bh); -} - -/* - * ext3_xattr_put_super() - * - * This is called when a file system is unmounted. - */ -void -ext3_xattr_put_super(struct super_block *sb) -{ - mb_cache_shrink(sb->s_bdev); -} - -/* - * ext3_xattr_cache_insert() - * - * Create a new entry in the extended attribute cache, and insert - * it unless such an entry is already in the cache. - * - * Returns 0, or a negative error number on failure. - */ -static void -ext3_xattr_cache_insert(struct buffer_head *bh) -{ - __u32 hash = le32_to_cpu(BHDR(bh)->h_hash); - struct mb_cache_entry *ce; - int error; - - ce = mb_cache_entry_alloc(ext3_xattr_cache, GFP_NOFS); - if (!ce) { - ea_bdebug(bh, "out of memory"); - return; - } - error = mb_cache_entry_insert(ce, bh->b_bdev, bh->b_blocknr, hash); - if (error) { - mb_cache_entry_free(ce); - if (error == -EBUSY) { - ea_bdebug(bh, "already in cache"); - error = 0; - } - } else { - ea_bdebug(bh, "inserting [%x]", (int)hash); - mb_cache_entry_release(ce); - } -} - -/* - * ext3_xattr_cmp() - * - * Compare two extended attribute blocks for equality. - * - * Returns 0 if the blocks are equal, 1 if they differ, and - * a negative error number on errors. - */ -static int -ext3_xattr_cmp(struct ext3_xattr_header *header1, - struct ext3_xattr_header *header2) -{ - struct ext3_xattr_entry *entry1, *entry2; - - entry1 = ENTRY(header1+1); - entry2 = ENTRY(header2+1); - while (!IS_LAST_ENTRY(entry1)) { - if (IS_LAST_ENTRY(entry2)) - return 1; - if (entry1->e_hash != entry2->e_hash || - entry1->e_name_index != entry2->e_name_index || - entry1->e_name_len != entry2->e_name_len || - entry1->e_value_size != entry2->e_value_size || - memcmp(entry1->e_name, entry2->e_name, entry1->e_name_len)) - return 1; - if (entry1->e_value_block != 0 || entry2->e_value_block != 0) - return -EIO; - if (memcmp((char *)header1 + le16_to_cpu(entry1->e_value_offs), - (char *)header2 + le16_to_cpu(entry2->e_value_offs), - le32_to_cpu(entry1->e_value_size))) - return 1; - - entry1 = EXT3_XATTR_NEXT(entry1); - entry2 = EXT3_XATTR_NEXT(entry2); - } - if (!IS_LAST_ENTRY(entry2)) - return 1; - return 0; -} - -/* - * ext3_xattr_cache_find() - * - * Find an identical extended attribute block. - * - * Returns a pointer to the block found, or NULL if such a block was - * not found or an error occurred. - */ -static struct buffer_head * -ext3_xattr_cache_find(struct inode *inode, struct ext3_xattr_header *header, - struct mb_cache_entry **pce) -{ - __u32 hash = le32_to_cpu(header->h_hash); - struct mb_cache_entry *ce; - - if (!header->h_hash) - return NULL; /* never share */ - ea_idebug(inode, "looking for cached blocks [%x]", (int)hash); -again: - ce = mb_cache_entry_find_first(ext3_xattr_cache, inode->i_sb->s_bdev, - hash); - while (ce) { - struct buffer_head *bh; - - if (IS_ERR(ce)) { - if (PTR_ERR(ce) == -EAGAIN) - goto again; - break; - } - bh = sb_bread(inode->i_sb, ce->e_block); - if (!bh) { - ext3_error(inode->i_sb, __func__, - "inode %lu: block %lu read error", - inode->i_ino, (unsigned long) ce->e_block); - } else if (le32_to_cpu(BHDR(bh)->h_refcount) >= - EXT3_XATTR_REFCOUNT_MAX) { - ea_idebug(inode, "block %lu refcount %d>=%d", - (unsigned long) ce->e_block, - le32_to_cpu(BHDR(bh)->h_refcount), - EXT3_XATTR_REFCOUNT_MAX); - } else if (ext3_xattr_cmp(header, BHDR(bh)) == 0) { - *pce = ce; - return bh; - } - brelse(bh); - ce = mb_cache_entry_find_next(ce, inode->i_sb->s_bdev, hash); - } - return NULL; -} - -#define NAME_HASH_SHIFT 5 -#define VALUE_HASH_SHIFT 16 - -/* - * ext3_xattr_hash_entry() - * - * Compute the hash of an extended attribute. - */ -static inline void ext3_xattr_hash_entry(struct ext3_xattr_header *header, - struct ext3_xattr_entry *entry) -{ - __u32 hash = 0; - char *name = entry->e_name; - int n; - - for (n=0; n < entry->e_name_len; n++) { - hash = (hash << NAME_HASH_SHIFT) ^ - (hash >> (8*sizeof(hash) - NAME_HASH_SHIFT)) ^ - *name++; - } - - if (entry->e_value_block == 0 && entry->e_value_size != 0) { - __le32 *value = (__le32 *)((char *)header + - le16_to_cpu(entry->e_value_offs)); - for (n = (le32_to_cpu(entry->e_value_size) + - EXT3_XATTR_ROUND) >> EXT3_XATTR_PAD_BITS; n; n--) { - hash = (hash << VALUE_HASH_SHIFT) ^ - (hash >> (8*sizeof(hash) - VALUE_HASH_SHIFT)) ^ - le32_to_cpu(*value++); - } - } - entry->e_hash = cpu_to_le32(hash); -} - -#undef NAME_HASH_SHIFT -#undef VALUE_HASH_SHIFT - -#define BLOCK_HASH_SHIFT 16 - -/* - * ext3_xattr_rehash() - * - * Re-compute the extended attribute hash value after an entry has changed. - */ -static void ext3_xattr_rehash(struct ext3_xattr_header *header, - struct ext3_xattr_entry *entry) -{ - struct ext3_xattr_entry *here; - __u32 hash = 0; - - ext3_xattr_hash_entry(header, entry); - here = ENTRY(header+1); - while (!IS_LAST_ENTRY(here)) { - if (!here->e_hash) { - /* Block is not shared if an entry's hash value == 0 */ - hash = 0; - break; - } - hash = (hash << BLOCK_HASH_SHIFT) ^ - (hash >> (8*sizeof(hash) - BLOCK_HASH_SHIFT)) ^ - le32_to_cpu(here->e_hash); - here = EXT3_XATTR_NEXT(here); - } - header->h_hash = cpu_to_le32(hash); -} - -#undef BLOCK_HASH_SHIFT - -int __init -init_ext3_xattr(void) -{ - ext3_xattr_cache = mb_cache_create("ext3_xattr", 6); - if (!ext3_xattr_cache) - return -ENOMEM; - return 0; -} - -void -exit_ext3_xattr(void) -{ - if (ext3_xattr_cache) - mb_cache_destroy(ext3_xattr_cache); - ext3_xattr_cache = NULL; -} diff --git a/fs/ext3/xattr.h b/fs/ext3/xattr.h deleted file mode 100644 index 32e93ebf80315b..00000000000000 --- a/fs/ext3/xattr.h +++ /dev/null @@ -1,136 +0,0 @@ -/* - File: fs/ext3/xattr.h - - On-disk format of extended attributes for the ext3 filesystem. - - (C) 2001 Andreas Gruenbacher, -*/ - -#include - -/* Magic value in attribute blocks */ -#define EXT3_XATTR_MAGIC 0xEA020000 - -/* Maximum number of references to one attribute block */ -#define EXT3_XATTR_REFCOUNT_MAX 1024 - -/* Name indexes */ -#define EXT3_XATTR_INDEX_USER 1 -#define EXT3_XATTR_INDEX_POSIX_ACL_ACCESS 2 -#define EXT3_XATTR_INDEX_POSIX_ACL_DEFAULT 3 -#define EXT3_XATTR_INDEX_TRUSTED 4 -#define EXT3_XATTR_INDEX_LUSTRE 5 -#define EXT3_XATTR_INDEX_SECURITY 6 - -struct ext3_xattr_header { - __le32 h_magic; /* magic number for identification */ - __le32 h_refcount; /* reference count */ - __le32 h_blocks; /* number of disk blocks used */ - __le32 h_hash; /* hash value of all attributes */ - __u32 h_reserved[4]; /* zero right now */ -}; - -struct ext3_xattr_ibody_header { - __le32 h_magic; /* magic number for identification */ -}; - -struct ext3_xattr_entry { - __u8 e_name_len; /* length of name */ - __u8 e_name_index; /* attribute name index */ - __le16 e_value_offs; /* offset in disk block of value */ - __le32 e_value_block; /* disk block attribute is stored on (n/i) */ - __le32 e_value_size; /* size of attribute value */ - __le32 e_hash; /* hash value of name and value */ - char e_name[0]; /* attribute name */ -}; - -#define EXT3_XATTR_PAD_BITS 2 -#define EXT3_XATTR_PAD (1<e_name_len)) ) -#define EXT3_XATTR_SIZE(size) \ - (((size) + EXT3_XATTR_ROUND) & ~EXT3_XATTR_ROUND) - -# ifdef CONFIG_EXT3_FS_XATTR - -extern const struct xattr_handler ext3_xattr_user_handler; -extern const struct xattr_handler ext3_xattr_trusted_handler; -extern const struct xattr_handler ext3_xattr_security_handler; - -extern ssize_t ext3_listxattr(struct dentry *, char *, size_t); - -extern int ext3_xattr_get(struct inode *, int, const char *, void *, size_t); -extern int ext3_xattr_set(struct inode *, int, const char *, const void *, size_t, int); -extern int ext3_xattr_set_handle(handle_t *, struct inode *, int, const char *, const void *, size_t, int); - -extern void ext3_xattr_delete_inode(handle_t *, struct inode *); -extern void ext3_xattr_put_super(struct super_block *); - -extern int init_ext3_xattr(void); -extern void exit_ext3_xattr(void); - -extern const struct xattr_handler *ext3_xattr_handlers[]; - -# else /* CONFIG_EXT3_FS_XATTR */ - -static inline int -ext3_xattr_get(struct inode *inode, int name_index, const char *name, - void *buffer, size_t size, int flags) -{ - return -EOPNOTSUPP; -} - -static inline int -ext3_xattr_set(struct inode *inode, int name_index, const char *name, - const void *value, size_t size, int flags) -{ - return -EOPNOTSUPP; -} - -static inline int -ext3_xattr_set_handle(handle_t *handle, struct inode *inode, int name_index, - const char *name, const void *value, size_t size, int flags) -{ - return -EOPNOTSUPP; -} - -static inline void -ext3_xattr_delete_inode(handle_t *handle, struct inode *inode) -{ -} - -static inline void -ext3_xattr_put_super(struct super_block *sb) -{ -} - -static inline int -init_ext3_xattr(void) -{ - return 0; -} - -static inline void -exit_ext3_xattr(void) -{ -} - -#define ext3_xattr_handlers NULL - -# endif /* CONFIG_EXT3_FS_XATTR */ - -#ifdef CONFIG_EXT3_FS_SECURITY -extern int ext3_init_security(handle_t *handle, struct inode *inode, - struct inode *dir, const struct qstr *qstr); -#else -static inline int ext3_init_security(handle_t *handle, struct inode *inode, - struct inode *dir, const struct qstr *qstr) -{ - return 0; -} -#endif diff --git a/fs/ext3/xattr_security.c b/fs/ext3/xattr_security.c deleted file mode 100644 index c9506d5e3b13b5..00000000000000 --- a/fs/ext3/xattr_security.c +++ /dev/null @@ -1,78 +0,0 @@ -/* - * linux/fs/ext3/xattr_security.c - * Handler for storing security labels as extended attributes. - */ - -#include -#include "ext3.h" -#include "xattr.h" - -static size_t -ext3_xattr_security_list(struct dentry *dentry, char *list, size_t list_size, - const char *name, size_t name_len, int type) -{ - const size_t prefix_len = XATTR_SECURITY_PREFIX_LEN; - const size_t total_len = prefix_len + name_len + 1; - - - if (list && total_len <= list_size) { - memcpy(list, XATTR_SECURITY_PREFIX, prefix_len); - memcpy(list+prefix_len, name, name_len); - list[prefix_len + name_len] = '\0'; - } - return total_len; -} - -static int -ext3_xattr_security_get(struct dentry *dentry, const char *name, - void *buffer, size_t size, int type) -{ - if (strcmp(name, "") == 0) - return -EINVAL; - return ext3_xattr_get(d_inode(dentry), EXT3_XATTR_INDEX_SECURITY, - name, buffer, size); -} - -static int -ext3_xattr_security_set(struct dentry *dentry, const char *name, - const void *value, size_t size, int flags, int type) -{ - if (strcmp(name, "") == 0) - return -EINVAL; - return ext3_xattr_set(d_inode(dentry), EXT3_XATTR_INDEX_SECURITY, - name, value, size, flags); -} - -static int ext3_initxattrs(struct inode *inode, - const struct xattr *xattr_array, - void *fs_info) -{ - const struct xattr *xattr; - handle_t *handle = fs_info; - int err = 0; - - for (xattr = xattr_array; xattr->name != NULL; xattr++) { - err = ext3_xattr_set_handle(handle, inode, - EXT3_XATTR_INDEX_SECURITY, - xattr->name, xattr->value, - xattr->value_len, 0); - if (err < 0) - break; - } - return err; -} - -int -ext3_init_security(handle_t *handle, struct inode *inode, struct inode *dir, - const struct qstr *qstr) -{ - return security_inode_init_security(inode, dir, qstr, - &ext3_initxattrs, handle); -} - -const struct xattr_handler ext3_xattr_security_handler = { - .prefix = XATTR_SECURITY_PREFIX, - .list = ext3_xattr_security_list, - .get = ext3_xattr_security_get, - .set = ext3_xattr_security_set, -}; diff --git a/fs/ext3/xattr_trusted.c b/fs/ext3/xattr_trusted.c deleted file mode 100644 index 206cc66dc2852a..00000000000000 --- a/fs/ext3/xattr_trusted.c +++ /dev/null @@ -1,54 +0,0 @@ -/* - * linux/fs/ext3/xattr_trusted.c - * Handler for trusted extended attributes. - * - * Copyright (C) 2003 by Andreas Gruenbacher, - */ - -#include "ext3.h" -#include "xattr.h" - -static size_t -ext3_xattr_trusted_list(struct dentry *dentry, char *list, size_t list_size, - const char *name, size_t name_len, int type) -{ - const size_t prefix_len = XATTR_TRUSTED_PREFIX_LEN; - const size_t total_len = prefix_len + name_len + 1; - - if (!capable(CAP_SYS_ADMIN)) - return 0; - - if (list && total_len <= list_size) { - memcpy(list, XATTR_TRUSTED_PREFIX, prefix_len); - memcpy(list+prefix_len, name, name_len); - list[prefix_len + name_len] = '\0'; - } - return total_len; -} - -static int -ext3_xattr_trusted_get(struct dentry *dentry, const char *name, - void *buffer, size_t size, int type) -{ - if (strcmp(name, "") == 0) - return -EINVAL; - return ext3_xattr_get(d_inode(dentry), EXT3_XATTR_INDEX_TRUSTED, - name, buffer, size); -} - -static int -ext3_xattr_trusted_set(struct dentry *dentry, const char *name, - const void *value, size_t size, int flags, int type) -{ - if (strcmp(name, "") == 0) - return -EINVAL; - return ext3_xattr_set(d_inode(dentry), EXT3_XATTR_INDEX_TRUSTED, name, - value, size, flags); -} - -const struct xattr_handler ext3_xattr_trusted_handler = { - .prefix = XATTR_TRUSTED_PREFIX, - .list = ext3_xattr_trusted_list, - .get = ext3_xattr_trusted_get, - .set = ext3_xattr_trusted_set, -}; diff --git a/fs/ext3/xattr_user.c b/fs/ext3/xattr_user.c deleted file mode 100644 index 021508ad16168c..00000000000000 --- a/fs/ext3/xattr_user.c +++ /dev/null @@ -1,58 +0,0 @@ -/* - * linux/fs/ext3/xattr_user.c - * Handler for extended user attributes. - * - * Copyright (C) 2001 by Andreas Gruenbacher, - */ - -#include "ext3.h" -#include "xattr.h" - -static size_t -ext3_xattr_user_list(struct dentry *dentry, char *list, size_t list_size, - const char *name, size_t name_len, int type) -{ - const size_t prefix_len = XATTR_USER_PREFIX_LEN; - const size_t total_len = prefix_len + name_len + 1; - - if (!test_opt(dentry->d_sb, XATTR_USER)) - return 0; - - if (list && total_len <= list_size) { - memcpy(list, XATTR_USER_PREFIX, prefix_len); - memcpy(list+prefix_len, name, name_len); - list[prefix_len + name_len] = '\0'; - } - return total_len; -} - -static int -ext3_xattr_user_get(struct dentry *dentry, const char *name, void *buffer, - size_t size, int type) -{ - if (strcmp(name, "") == 0) - return -EINVAL; - if (!test_opt(dentry->d_sb, XATTR_USER)) - return -EOPNOTSUPP; - return ext3_xattr_get(d_inode(dentry), EXT3_XATTR_INDEX_USER, - name, buffer, size); -} - -static int -ext3_xattr_user_set(struct dentry *dentry, const char *name, - const void *value, size_t size, int flags, int type) -{ - if (strcmp(name, "") == 0) - return -EINVAL; - if (!test_opt(dentry->d_sb, XATTR_USER)) - return -EOPNOTSUPP; - return ext3_xattr_set(d_inode(dentry), EXT3_XATTR_INDEX_USER, - name, value, size, flags); -} - -const struct xattr_handler ext3_xattr_user_handler = { - .prefix = XATTR_USER_PREFIX, - .list = ext3_xattr_user_list, - .get = ext3_xattr_user_get, - .set = ext3_xattr_user_set, -}; diff --git a/fs/ext4/Kconfig b/fs/ext4/Kconfig index bf8bc8aba471c6..47728da7702cdf 100644 --- a/fs/ext4/Kconfig +++ b/fs/ext4/Kconfig @@ -1,5 +1,38 @@ +# Ext3 configs are here for backward compatibility with old configs which may +# have EXT3_FS set but not EXT4_FS set and thus would result in non-bootable +# kernels after the removal of ext3 driver. +config EXT3_FS + tristate "The Extended 3 (ext3) filesystem" + # These must match EXT4_FS selects... + select EXT4_FS + select JBD2 + select CRC16 + select CRYPTO + select CRYPTO_CRC32C + help + This config option is here only for backward compatibility. ext3 + filesystem is now handled by the ext4 driver. + +config EXT3_FS_POSIX_ACL + bool "Ext3 POSIX Access Control Lists" + depends on EXT3_FS + select EXT4_FS_POSIX_ACL + select FS_POSIX_ACL + help + This config option is here only for backward compatibility. ext3 + filesystem is now handled by the ext4 driver. + +config EXT3_FS_SECURITY + bool "Ext3 Security Labels" + depends on EXT3_FS + select EXT4_FS_SECURITY + help + This config option is here only for backward compatibility. ext3 + filesystem is now handled by the ext4 driver. + config EXT4_FS tristate "The Extended 4 (ext4) filesystem" + # Please update EXT3_FS selects when changing these select JBD2 select CRC16 select CRYPTO @@ -16,26 +49,27 @@ config EXT4_FS up fsck time. For more information, please see the web pages at http://ext4.wiki.kernel.org. - The ext4 filesystem will support mounting an ext3 - filesystem; while there will be some performance gains from - the delayed allocation and inode table readahead, the best - performance gains will require enabling ext4 features in the - filesystem, or formatting a new filesystem as an ext4 - filesystem initially. + The ext4 filesystem supports mounting an ext3 filesystem; while there + are some performance gains from the delayed allocation and inode + table readahead, the best performance gains require enabling ext4 + features in the filesystem using tune2fs, or formatting a new + filesystem as an ext4 filesystem initially. Without explicit enabling + of ext4 features, the on disk filesystem format stays fully backward + compatible. To compile this file system support as a module, choose M here. The module will be called ext4. If unsure, say N. -config EXT4_USE_FOR_EXT23 +config EXT4_USE_FOR_EXT2 bool "Use ext4 for ext2/ext3 file systems" depends on EXT4_FS - depends on EXT3_FS=n || EXT2_FS=n + depends on EXT2_FS=n default y help - Allow the ext4 file system driver code to be used for ext2 or - ext3 file system mounts. This allows users to reduce their + Allow the ext4 file system driver code to be used for ext2 + file system mounts. This allows users to reduce their compiled kernel size by using one file system driver for ext2, ext3, and ext4 file systems. diff --git a/fs/ext4/crypto_fname.c b/fs/ext4/crypto_fname.c index 7dc4eb55913c44..847f919c84d9cc 100644 --- a/fs/ext4/crypto_fname.c +++ b/fs/ext4/crypto_fname.c @@ -19,7 +19,6 @@ #include #include #include -#include #include #include #include @@ -329,6 +328,10 @@ int _ext4_fname_disk_to_usr(struct inode *inode, return oname->len; } } + if (iname->len < EXT4_CRYPTO_BLOCK_SIZE) { + EXT4_ERROR_INODE(inode, "encrypted inode too small"); + return -EUCLEAN; + } if (EXT4_I(inode)->i_crypt_info) return ext4_fname_decrypt(inode, iname, oname); diff --git a/fs/ext4/crypto_key.c b/fs/ext4/crypto_key.c index 442d24e8efc0ce..1d510c11b100cf 100644 --- a/fs/ext4/crypto_key.c +++ b/fs/ext4/crypto_key.c @@ -30,7 +30,7 @@ static void derive_crypt_complete(struct crypto_async_request *req, int rc) /** * ext4_derive_key_aes() - Derive a key using AES-128-ECB - * @deriving_key: Encryption key used for derivatio. + * @deriving_key: Encryption key used for derivation. * @source_key: Source key to which to apply derivation. * @derived_key: Derived key. * @@ -220,6 +220,8 @@ int _ext4_get_encryption_info(struct inode *inode) BUG_ON(master_key->size != EXT4_AES_256_XTS_KEY_SIZE); res = ext4_derive_key_aes(ctx.nonce, master_key->raw, raw_key); + if (res) + goto out; got_key: ctfm = crypto_alloc_ablkcipher(cipher_str, 0, 0); if (!ctfm || IS_ERR(ctfm)) { diff --git a/fs/ext4/crypto_policy.c b/fs/ext4/crypto_policy.c index 02c4e5df7afb92..a640ec2c4b134a 100644 --- a/fs/ext4/crypto_policy.c +++ b/fs/ext4/crypto_policy.c @@ -12,6 +12,7 @@ #include #include +#include "ext4_jbd2.h" #include "ext4.h" #include "xattr.h" @@ -49,7 +50,8 @@ static int ext4_create_encryption_context_from_policy( struct inode *inode, const struct ext4_encryption_policy *policy) { struct ext4_encryption_context ctx; - int res = 0; + handle_t *handle; + int res, res2; res = ext4_convert_inline_data(inode); if (res) @@ -78,11 +80,22 @@ static int ext4_create_encryption_context_from_policy( BUILD_BUG_ON(sizeof(ctx.nonce) != EXT4_KEY_DERIVATION_NONCE_SIZE); get_random_bytes(ctx.nonce, EXT4_KEY_DERIVATION_NONCE_SIZE); + handle = ext4_journal_start(inode, EXT4_HT_MISC, + ext4_jbd2_credits_xattr(inode)); + if (IS_ERR(handle)) + return PTR_ERR(handle); res = ext4_xattr_set(inode, EXT4_XATTR_INDEX_ENCRYPTION, EXT4_XATTR_NAME_ENCRYPTION_CONTEXT, &ctx, sizeof(ctx), 0); - if (!res) + if (!res) { ext4_set_inode_flag(inode, EXT4_INODE_ENCRYPT); + res = ext4_mark_inode_dirty(handle, inode); + if (res) + EXT4_ERROR_INODE(inode, "Failed to mark inode dirty"); + } + res2 = ext4_journal_stop(handle); + if (!res) + res = res2; return res; } diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index f5e9f04220c1d4..32071f5c1c2623 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -187,7 +187,7 @@ typedef struct ext4_io_end { } ext4_io_end_t; struct ext4_io_submit { - int io_op; + struct writeback_control *io_wbc; struct bio *io_bio; ext4_io_end_t *io_end; sector_t io_next_block; diff --git a/fs/ext4/ialloc.c b/fs/ext4/ialloc.c index 173c1ae213955a..619bfc1fda8cc0 100644 --- a/fs/ext4/ialloc.c +++ b/fs/ext4/ialloc.c @@ -721,7 +721,7 @@ struct inode *__ext4_new_inode(handle_t *handle, struct inode *dir, struct ext4_group_desc *gdp = NULL; struct ext4_inode_info *ei; struct ext4_sb_info *sbi; - int ret2, err = 0; + int ret2, err; struct inode *ret; ext4_group_t i; ext4_group_t flex_group; @@ -769,7 +769,9 @@ struct inode *__ext4_new_inode(handle_t *handle, struct inode *dir, inode->i_gid = dir->i_gid; } else inode_init_owner(inode, dir, mode); - dquot_initialize(inode); + err = dquot_initialize(inode); + if (err) + goto out; if (!goal) goal = sbi->s_inode_goal; diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index cecf9aa1081134..29f1af7c2cab9e 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -4661,8 +4661,11 @@ int ext4_setattr(struct dentry *dentry, struct iattr *attr) if (error) return error; - if (is_quota_modification(inode, attr)) - dquot_initialize(inode); + if (is_quota_modification(inode, attr)) { + error = dquot_initialize(inode); + if (error) + return error; + } if ((ia_valid & ATTR_UID && !uid_eq(attr->ia_uid, inode->i_uid)) || (ia_valid & ATTR_GID && !gid_eq(attr->ia_gid, inode->i_gid))) { handle_t *handle; @@ -4725,6 +4728,14 @@ int ext4_setattr(struct dentry *dentry, struct iattr *attr) error = ext4_orphan_add(handle, inode); orphan = 1; } + /* + * Update c/mtime on truncate up, ext4_truncate() will + * update c/mtime in shrink case below + */ + if (!shrink) { + inode->i_mtime = ext4_current_time(inode); + inode->i_ctime = inode->i_mtime; + } down_write(&EXT4_I(inode)->i_data_sem); EXT4_I(inode)->i_disksize = attr->ia_size; rc = ext4_mark_inode_dirty(handle, inode); diff --git a/fs/ext4/mmp.c b/fs/ext4/mmp.c index 8313ca3324ec96..6eb1a619890c58 100644 --- a/fs/ext4/mmp.c +++ b/fs/ext4/mmp.c @@ -69,6 +69,7 @@ static int read_mmp_block(struct super_block *sb, struct buffer_head **bh, ext4_fsblk_t mmp_block) { struct mmp_struct *mmp; + int ret; if (*bh) clear_buffer_uptodate(*bh); @@ -76,33 +77,36 @@ static int read_mmp_block(struct super_block *sb, struct buffer_head **bh, /* This would be sb_bread(sb, mmp_block), except we need to be sure * that the MD RAID device cache has been bypassed, and that the read * is not blocked in the elevator. */ - if (!*bh) + if (!*bh) { *bh = sb_getblk(sb, mmp_block); - if (!*bh) - return -ENOMEM; - if (*bh) { - get_bh(*bh); - lock_buffer(*bh); - (*bh)->b_end_io = end_buffer_read_sync; - submit_bh(READ_SYNC | REQ_META | REQ_PRIO, *bh); - wait_on_buffer(*bh); - if (!buffer_uptodate(*bh)) { - brelse(*bh); - *bh = NULL; + if (!*bh) { + ret = -ENOMEM; + goto warn_exit; } } - if (unlikely(!*bh)) { - ext4_warning(sb, "Error while reading MMP block %llu", - mmp_block); - return -EIO; + + get_bh(*bh); + lock_buffer(*bh); + (*bh)->b_end_io = end_buffer_read_sync; + submit_bh(READ_SYNC | REQ_META | REQ_PRIO, *bh); + wait_on_buffer(*bh); + if (!buffer_uptodate(*bh)) { + brelse(*bh); + *bh = NULL; + ret = -EIO; + goto warn_exit; } mmp = (struct mmp_struct *)((*bh)->b_data); - if (le32_to_cpu(mmp->mmp_magic) != EXT4_MMP_MAGIC || - !ext4_mmp_csum_verify(sb, mmp)) - return -EINVAL; - - return 0; + if (le32_to_cpu(mmp->mmp_magic) == EXT4_MMP_MAGIC && + ext4_mmp_csum_verify(sb, mmp)) + return 0; + ret = -EINVAL; + +warn_exit: + ext4_warning(sb, "Error %d while reading MMP block %llu", + ret, mmp_block); + return ret; } /* @@ -111,7 +115,7 @@ static int read_mmp_block(struct super_block *sb, struct buffer_head **bh, void __dump_mmp_msg(struct super_block *sb, struct mmp_struct *mmp, const char *function, unsigned int line, const char *msg) { - __ext4_warning(sb, function, line, msg); + __ext4_warning(sb, function, line, "%s", msg); __ext4_warning(sb, function, line, "MMP failure info: last update time: %llu, last update " "node: %s, last update device: %s\n", diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c index 011dcfb5cce376..9f61e7679a6de5 100644 --- a/fs/ext4/namei.c +++ b/fs/ext4/namei.c @@ -2436,7 +2436,9 @@ static int ext4_create(struct inode *dir, struct dentry *dentry, umode_t mode, struct inode *inode; int err, credits, retries = 0; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + return err; credits = (EXT4_DATA_TRANS_BLOCKS(dir->i_sb) + EXT4_INDEX_EXTRA_TRANS_BLOCKS + 3); @@ -2470,7 +2472,9 @@ static int ext4_mknod(struct inode *dir, struct dentry *dentry, if (!new_valid_dev(rdev)) return -EINVAL; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + return err; credits = (EXT4_DATA_TRANS_BLOCKS(dir->i_sb) + EXT4_INDEX_EXTRA_TRANS_BLOCKS + 3); @@ -2499,7 +2503,9 @@ static int ext4_tmpfile(struct inode *dir, struct dentry *dentry, umode_t mode) struct inode *inode; int err, retries = 0; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + return err; retry: inode = ext4_new_inode_start_handle(dir, mode, @@ -2612,7 +2618,9 @@ static int ext4_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode) if (EXT4_DIR_LINK_MAX(dir)) return -EMLINK; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + return err; credits = (EXT4_DATA_TRANS_BLOCKS(dir->i_sb) + EXT4_INDEX_EXTRA_TRANS_BLOCKS + 3); @@ -2910,8 +2918,12 @@ static int ext4_rmdir(struct inode *dir, struct dentry *dentry) /* Initialize quotas before so that eventual writes go in * separate transaction */ - dquot_initialize(dir); - dquot_initialize(d_inode(dentry)); + retval = dquot_initialize(dir); + if (retval) + return retval; + retval = dquot_initialize(d_inode(dentry)); + if (retval) + return retval; retval = -ENOENT; bh = ext4_find_entry(dir, &dentry->d_name, &de, NULL); @@ -2980,8 +2992,12 @@ static int ext4_unlink(struct inode *dir, struct dentry *dentry) trace_ext4_unlink_enter(dir, dentry); /* Initialize quotas before so that eventual writes go * in separate transaction */ - dquot_initialize(dir); - dquot_initialize(d_inode(dentry)); + retval = dquot_initialize(dir); + if (retval) + return retval; + retval = dquot_initialize(d_inode(dentry)); + if (retval) + return retval; retval = -ENOENT; bh = ext4_find_entry(dir, &dentry->d_name, &de, NULL); @@ -3066,7 +3082,9 @@ static int ext4_symlink(struct inode *dir, goto err_free_sd; } - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + goto err_free_sd; if ((disk_link.len > EXT4_N_BLOCKS * 4)) { /* @@ -3197,7 +3215,9 @@ static int ext4_link(struct dentry *old_dentry, if (ext4_encrypted_inode(dir) && !ext4_is_child_context_consistent_with_parent(dir, inode)) return -EPERM; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) + return err; retry: handle = ext4_journal_start(dir, EXT4_HT_DIR, @@ -3476,13 +3496,20 @@ static int ext4_rename(struct inode *old_dir, struct dentry *old_dentry, int credits; u8 old_file_type; - dquot_initialize(old.dir); - dquot_initialize(new.dir); + retval = dquot_initialize(old.dir); + if (retval) + return retval; + retval = dquot_initialize(new.dir); + if (retval) + return retval; /* Initialize quotas before so that eventual writes go * in separate transaction */ - if (new.inode) - dquot_initialize(new.inode); + if (new.inode) { + retval = dquot_initialize(new.inode); + if (retval) + return retval; + } old.bh = ext4_find_entry(old.dir, &old.dentry->d_name, &old.de, NULL); if (IS_ERR(old.bh)) @@ -3678,8 +3705,12 @@ static int ext4_cross_rename(struct inode *old_dir, struct dentry *old_dentry, new.inode))) return -EPERM; - dquot_initialize(old.dir); - dquot_initialize(new.dir); + retval = dquot_initialize(old.dir); + if (retval) + return retval; + retval = dquot_initialize(new.dir); + if (retval) + return retval; old.bh = ext4_find_entry(old.dir, &old.dentry->d_name, &old.de, &old.inlined); diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c index 8a9d63a0c07103..84ba4d2b3a35f5 100644 --- a/fs/ext4/page-io.c +++ b/fs/ext4/page-io.c @@ -354,8 +354,10 @@ void ext4_io_submit(struct ext4_io_submit *io) struct bio *bio = io->io_bio; if (bio) { + int io_op = io->io_wbc->sync_mode == WB_SYNC_ALL ? + WRITE_SYNC : WRITE; bio_get(io->io_bio); - submit_bio(io->io_op, io->io_bio); + submit_bio(io_op, io->io_bio); bio_put(io->io_bio); } io->io_bio = NULL; @@ -364,7 +366,7 @@ void ext4_io_submit(struct ext4_io_submit *io) void ext4_io_submit_init(struct ext4_io_submit *io, struct writeback_control *wbc) { - io->io_op = (wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE); + io->io_wbc = wbc; io->io_bio = NULL; io->io_end = NULL; } @@ -377,6 +379,7 @@ static int io_submit_init_bio(struct ext4_io_submit *io, bio = bio_alloc(GFP_NOIO, BIO_MAX_PAGES); if (!bio) return -ENOMEM; + wbc_init_bio(io->io_wbc, bio); bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9); bio->bi_bdev = bh->b_bdev; bio->bi_end_io = ext4_end_bio; @@ -405,6 +408,7 @@ static int io_submit_add_bh(struct ext4_io_submit *io, ret = bio_add_page(io->io_bio, page, bh->b_size, bh_offset(bh)); if (ret != bh->b_size) goto submit_and_retry; + wbc_account_io(io->io_wbc, page, bh->b_size); io->io_next_block++; return 0; } diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 58987b5c514b2b..ee3878262a495c 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -60,6 +60,7 @@ static struct ext4_lazy_init *ext4_li_info; static struct mutex ext4_li_mtx; static struct ext4_features *ext4_feat; static int ext4_mballoc_ready; +static struct ratelimit_state ext4_mount_msg_ratelimit; static int ext4_load_journal(struct super_block *, struct ext4_super_block *, unsigned long journal_devnum); @@ -84,7 +85,7 @@ static void ext4_unregister_li_request(struct super_block *sb); static void ext4_clear_request_list(void); static int ext4_reserve_clusters(struct ext4_sb_info *, ext4_fsblk_t); -#if !defined(CONFIG_EXT2_FS) && !defined(CONFIG_EXT2_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT23) +#if !defined(CONFIG_EXT2_FS) && !defined(CONFIG_EXT2_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT2) static struct file_system_type ext2_fs_type = { .owner = THIS_MODULE, .name = "ext2", @@ -100,7 +101,6 @@ MODULE_ALIAS("ext2"); #endif -#if !defined(CONFIG_EXT3_FS) && !defined(CONFIG_EXT3_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT23) static struct file_system_type ext3_fs_type = { .owner = THIS_MODULE, .name = "ext3", @@ -111,9 +111,6 @@ static struct file_system_type ext3_fs_type = { MODULE_ALIAS_FS("ext3"); MODULE_ALIAS("ext3"); #define IS_EXT3_SB(sb) ((sb)->s_bdev->bd_holder == &ext3_fs_type) -#else -#define IS_EXT3_SB(sb) (0) -#endif static int ext4_verify_csum_type(struct super_block *sb, struct ext4_super_block *es) @@ -325,6 +322,22 @@ static void save_error_info(struct super_block *sb, const char *func, ext4_commit_super(sb, 1); } +/* + * The del_gendisk() function uninitializes the disk-specific data + * structures, including the bdi structure, without telling anyone + * else. Once this happens, any attempt to call mark_buffer_dirty() + * (for example, by ext4_commit_super), will cause a kernel OOPS. + * This is a kludge to prevent these oops until we can put in a proper + * hook in del_gendisk() to inform the VFS and file system layers. + */ +static int block_device_ejected(struct super_block *sb) +{ + struct inode *bd_inode = sb->s_bdev->bd_inode; + struct backing_dev_info *bdi = inode_to_bdi(bd_inode); + + return bdi->dev == NULL; +} + static void ext4_journal_commit_callback(journal_t *journal, transaction_t *txn) { struct super_block *sb = journal->j_private; @@ -1394,9 +1407,9 @@ static const struct mount_opts { {Opt_stripe, 0, MOPT_GTE0}, {Opt_resuid, 0, MOPT_GTE0}, {Opt_resgid, 0, MOPT_GTE0}, - {Opt_journal_dev, 0, MOPT_GTE0}, - {Opt_journal_path, 0, MOPT_STRING}, - {Opt_journal_ioprio, 0, MOPT_GTE0}, + {Opt_journal_dev, 0, MOPT_NO_EXT2 | MOPT_GTE0}, + {Opt_journal_path, 0, MOPT_NO_EXT2 | MOPT_STRING}, + {Opt_journal_ioprio, 0, MOPT_NO_EXT2 | MOPT_GTE0}, {Opt_data_journal, EXT4_MOUNT_JOURNAL_DATA, MOPT_NO_EXT2 | MOPT_DATAJ}, {Opt_data_ordered, EXT4_MOUNT_ORDERED_DATA, MOPT_NO_EXT2 | MOPT_DATAJ}, {Opt_data_writeback, EXT4_MOUNT_WRITEBACK_DATA, @@ -3643,6 +3656,8 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent) } if (test_opt(sb, DELALLOC)) clear_opt(sb, DELALLOC); + } else { + sb->s_iflags |= SB_I_CGROUPWB; } sb->s_flags = (sb->s_flags & ~MS_POSIXACL) | @@ -4275,9 +4290,10 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent) "the device does not support discard"); } - ext4_msg(sb, KERN_INFO, "mounted filesystem with%s. " - "Opts: %s%s%s", descr, sbi->s_es->s_mount_opts, - *sbi->s_es->s_mount_opts ? "; " : "", orig_data); + if (___ratelimit(&ext4_mount_msg_ratelimit, "EXT4-fs mount")) + ext4_msg(sb, KERN_INFO, "mounted filesystem with%s. " + "Opts: %s%s%s", descr, sbi->s_es->s_mount_opts, + *sbi->s_es->s_mount_opts ? "; " : "", orig_data); if (es->s_error_count) mod_timer(&sbi->s_err_report, jiffies + 300*HZ); /* 5 minutes */ @@ -4617,7 +4633,7 @@ static int ext4_commit_super(struct super_block *sb, int sync) struct buffer_head *sbh = EXT4_SB(sb)->s_sbh; int error = 0; - if (!sbh) + if (!sbh || block_device_ejected(sb)) return error; if (buffer_write_io_error(sbh)) { /* @@ -4665,7 +4681,8 @@ static int ext4_commit_super(struct super_block *sb, int sync) ext4_superblock_csum_set(sb); mark_buffer_dirty(sbh); if (sync) { - error = sync_dirty_buffer(sbh); + error = __sync_dirty_buffer(sbh, + test_opt(sb, BARRIER) ? WRITE_FUA : WRITE_SYNC); if (error) return error; @@ -4833,10 +4850,11 @@ static int ext4_freeze(struct super_block *sb) error = jbd2_journal_flush(journal); if (error < 0) goto out; + + /* Journal blocked and flushed, clear needs_recovery flag. */ + EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); } - /* Journal blocked and flushed, clear needs_recovery flag. */ - EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); error = ext4_commit_super(sb, 1); out: if (journal) @@ -4854,8 +4872,11 @@ static int ext4_unfreeze(struct super_block *sb) if (sb->s_flags & MS_RDONLY) return 0; - /* Reset the needs_recovery flag before the fs is unlocked. */ - EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); + if (EXT4_SB(sb)->s_journal) { + /* Reset the needs_recovery flag before the fs is unlocked. */ + EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); + } + ext4_commit_super(sb, 1); return 0; } @@ -5500,7 +5521,7 @@ static struct dentry *ext4_mount(struct file_system_type *fs_type, int flags, return mount_bdev(fs_type, flags, dev_name, data, ext4_fill_super); } -#if !defined(CONFIG_EXT2_FS) && !defined(CONFIG_EXT2_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT23) +#if !defined(CONFIG_EXT2_FS) && !defined(CONFIG_EXT2_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT2) static inline void register_as_ext2(void) { int err = register_filesystem(&ext2_fs_type); @@ -5530,7 +5551,6 @@ static inline void unregister_as_ext2(void) { } static inline int ext2_feature_set_ok(struct super_block *sb) { return 0; } #endif -#if !defined(CONFIG_EXT3_FS) && !defined(CONFIG_EXT3_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT23) static inline void register_as_ext3(void) { int err = register_filesystem(&ext3_fs_type); @@ -5556,11 +5576,6 @@ static inline int ext3_feature_set_ok(struct super_block *sb) return 0; return 1; } -#else -static inline void register_as_ext3(void) { } -static inline void unregister_as_ext3(void) { } -static inline int ext3_feature_set_ok(struct super_block *sb) { return 0; } -#endif static struct file_system_type ext4_fs_type = { .owner = THIS_MODULE, @@ -5610,6 +5625,7 @@ static int __init ext4_init_fs(void) { int i, err; + ratelimit_state_init(&ext4_mount_msg_ratelimit, 30 * HZ, 64); ext4_li_info = NULL; mutex_init(&ext4_li_mtx); diff --git a/fs/f2fs/Kconfig b/fs/f2fs/Kconfig index c629762005bc3f..b0a9dc929f887f 100644 --- a/fs/f2fs/Kconfig +++ b/fs/f2fs/Kconfig @@ -45,7 +45,7 @@ config F2FS_FS_POSIX_ACL default y help Posix Access Control Lists (ACLs) support permissions for users and - gourps beyond the owner/group/world scheme. + groups beyond the owner/group/world scheme. To learn more about Access Control Lists, visit the POSIX ACLs for Linux website . diff --git a/fs/f2fs/Makefile b/fs/f2fs/Makefile index 396be1a39e554c..08e101ed914ced 100644 --- a/fs/f2fs/Makefile +++ b/fs/f2fs/Makefile @@ -2,6 +2,7 @@ obj-$(CONFIG_F2FS_FS) += f2fs.o f2fs-y := dir.o file.o inode.o namei.o hash.o super.o inline.o f2fs-y += checkpoint.o gc.o data.o node.o segment.o recovery.o +f2fs-y += shrinker.o extent_cache.o f2fs-$(CONFIG_F2FS_STAT_FS) += debug.o f2fs-$(CONFIG_F2FS_FS_XATTR) += xattr.o f2fs-$(CONFIG_F2FS_FS_POSIX_ACL) += acl.o diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index b70bbe1a6a8ca6..c5a38e352a8095 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -69,14 +69,24 @@ struct page *get_meta_page(struct f2fs_sb_info *sbi, pgoff_t index) fio.page = page; - if (f2fs_submit_page_bio(&fio)) + if (f2fs_submit_page_bio(&fio)) { + f2fs_put_page(page, 1); goto repeat; + } lock_page(page); if (unlikely(page->mapping != mapping)) { f2fs_put_page(page, 1); goto repeat; } + + /* + * if there is any IO error when accessing device, make our filesystem + * readonly and make sure do not write checkpoint with non-uptodate + * meta page. + */ + if (unlikely(!PageUptodate(page))) + f2fs_stop_checkpoint(sbi); out: return page; } @@ -326,26 +336,18 @@ const struct address_space_operations f2fs_meta_aops = { static void __add_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type) { struct inode_management *im = &sbi->im[type]; - struct ino_entry *e; + struct ino_entry *e, *tmp; + + tmp = f2fs_kmem_cache_alloc(ino_entry_slab, GFP_NOFS); retry: - if (radix_tree_preload(GFP_NOFS)) { - cond_resched(); - goto retry; - } + radix_tree_preload(GFP_NOFS | __GFP_NOFAIL); spin_lock(&im->ino_lock); - e = radix_tree_lookup(&im->ino_root, ino); if (!e) { - e = kmem_cache_alloc(ino_entry_slab, GFP_ATOMIC); - if (!e) { - spin_unlock(&im->ino_lock); - radix_tree_preload_end(); - goto retry; - } + e = tmp; if (radix_tree_insert(&im->ino_root, ino, e)) { spin_unlock(&im->ino_lock); - kmem_cache_free(ino_entry_slab, e); radix_tree_preload_end(); goto retry; } @@ -358,6 +360,9 @@ static void __add_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type) } spin_unlock(&im->ino_lock); radix_tree_preload_end(); + + if (e != tmp) + kmem_cache_free(ino_entry_slab, tmp); } static void __remove_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type) @@ -458,24 +463,34 @@ void remove_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino) __remove_ino_entry(sbi, ino, ORPHAN_INO); } -static void recover_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino) +static int recover_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino) { - struct inode *inode = f2fs_iget(sbi->sb, ino); - f2fs_bug_on(sbi, IS_ERR(inode)); + struct inode *inode; + + inode = f2fs_iget(sbi->sb, ino); + if (IS_ERR(inode)) { + /* + * there should be a bug that we can't find the entry + * to orphan inode. + */ + f2fs_bug_on(sbi, PTR_ERR(inode) == -ENOENT); + return PTR_ERR(inode); + } + clear_nlink(inode); /* truncate all the data during iput */ iput(inode); + return 0; } -void recover_orphan_inodes(struct f2fs_sb_info *sbi) +int recover_orphan_inodes(struct f2fs_sb_info *sbi) { block_t start_blk, orphan_blocks, i, j; + int err; if (!is_set_ckpt_flags(F2FS_CKPT(sbi), CP_ORPHAN_PRESENT_FLAG)) - return; - - set_sbi_flag(sbi, SBI_POR_DOING); + return 0; start_blk = __start_cp_addr(sbi) + 1 + __cp_payload(sbi); orphan_blocks = __start_sum_addr(sbi) - 1 - __cp_payload(sbi); @@ -489,14 +504,17 @@ void recover_orphan_inodes(struct f2fs_sb_info *sbi) orphan_blk = (struct f2fs_orphan_block *)page_address(page); for (j = 0; j < le32_to_cpu(orphan_blk->entry_count); j++) { nid_t ino = le32_to_cpu(orphan_blk->ino[j]); - recover_orphan_inode(sbi, ino); + err = recover_orphan_inode(sbi, ino); + if (err) { + f2fs_put_page(page, 1); + return err; + } } f2fs_put_page(page, 1); } /* clear Orphan Flag */ clear_ckpt_flags(F2FS_CKPT(sbi), CP_ORPHAN_PRESENT_FLAG); - clear_sbi_flag(sbi, SBI_POR_DOING); - return; + return 0; } static void write_orphan_inodes(struct f2fs_sb_info *sbi, block_t start_blk) @@ -504,7 +522,7 @@ static void write_orphan_inodes(struct f2fs_sb_info *sbi, block_t start_blk) struct list_head *head; struct f2fs_orphan_block *orphan_blk = NULL; unsigned int nentries = 0; - unsigned short index; + unsigned short index = 1; unsigned short orphan_blocks; struct page *page = NULL; struct ino_entry *orphan = NULL; @@ -512,11 +530,6 @@ static void write_orphan_inodes(struct f2fs_sb_info *sbi, block_t start_blk) orphan_blocks = GET_ORPHAN_BLOCKS(im->ino_num); - for (index = 0; index < orphan_blocks; index++) - grab_meta_page(sbi, start_blk + index); - - index = 1; - /* * we don't need to do spin_lock(&im->ino_lock) here, since all the * orphan inode operations are covered under f2fs_lock_op(). @@ -527,12 +540,10 @@ static void write_orphan_inodes(struct f2fs_sb_info *sbi, block_t start_blk) /* loop for each orphan inode entry and write them in Jornal block */ list_for_each_entry(orphan, head, list) { if (!page) { - page = find_get_page(META_MAPPING(sbi), start_blk++); - f2fs_bug_on(sbi, !page); + page = grab_meta_page(sbi, start_blk++); orphan_blk = (struct f2fs_orphan_block *)page_address(page); memset(orphan_blk, 0, sizeof(*orphan_blk)); - f2fs_put_page(page, 0); } orphan_blk->ino[nentries++] = cpu_to_le32(orphan->ino); @@ -704,7 +715,8 @@ void update_dirty_page(struct inode *inode, struct page *page) struct inode_entry *new; int ret = 0; - if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode)) + if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) && + !S_ISLNK(inode->i_mode)) return; if (!S_ISDIR(inode->i_mode)) { @@ -892,12 +904,15 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc) __u32 crc32 = 0; int i; int cp_payload_blks = __cp_payload(sbi); + block_t discard_blk = NEXT_FREE_BLKADDR(sbi, curseg); + bool invalidate = false; /* * This avoids to conduct wrong roll-forward operations and uses * metapages, so should be called prior to sync_meta_pages below. */ - discard_next_dnode(sbi, NEXT_FREE_BLKADDR(sbi, curseg)); + if (discard_next_dnode(sbi, discard_blk)) + invalidate = true; /* Flush all the NAT/SIT pages */ while (get_pages(sbi, F2FS_DIRTY_META)) { @@ -1026,6 +1041,14 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc) /* wait for previous submitted meta pages writeback */ wait_on_all_pages_writeback(sbi); + /* + * invalidate meta page which is used temporarily for zeroing out + * block at the end of warm node chain. + */ + if (invalidate) + invalidate_mapping_pages(META_MAPPING(sbi), discard_blk, + discard_blk); + release_dirty_inode(sbi); if (unlikely(f2fs_cp_error(sbi))) diff --git a/fs/f2fs/crypto_key.c b/fs/f2fs/crypto_key.c index 95b8f936f00b64..9f77de2ef317e5 100644 --- a/fs/f2fs/crypto_key.c +++ b/fs/f2fs/crypto_key.c @@ -92,8 +92,7 @@ static void f2fs_free_crypt_info(struct f2fs_crypt_info *ci) if (!ci) return; - if (ci->ci_keyring_key) - key_put(ci->ci_keyring_key); + key_put(ci->ci_keyring_key); crypto_free_ablkcipher(ci->ci_ctfm); kmem_cache_free(f2fs_crypt_info_cachep, ci); } diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index c414d49aa2de17..a82abe921b895c 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -26,9 +27,6 @@ #include "trace.h" #include -static struct kmem_cache *extent_tree_slab; -static struct kmem_cache *extent_node_slab; - static void f2fs_read_end_io(struct bio *bio) { struct bio_vec *bvec; @@ -92,8 +90,7 @@ static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr, { struct bio *bio; - /* No failure on bio allocation */ - bio = bio_alloc(GFP_NOIO, npages); + bio = f2fs_bio_alloc(npages); bio->bi_bdev = sbi->sb->s_bdev; bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blk_addr); @@ -158,7 +155,6 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio) if (bio_add_page(bio, page, PAGE_CACHE_SIZE, 0) < PAGE_CACHE_SIZE) { bio_put(bio); - f2fs_put_page(page, 1); return -EFAULT; } @@ -266,645 +262,17 @@ int f2fs_reserve_block(struct dnode_of_data *dn, pgoff_t index) return err; } -static bool lookup_extent_info(struct inode *inode, pgoff_t pgofs, - struct extent_info *ei) -{ - struct f2fs_inode_info *fi = F2FS_I(inode); - pgoff_t start_fofs, end_fofs; - block_t start_blkaddr; - - read_lock(&fi->ext_lock); - if (fi->ext.len == 0) { - read_unlock(&fi->ext_lock); - return false; - } - - stat_inc_total_hit(inode->i_sb); - - start_fofs = fi->ext.fofs; - end_fofs = fi->ext.fofs + fi->ext.len - 1; - start_blkaddr = fi->ext.blk; - - if (pgofs >= start_fofs && pgofs <= end_fofs) { - *ei = fi->ext; - stat_inc_read_hit(inode->i_sb); - read_unlock(&fi->ext_lock); - return true; - } - read_unlock(&fi->ext_lock); - return false; -} - -static bool update_extent_info(struct inode *inode, pgoff_t fofs, - block_t blkaddr) +int f2fs_get_block(struct dnode_of_data *dn, pgoff_t index) { - struct f2fs_inode_info *fi = F2FS_I(inode); - pgoff_t start_fofs, end_fofs; - block_t start_blkaddr, end_blkaddr; - int need_update = true; - - write_lock(&fi->ext_lock); - - start_fofs = fi->ext.fofs; - end_fofs = fi->ext.fofs + fi->ext.len - 1; - start_blkaddr = fi->ext.blk; - end_blkaddr = fi->ext.blk + fi->ext.len - 1; - - /* Drop and initialize the matched extent */ - if (fi->ext.len == 1 && fofs == start_fofs) - fi->ext.len = 0; - - /* Initial extent */ - if (fi->ext.len == 0) { - if (blkaddr != NULL_ADDR) { - fi->ext.fofs = fofs; - fi->ext.blk = blkaddr; - fi->ext.len = 1; - } - goto end_update; - } - - /* Front merge */ - if (fofs == start_fofs - 1 && blkaddr == start_blkaddr - 1) { - fi->ext.fofs--; - fi->ext.blk--; - fi->ext.len++; - goto end_update; - } - - /* Back merge */ - if (fofs == end_fofs + 1 && blkaddr == end_blkaddr + 1) { - fi->ext.len++; - goto end_update; - } - - /* Split the existing extent */ - if (fi->ext.len > 1 && - fofs >= start_fofs && fofs <= end_fofs) { - if ((end_fofs - fofs) < (fi->ext.len >> 1)) { - fi->ext.len = fofs - start_fofs; - } else { - fi->ext.fofs = fofs + 1; - fi->ext.blk = start_blkaddr + fofs - start_fofs + 1; - fi->ext.len -= fofs - start_fofs + 1; - } - } else { - need_update = false; - } - - /* Finally, if the extent is very fragmented, let's drop the cache. */ - if (fi->ext.len < F2FS_MIN_EXTENT_LEN) { - fi->ext.len = 0; - set_inode_flag(fi, FI_NO_EXTENT); - need_update = true; - } -end_update: - write_unlock(&fi->ext_lock); - return need_update; -} - -static struct extent_node *__attach_extent_node(struct f2fs_sb_info *sbi, - struct extent_tree *et, struct extent_info *ei, - struct rb_node *parent, struct rb_node **p) -{ - struct extent_node *en; - - en = kmem_cache_alloc(extent_node_slab, GFP_ATOMIC); - if (!en) - return NULL; - - en->ei = *ei; - INIT_LIST_HEAD(&en->list); - - rb_link_node(&en->rb_node, parent, p); - rb_insert_color(&en->rb_node, &et->root); - et->count++; - atomic_inc(&sbi->total_ext_node); - return en; -} - -static void __detach_extent_node(struct f2fs_sb_info *sbi, - struct extent_tree *et, struct extent_node *en) -{ - rb_erase(&en->rb_node, &et->root); - et->count--; - atomic_dec(&sbi->total_ext_node); - - if (et->cached_en == en) - et->cached_en = NULL; -} - -static struct extent_tree *__find_extent_tree(struct f2fs_sb_info *sbi, - nid_t ino) -{ - struct extent_tree *et; - - down_read(&sbi->extent_tree_lock); - et = radix_tree_lookup(&sbi->extent_tree_root, ino); - if (!et) { - up_read(&sbi->extent_tree_lock); - return NULL; - } - atomic_inc(&et->refcount); - up_read(&sbi->extent_tree_lock); - - return et; -} - -static struct extent_tree *__grab_extent_tree(struct inode *inode) -{ - struct f2fs_sb_info *sbi = F2FS_I_SB(inode); - struct extent_tree *et; - nid_t ino = inode->i_ino; - - down_write(&sbi->extent_tree_lock); - et = radix_tree_lookup(&sbi->extent_tree_root, ino); - if (!et) { - et = f2fs_kmem_cache_alloc(extent_tree_slab, GFP_NOFS); - f2fs_radix_tree_insert(&sbi->extent_tree_root, ino, et); - memset(et, 0, sizeof(struct extent_tree)); - et->ino = ino; - et->root = RB_ROOT; - et->cached_en = NULL; - rwlock_init(&et->lock); - atomic_set(&et->refcount, 0); - et->count = 0; - sbi->total_ext_tree++; - } - atomic_inc(&et->refcount); - up_write(&sbi->extent_tree_lock); - - return et; -} - -static struct extent_node *__lookup_extent_tree(struct extent_tree *et, - unsigned int fofs) -{ - struct rb_node *node = et->root.rb_node; - struct extent_node *en; - - if (et->cached_en) { - struct extent_info *cei = &et->cached_en->ei; - - if (cei->fofs <= fofs && cei->fofs + cei->len > fofs) - return et->cached_en; - } - - while (node) { - en = rb_entry(node, struct extent_node, rb_node); - - if (fofs < en->ei.fofs) { - node = node->rb_left; - } else if (fofs >= en->ei.fofs + en->ei.len) { - node = node->rb_right; - } else { - et->cached_en = en; - return en; - } - } - return NULL; -} - -static struct extent_node *__try_back_merge(struct f2fs_sb_info *sbi, - struct extent_tree *et, struct extent_node *en) -{ - struct extent_node *prev; - struct rb_node *node; - - node = rb_prev(&en->rb_node); - if (!node) - return NULL; - - prev = rb_entry(node, struct extent_node, rb_node); - if (__is_back_mergeable(&en->ei, &prev->ei)) { - en->ei.fofs = prev->ei.fofs; - en->ei.blk = prev->ei.blk; - en->ei.len += prev->ei.len; - __detach_extent_node(sbi, et, prev); - return prev; - } - return NULL; -} - -static struct extent_node *__try_front_merge(struct f2fs_sb_info *sbi, - struct extent_tree *et, struct extent_node *en) -{ - struct extent_node *next; - struct rb_node *node; - - node = rb_next(&en->rb_node); - if (!node) - return NULL; - - next = rb_entry(node, struct extent_node, rb_node); - if (__is_front_mergeable(&en->ei, &next->ei)) { - en->ei.len += next->ei.len; - __detach_extent_node(sbi, et, next); - return next; - } - return NULL; -} - -static struct extent_node *__insert_extent_tree(struct f2fs_sb_info *sbi, - struct extent_tree *et, struct extent_info *ei, - struct extent_node **den) -{ - struct rb_node **p = &et->root.rb_node; - struct rb_node *parent = NULL; - struct extent_node *en; - - while (*p) { - parent = *p; - en = rb_entry(parent, struct extent_node, rb_node); - - if (ei->fofs < en->ei.fofs) { - if (__is_front_mergeable(ei, &en->ei)) { - f2fs_bug_on(sbi, !den); - en->ei.fofs = ei->fofs; - en->ei.blk = ei->blk; - en->ei.len += ei->len; - *den = __try_back_merge(sbi, et, en); - return en; - } - p = &(*p)->rb_left; - } else if (ei->fofs >= en->ei.fofs + en->ei.len) { - if (__is_back_mergeable(ei, &en->ei)) { - f2fs_bug_on(sbi, !den); - en->ei.len += ei->len; - *den = __try_front_merge(sbi, et, en); - return en; - } - p = &(*p)->rb_right; - } else { - f2fs_bug_on(sbi, 1); - } - } - - return __attach_extent_node(sbi, et, ei, parent, p); -} - -static unsigned int __free_extent_tree(struct f2fs_sb_info *sbi, - struct extent_tree *et, bool free_all) -{ - struct rb_node *node, *next; - struct extent_node *en; - unsigned int count = et->count; - - node = rb_first(&et->root); - while (node) { - next = rb_next(node); - en = rb_entry(node, struct extent_node, rb_node); - - if (free_all) { - spin_lock(&sbi->extent_lock); - if (!list_empty(&en->list)) - list_del_init(&en->list); - spin_unlock(&sbi->extent_lock); - } - - if (free_all || list_empty(&en->list)) { - __detach_extent_node(sbi, et, en); - kmem_cache_free(extent_node_slab, en); - } - node = next; - } - - return count - et->count; -} - -static void f2fs_init_extent_tree(struct inode *inode, - struct f2fs_extent *i_ext) -{ - struct f2fs_sb_info *sbi = F2FS_I_SB(inode); - struct extent_tree *et; - struct extent_node *en; struct extent_info ei; + struct inode *inode = dn->inode; - if (le32_to_cpu(i_ext->len) < F2FS_MIN_EXTENT_LEN) - return; - - et = __grab_extent_tree(inode); - - write_lock(&et->lock); - if (et->count) - goto out; - - set_extent_info(&ei, le32_to_cpu(i_ext->fofs), - le32_to_cpu(i_ext->blk), le32_to_cpu(i_ext->len)); - - en = __insert_extent_tree(sbi, et, &ei, NULL); - if (en) { - et->cached_en = en; - - spin_lock(&sbi->extent_lock); - list_add_tail(&en->list, &sbi->extent_list); - spin_unlock(&sbi->extent_lock); - } -out: - write_unlock(&et->lock); - atomic_dec(&et->refcount); -} - -static bool f2fs_lookup_extent_tree(struct inode *inode, pgoff_t pgofs, - struct extent_info *ei) -{ - struct f2fs_sb_info *sbi = F2FS_I_SB(inode); - struct extent_tree *et; - struct extent_node *en; - - trace_f2fs_lookup_extent_tree_start(inode, pgofs); - - et = __find_extent_tree(sbi, inode->i_ino); - if (!et) - return false; - - read_lock(&et->lock); - en = __lookup_extent_tree(et, pgofs); - if (en) { - *ei = en->ei; - spin_lock(&sbi->extent_lock); - if (!list_empty(&en->list)) - list_move_tail(&en->list, &sbi->extent_list); - spin_unlock(&sbi->extent_lock); - stat_inc_read_hit(sbi->sb); - } - stat_inc_total_hit(sbi->sb); - read_unlock(&et->lock); - - trace_f2fs_lookup_extent_tree_end(inode, pgofs, en); - - atomic_dec(&et->refcount); - return en ? true : false; -} - -static void f2fs_update_extent_tree(struct inode *inode, pgoff_t fofs, - block_t blkaddr) -{ - struct f2fs_sb_info *sbi = F2FS_I_SB(inode); - struct extent_tree *et; - struct extent_node *en = NULL, *en1 = NULL, *en2 = NULL, *en3 = NULL; - struct extent_node *den = NULL; - struct extent_info ei, dei; - unsigned int endofs; - - trace_f2fs_update_extent_tree(inode, fofs, blkaddr); - - et = __grab_extent_tree(inode); - - write_lock(&et->lock); - - /* 1. lookup and remove existing extent info in cache */ - en = __lookup_extent_tree(et, fofs); - if (!en) - goto update_extent; - - dei = en->ei; - __detach_extent_node(sbi, et, en); - - /* 2. if extent can be split more, split and insert the left part */ - if (dei.len > 1) { - /* insert left part of split extent into cache */ - if (fofs - dei.fofs >= F2FS_MIN_EXTENT_LEN) { - set_extent_info(&ei, dei.fofs, dei.blk, - fofs - dei.fofs); - en1 = __insert_extent_tree(sbi, et, &ei, NULL); - } - - /* insert right part of split extent into cache */ - endofs = dei.fofs + dei.len - 1; - if (endofs - fofs >= F2FS_MIN_EXTENT_LEN) { - set_extent_info(&ei, fofs + 1, - fofs - dei.fofs + dei.blk, endofs - fofs); - en2 = __insert_extent_tree(sbi, et, &ei, NULL); - } - } - -update_extent: - /* 3. update extent in extent cache */ - if (blkaddr) { - set_extent_info(&ei, fofs, blkaddr, 1); - en3 = __insert_extent_tree(sbi, et, &ei, &den); - } - - /* 4. update in global extent list */ - spin_lock(&sbi->extent_lock); - if (en && !list_empty(&en->list)) - list_del(&en->list); - /* - * en1 and en2 split from en, they will become more and more smaller - * fragments after splitting several times. So if the length is smaller - * than F2FS_MIN_EXTENT_LEN, we will not add them into extent tree. - */ - if (en1) - list_add_tail(&en1->list, &sbi->extent_list); - if (en2) - list_add_tail(&en2->list, &sbi->extent_list); - if (en3) { - if (list_empty(&en3->list)) - list_add_tail(&en3->list, &sbi->extent_list); - else - list_move_tail(&en3->list, &sbi->extent_list); - } - if (den && !list_empty(&den->list)) - list_del(&den->list); - spin_unlock(&sbi->extent_lock); - - /* 5. release extent node */ - if (en) - kmem_cache_free(extent_node_slab, en); - if (den) - kmem_cache_free(extent_node_slab, den); - - write_unlock(&et->lock); - atomic_dec(&et->refcount); -} - -void f2fs_preserve_extent_tree(struct inode *inode) -{ - struct extent_tree *et; - struct extent_info *ext = &F2FS_I(inode)->ext; - bool sync = false; - - if (!test_opt(F2FS_I_SB(inode), EXTENT_CACHE)) - return; - - et = __find_extent_tree(F2FS_I_SB(inode), inode->i_ino); - if (!et) { - if (ext->len) { - ext->len = 0; - update_inode_page(inode); - } - return; - } - - read_lock(&et->lock); - if (et->count) { - struct extent_node *en; - - if (et->cached_en) { - en = et->cached_en; - } else { - struct rb_node *node = rb_first(&et->root); - - if (!node) - node = rb_last(&et->root); - en = rb_entry(node, struct extent_node, rb_node); - } - - if (__is_extent_same(ext, &en->ei)) - goto out; - - *ext = en->ei; - sync = true; - } else if (ext->len) { - ext->len = 0; - sync = true; - } -out: - read_unlock(&et->lock); - atomic_dec(&et->refcount); - - if (sync) - update_inode_page(inode); -} - -void f2fs_shrink_extent_tree(struct f2fs_sb_info *sbi, int nr_shrink) -{ - struct extent_tree *treevec[EXT_TREE_VEC_SIZE]; - struct extent_node *en, *tmp; - unsigned long ino = F2FS_ROOT_INO(sbi); - struct radix_tree_iter iter; - void **slot; - unsigned int found; - unsigned int node_cnt = 0, tree_cnt = 0; - - if (!test_opt(sbi, EXTENT_CACHE)) - return; - - if (available_free_memory(sbi, EXTENT_CACHE)) - return; - - spin_lock(&sbi->extent_lock); - list_for_each_entry_safe(en, tmp, &sbi->extent_list, list) { - if (!nr_shrink--) - break; - list_del_init(&en->list); - } - spin_unlock(&sbi->extent_lock); - - down_read(&sbi->extent_tree_lock); - while ((found = radix_tree_gang_lookup(&sbi->extent_tree_root, - (void **)treevec, ino, EXT_TREE_VEC_SIZE))) { - unsigned i; - - ino = treevec[found - 1]->ino + 1; - for (i = 0; i < found; i++) { - struct extent_tree *et = treevec[i]; - - atomic_inc(&et->refcount); - write_lock(&et->lock); - node_cnt += __free_extent_tree(sbi, et, false); - write_unlock(&et->lock); - atomic_dec(&et->refcount); - } - } - up_read(&sbi->extent_tree_lock); - - down_write(&sbi->extent_tree_lock); - radix_tree_for_each_slot(slot, &sbi->extent_tree_root, &iter, - F2FS_ROOT_INO(sbi)) { - struct extent_tree *et = (struct extent_tree *)*slot; - - if (!atomic_read(&et->refcount) && !et->count) { - radix_tree_delete(&sbi->extent_tree_root, et->ino); - kmem_cache_free(extent_tree_slab, et); - sbi->total_ext_tree--; - tree_cnt++; - } - } - up_write(&sbi->extent_tree_lock); - - trace_f2fs_shrink_extent_tree(sbi, node_cnt, tree_cnt); -} - -void f2fs_destroy_extent_tree(struct inode *inode) -{ - struct f2fs_sb_info *sbi = F2FS_I_SB(inode); - struct extent_tree *et; - unsigned int node_cnt = 0; - - if (!test_opt(sbi, EXTENT_CACHE)) - return; - - et = __find_extent_tree(sbi, inode->i_ino); - if (!et) - goto out; - - /* free all extent info belong to this extent tree */ - write_lock(&et->lock); - node_cnt = __free_extent_tree(sbi, et, true); - write_unlock(&et->lock); - - atomic_dec(&et->refcount); - - /* try to find and delete extent tree entry in radix tree */ - down_write(&sbi->extent_tree_lock); - et = radix_tree_lookup(&sbi->extent_tree_root, inode->i_ino); - if (!et) { - up_write(&sbi->extent_tree_lock); - goto out; + if (f2fs_lookup_extent_cache(inode, index, &ei)) { + dn->data_blkaddr = ei.blk + index - ei.fofs; + return 0; } - f2fs_bug_on(sbi, atomic_read(&et->refcount) || et->count); - radix_tree_delete(&sbi->extent_tree_root, inode->i_ino); - kmem_cache_free(extent_tree_slab, et); - sbi->total_ext_tree--; - up_write(&sbi->extent_tree_lock); -out: - trace_f2fs_destroy_extent_tree(inode, node_cnt); - return; -} - -void f2fs_init_extent_cache(struct inode *inode, struct f2fs_extent *i_ext) -{ - if (test_opt(F2FS_I_SB(inode), EXTENT_CACHE)) - f2fs_init_extent_tree(inode, i_ext); - - write_lock(&F2FS_I(inode)->ext_lock); - get_extent_info(&F2FS_I(inode)->ext, *i_ext); - write_unlock(&F2FS_I(inode)->ext_lock); -} -static bool f2fs_lookup_extent_cache(struct inode *inode, pgoff_t pgofs, - struct extent_info *ei) -{ - if (is_inode_flag_set(F2FS_I(inode), FI_NO_EXTENT)) - return false; - - if (test_opt(F2FS_I_SB(inode), EXTENT_CACHE)) - return f2fs_lookup_extent_tree(inode, pgofs, ei); - - return lookup_extent_info(inode, pgofs, ei); -} - -void f2fs_update_extent_cache(struct dnode_of_data *dn) -{ - struct f2fs_inode_info *fi = F2FS_I(dn->inode); - pgoff_t fofs; - - f2fs_bug_on(F2FS_I_SB(dn->inode), dn->data_blkaddr == NEW_ADDR); - - if (is_inode_flag_set(fi, FI_NO_EXTENT)) - return; - - fofs = start_bidx_of_node(ofs_of_node(dn->node_page), fi) + - dn->ofs_in_node; - - if (test_opt(F2FS_I_SB(dn->inode), EXTENT_CACHE)) - return f2fs_update_extent_tree(dn->inode, fofs, - dn->data_blkaddr); - - if (update_extent_info(dn->inode, fofs, dn->data_blkaddr)) - sync_inode_page(dn); + return f2fs_reserve_block(dn, index); } struct page *get_read_data_page(struct inode *inode, pgoff_t index, int rw) @@ -935,15 +303,13 @@ struct page *get_read_data_page(struct inode *inode, pgoff_t index, int rw) set_new_dnode(&dn, inode, NULL, NULL, 0); err = get_dnode_of_data(&dn, index, LOOKUP_NODE); - if (err) { - f2fs_put_page(page, 1); - return ERR_PTR(err); - } + if (err) + goto put_err; f2fs_put_dnode(&dn); if (unlikely(dn.data_blkaddr == NULL_ADDR)) { - f2fs_put_page(page, 1); - return ERR_PTR(-ENOENT); + err = -ENOENT; + goto put_err; } got_it: if (PageUptodate(page)) { @@ -968,8 +334,12 @@ struct page *get_read_data_page(struct inode *inode, pgoff_t index, int rw) fio.page = page; err = f2fs_submit_page_bio(&fio); if (err) - return ERR_PTR(err); + goto put_err; return page; + +put_err: + f2fs_put_page(page, 1); + return ERR_PTR(err); } struct page *find_data_page(struct inode *inode, pgoff_t index) @@ -1030,7 +400,8 @@ struct page *get_lock_data_page(struct inode *inode, pgoff_t index) * * Also, caller should grab and release a rwsem by calling f2fs_lock_op() and * f2fs_unlock_op(). - * Note that, ipage is set only by make_empty_dir. + * Note that, ipage is set only by make_empty_dir, and if any error occur, + * ipage should be released by this function. */ struct page *get_new_data_page(struct inode *inode, struct page *ipage, pgoff_t index, bool new_i_size) @@ -1041,8 +412,14 @@ struct page *get_new_data_page(struct inode *inode, int err; repeat: page = grab_cache_page(mapping, index); - if (!page) + if (!page) { + /* + * before exiting, we should make sure ipage will be released + * if any error occur. + */ + f2fs_put_page(ipage, 1); return ERR_PTR(-ENOMEM); + } set_new_dnode(&dn, inode, ipage, NULL, 0); err = f2fs_reserve_block(&dn, index); @@ -1107,8 +484,6 @@ static int __allocate_data_block(struct dnode_of_data *dn) allocate_data_block(sbi, NULL, dn->data_blkaddr, &dn->data_blkaddr, &sum, seg); - - /* direct IO doesn't use extent cache to maximize the performance */ set_data_blkaddr(dn); /* update i_size */ @@ -1117,6 +492,9 @@ static int __allocate_data_block(struct dnode_of_data *dn) if (i_size_read(dn->inode) < ((fofs + 1) << PAGE_CACHE_SHIFT)) i_size_write(dn->inode, ((fofs + 1) << PAGE_CACHE_SHIFT)); + /* direct IO doesn't use extent cache to maximize the performance */ + f2fs_drop_largest_extent(dn->inode, fofs); + return 0; } @@ -1183,7 +561,7 @@ static void __allocate_data_blocks(struct inode *inode, loff_t offset, * c. give the block addresses to blockdev */ static int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map, - int create, bool fiemap) + int create, int flag) { unsigned int maxblocks = map->m_len; struct dnode_of_data dn; @@ -1217,8 +595,19 @@ static int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map, err = 0; goto unlock_out; } - if (dn.data_blkaddr == NEW_ADDR && !fiemap) - goto put_out; + if (dn.data_blkaddr == NEW_ADDR) { + if (flag == F2FS_GET_BLOCK_BMAP) { + err = -ENOENT; + goto put_out; + } else if (flag == F2FS_GET_BLOCK_READ || + flag == F2FS_GET_BLOCK_DIO) { + goto put_out; + } + /* + * if it is in fiemap call path (flag = F2FS_GET_BLOCK_FIEMAP), + * mark it as mapped and unwritten block. + */ + } if (dn.data_blkaddr != NULL_ADDR) { map->m_flags = F2FS_MAP_MAPPED; @@ -1233,6 +622,8 @@ static int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map, map->m_flags = F2FS_MAP_NEW | F2FS_MAP_MAPPED; map->m_pblk = dn.data_blkaddr; } else { + if (flag == F2FS_GET_BLOCK_BMAP) + err = -ENOENT; goto put_out; } @@ -1255,7 +646,9 @@ static int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map, err = 0; goto unlock_out; } - if (dn.data_blkaddr == NEW_ADDR && !fiemap) + + if (dn.data_blkaddr == NEW_ADDR && + flag != F2FS_GET_BLOCK_FIEMAP) goto put_out; end_offset = ADDRS_PER_PAGE(dn.node_page, F2FS_I(inode)); @@ -1297,7 +690,7 @@ static int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map, } static int __get_data_block(struct inode *inode, sector_t iblock, - struct buffer_head *bh, int create, bool fiemap) + struct buffer_head *bh, int create, int flag) { struct f2fs_map_blocks map; int ret; @@ -1305,7 +698,7 @@ static int __get_data_block(struct inode *inode, sector_t iblock, map.m_lblk = iblock; map.m_len = bh->b_size >> inode->i_blkbits; - ret = f2fs_map_blocks(inode, &map, create, fiemap); + ret = f2fs_map_blocks(inode, &map, create, flag); if (!ret) { map_bh(bh, inode->i_sb, map.m_pblk); bh->b_state = (bh->b_state & ~F2FS_MAP_FLAGS) | map.m_flags; @@ -1315,15 +708,23 @@ static int __get_data_block(struct inode *inode, sector_t iblock, } static int get_data_block(struct inode *inode, sector_t iblock, + struct buffer_head *bh_result, int create, int flag) +{ + return __get_data_block(inode, iblock, bh_result, create, flag); +} + +static int get_data_block_dio(struct inode *inode, sector_t iblock, struct buffer_head *bh_result, int create) { - return __get_data_block(inode, iblock, bh_result, create, false); + return __get_data_block(inode, iblock, bh_result, create, + F2FS_GET_BLOCK_DIO); } -static int get_data_block_fiemap(struct inode *inode, sector_t iblock, +static int get_data_block_bmap(struct inode *inode, sector_t iblock, struct buffer_head *bh_result, int create) { - return __get_data_block(inode, iblock, bh_result, create, true); + return __get_data_block(inode, iblock, bh_result, create, + F2FS_GET_BLOCK_BMAP); } static inline sector_t logical_to_blk(struct inode *inode, loff_t offset) @@ -1367,7 +768,8 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, memset(&map_bh, 0, sizeof(struct buffer_head)); map_bh.b_size = len; - ret = get_data_block_fiemap(inode, start_blk, &map_bh, 0); + ret = get_data_block(inode, start_blk, &map_bh, 0, + F2FS_GET_BLOCK_FIEMAP); if (ret) goto out; @@ -1770,6 +1172,137 @@ static int __f2fs_writepage(struct page *page, struct writeback_control *wbc, return ret; } +/* + * This function was copied from write_cche_pages from mm/page-writeback.c. + * The major change is making write step of cold data page separately from + * warm/hot data page. + */ +static int f2fs_write_cache_pages(struct address_space *mapping, + struct writeback_control *wbc, writepage_t writepage, + void *data) +{ + int ret = 0; + int done = 0; + struct pagevec pvec; + int nr_pages; + pgoff_t uninitialized_var(writeback_index); + pgoff_t index; + pgoff_t end; /* Inclusive */ + pgoff_t done_index; + int cycled; + int range_whole = 0; + int tag; + int step = 0; + + pagevec_init(&pvec, 0); +next: + if (wbc->range_cyclic) { + writeback_index = mapping->writeback_index; /* prev offset */ + index = writeback_index; + if (index == 0) + cycled = 1; + else + cycled = 0; + end = -1; + } else { + index = wbc->range_start >> PAGE_CACHE_SHIFT; + end = wbc->range_end >> PAGE_CACHE_SHIFT; + if (wbc->range_start == 0 && wbc->range_end == LLONG_MAX) + range_whole = 1; + cycled = 1; /* ignore range_cyclic tests */ + } + if (wbc->sync_mode == WB_SYNC_ALL || wbc->tagged_writepages) + tag = PAGECACHE_TAG_TOWRITE; + else + tag = PAGECACHE_TAG_DIRTY; +retry: + if (wbc->sync_mode == WB_SYNC_ALL || wbc->tagged_writepages) + tag_pages_for_writeback(mapping, index, end); + done_index = index; + while (!done && (index <= end)) { + int i; + + nr_pages = pagevec_lookup_tag(&pvec, mapping, &index, tag, + min(end - index, (pgoff_t)PAGEVEC_SIZE - 1) + 1); + if (nr_pages == 0) + break; + + for (i = 0; i < nr_pages; i++) { + struct page *page = pvec.pages[i]; + + if (page->index > end) { + done = 1; + break; + } + + done_index = page->index; + + lock_page(page); + + if (unlikely(page->mapping != mapping)) { +continue_unlock: + unlock_page(page); + continue; + } + + if (!PageDirty(page)) { + /* someone wrote it for us */ + goto continue_unlock; + } + + if (step == is_cold_data(page)) + goto continue_unlock; + + if (PageWriteback(page)) { + if (wbc->sync_mode != WB_SYNC_NONE) + f2fs_wait_on_page_writeback(page, DATA); + else + goto continue_unlock; + } + + BUG_ON(PageWriteback(page)); + if (!clear_page_dirty_for_io(page)) + goto continue_unlock; + + ret = (*writepage)(page, wbc, data); + if (unlikely(ret)) { + if (ret == AOP_WRITEPAGE_ACTIVATE) { + unlock_page(page); + ret = 0; + } else { + done_index = page->index + 1; + done = 1; + break; + } + } + + if (--wbc->nr_to_write <= 0 && + wbc->sync_mode == WB_SYNC_NONE) { + done = 1; + break; + } + } + pagevec_release(&pvec); + cond_resched(); + } + + if (step < 1) { + step++; + goto next; + } + + if (!cycled && !done) { + cycled = 1; + index = 0; + end = writeback_index - 1; + goto retry; + } + if (wbc->range_cyclic || (range_whole && wbc->nr_to_write > 0)) + mapping->writeback_index = done_index; + + return ret; +} + static int f2fs_write_data_pages(struct address_space *mapping, struct writeback_control *wbc) { @@ -1785,6 +1318,10 @@ static int f2fs_write_data_pages(struct address_space *mapping, if (!mapping->a_ops->writepage) return 0; + /* skip writing if there is no dirty page in this inode */ + if (!get_dirty_pages(inode) && wbc->sync_mode == WB_SYNC_NONE) + return 0; + if (S_ISDIR(inode->i_mode) && wbc->sync_mode == WB_SYNC_NONE && get_dirty_pages(inode) < nr_pages_to_skip(sbi, DATA) && available_free_memory(sbi, DIRTY_DENTS)) @@ -1800,12 +1337,11 @@ static int f2fs_write_data_pages(struct address_space *mapping, mutex_lock(&sbi->writepages); locked = true; } - ret = write_cache_pages(mapping, wbc, __f2fs_writepage, mapping); + ret = f2fs_write_cache_pages(mapping, wbc, __f2fs_writepage, mapping); + f2fs_submit_merged_bio(sbi, DATA, WRITE); if (locked) mutex_unlock(&sbi->writepages); - f2fs_submit_merged_bio(sbi, DATA, WRITE); - remove_dirty_dir_inode(inode); wbc->nr_to_write = max((long)0, wbc->nr_to_write - diff); @@ -1832,7 +1368,8 @@ static int f2fs_write_begin(struct file *file, struct address_space *mapping, { struct inode *inode = mapping->host; struct f2fs_sb_info *sbi = F2FS_I_SB(inode); - struct page *page, *ipage; + struct page *page = NULL; + struct page *ipage; pgoff_t index = ((unsigned long long) pos) >> PAGE_CACHE_SHIFT; struct dnode_of_data dn; int err = 0; @@ -1882,25 +1419,28 @@ static int f2fs_write_begin(struct file *file, struct address_space *mapping, if (err) goto put_fail; } - err = f2fs_reserve_block(&dn, index); + + err = f2fs_get_block(&dn, index); if (err) goto put_fail; put_next: f2fs_put_dnode(&dn); f2fs_unlock_op(sbi); - if ((len == PAGE_CACHE_SIZE) || PageUptodate(page)) - return 0; - f2fs_wait_on_page_writeback(page, DATA); + if (len == PAGE_CACHE_SIZE) + goto out_update; + if (PageUptodate(page)) + goto out_clear; + if ((pos & PAGE_CACHE_MASK) >= i_size_read(inode)) { unsigned start = pos & (PAGE_CACHE_SIZE - 1); unsigned end = start + len; /* Reading beyond i_size is simple: memset to zero */ zero_user_segments(page, 0, start, end, PAGE_CACHE_SIZE); - goto out; + goto out_update; } if (dn.data_blkaddr == NEW_ADDR) { @@ -1920,7 +1460,6 @@ static int f2fs_write_begin(struct file *file, struct address_space *mapping, lock_page(page); if (unlikely(!PageUptodate(page))) { - f2fs_put_page(page, 1); err = -EIO; goto fail; } @@ -1932,14 +1471,13 @@ static int f2fs_write_begin(struct file *file, struct address_space *mapping, /* avoid symlink page */ if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) { err = f2fs_decrypt_one(inode, page); - if (err) { - f2fs_put_page(page, 1); + if (err) goto fail; - } } } -out: +out_update: SetPageUptodate(page); +out_clear: clear_cold_data(page); return 0; @@ -1947,8 +1485,8 @@ static int f2fs_write_begin(struct file *file, struct address_space *mapping, f2fs_put_dnode(&dn); unlock_fail: f2fs_unlock_op(sbi); - f2fs_put_page(page, 1); fail: + f2fs_put_page(page, 1); f2fs_write_failed(mapping, pos + len); return err; } @@ -1979,9 +1517,6 @@ static int check_direct_IO(struct inode *inode, struct iov_iter *iter, { unsigned blocksize_mask = inode->i_sb->s_blocksize - 1; - if (iov_iter_rw(iter) == READ) - return 0; - if (offset & blocksize_mask) return -EINVAL; @@ -2010,15 +1545,16 @@ static ssize_t f2fs_direct_IO(struct kiocb *iocb, struct iov_iter *iter, if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) return 0; - if (check_direct_IO(inode, iter, offset)) - return 0; + err = check_direct_IO(inode, iter, offset); + if (err) + return err; trace_f2fs_direct_IO_enter(inode, offset, count, iov_iter_rw(iter)); if (iov_iter_rw(iter) == WRITE) __allocate_data_blocks(inode, offset, count); - err = blockdev_direct_IO(iocb, inode, iter, offset, get_data_block); + err = blockdev_direct_IO(iocb, inode, iter, offset, get_data_block_dio); if (err < 0 && iov_iter_rw(iter) == WRITE) f2fs_write_failed(mapping, offset + count); @@ -2045,6 +1581,11 @@ void f2fs_invalidate_page(struct page *page, unsigned int offset, else inode_dec_dirty_pages(inode); } + + /* This is atomic written page, keep Private */ + if (IS_ATOMIC_WRITTEN_PAGE(page)) + return; + ClearPagePrivate(page); } @@ -2054,6 +1595,10 @@ int f2fs_release_page(struct page *page, gfp_t wait) if (PageDirty(page)) return 0; + /* This is atomic written page, keep Private */ + if (IS_ATOMIC_WRITTEN_PAGE(page)) + return 0; + ClearPagePrivate(page); return 1; } @@ -2068,8 +1613,15 @@ static int f2fs_set_data_page_dirty(struct page *page) SetPageUptodate(page); if (f2fs_is_atomic_file(inode)) { - register_inmem_page(inode, page); - return 1; + if (!IS_ATOMIC_WRITTEN_PAGE(page)) { + register_inmem_page(inode, page); + return 1; + } + /* + * Previously, this page has been registered, we just + * return here. + */ + return 0; } if (!PageDirty(page)) { @@ -2090,38 +1642,7 @@ static sector_t f2fs_bmap(struct address_space *mapping, sector_t block) if (err) return err; } - return generic_block_bmap(mapping, block, get_data_block); -} - -void init_extent_cache_info(struct f2fs_sb_info *sbi) -{ - INIT_RADIX_TREE(&sbi->extent_tree_root, GFP_NOIO); - init_rwsem(&sbi->extent_tree_lock); - INIT_LIST_HEAD(&sbi->extent_list); - spin_lock_init(&sbi->extent_lock); - sbi->total_ext_tree = 0; - atomic_set(&sbi->total_ext_node, 0); -} - -int __init create_extent_cache(void) -{ - extent_tree_slab = f2fs_kmem_cache_create("f2fs_extent_tree", - sizeof(struct extent_tree)); - if (!extent_tree_slab) - return -ENOMEM; - extent_node_slab = f2fs_kmem_cache_create("f2fs_extent_node", - sizeof(struct extent_node)); - if (!extent_node_slab) { - kmem_cache_destroy(extent_tree_slab); - return -ENOMEM; - } - return 0; -} - -void destroy_extent_cache(void) -{ - kmem_cache_destroy(extent_node_slab); - kmem_cache_destroy(extent_tree_slab); + return generic_block_bmap(mapping, block, get_data_block_bmap); } const struct address_space_operations f2fs_dblock_aops = { diff --git a/fs/f2fs/debug.c b/fs/f2fs/debug.c index 75176e0dd6c8b2..d013d847975379 100644 --- a/fs/f2fs/debug.c +++ b/fs/f2fs/debug.c @@ -33,8 +33,11 @@ static void update_general_status(struct f2fs_sb_info *sbi) int i; /* validation check of the segment numbers */ - si->hit_ext = sbi->read_hit_ext; - si->total_ext = sbi->total_hit_ext; + si->hit_largest = atomic_read(&sbi->read_hit_largest); + si->hit_cached = atomic_read(&sbi->read_hit_cached); + si->hit_rbtree = atomic_read(&sbi->read_hit_rbtree); + si->hit_total = si->hit_largest + si->hit_cached + si->hit_rbtree; + si->total_ext = atomic_read(&sbi->total_hit_ext); si->ext_tree = sbi->total_ext_tree; si->ext_node = atomic_read(&sbi->total_ext_node); si->ndirty_node = get_pages(sbi, F2FS_DIRTY_NODES); @@ -49,6 +52,7 @@ static void update_general_status(struct f2fs_sb_info *sbi) si->valid_count = valid_user_blocks(sbi); si->valid_node_count = valid_node_count(sbi); si->valid_inode_count = valid_inode_count(sbi); + si->inline_xattr = atomic_read(&sbi->inline_xattr); si->inline_inode = atomic_read(&sbi->inline_inode); si->inline_dir = atomic_read(&sbi->inline_dir); si->utilization = utilization(sbi); @@ -226,6 +230,8 @@ static int stat_show(struct seq_file *s, void *v) seq_printf(s, "Other: %u)\n - Data: %u\n", si->valid_node_count - si->valid_inode_count, si->valid_count - si->valid_node_count); + seq_printf(s, " - Inline_xattr Inode: %u\n", + si->inline_xattr); seq_printf(s, " - Inline_data Inode: %u\n", si->inline_inode); seq_printf(s, " - Inline_dentry Inode: %u\n", @@ -276,10 +282,16 @@ static int stat_show(struct seq_file *s, void *v) si->bg_data_blks); seq_printf(s, " - node blocks : %d (%d)\n", si->node_blks, si->bg_node_blks); - seq_printf(s, "\nExtent Hit Ratio: %d / %d\n", - si->hit_ext, si->total_ext); - seq_printf(s, "\nExtent Tree Count: %d\n", si->ext_tree); - seq_printf(s, "\nExtent Node Count: %d\n", si->ext_node); + seq_puts(s, "\nExtent Cache:\n"); + seq_printf(s, " - Hit Count: L1-1:%d L1-2:%d L2:%d\n", + si->hit_largest, si->hit_cached, + si->hit_rbtree); + seq_printf(s, " - Hit Ratio: %d%% (%d / %d)\n", + !si->total_ext ? 0 : + (si->hit_total * 100) / si->total_ext, + si->hit_total, si->total_ext); + seq_printf(s, " - Inner Struct Count: tree: %d, node: %d\n", + si->ext_tree, si->ext_node); seq_puts(s, "\nBalancing F2FS Async:\n"); seq_printf(s, " - inmem: %4d, wb: %4d\n", si->inmem_pages, si->wb_pages); @@ -366,6 +378,12 @@ int f2fs_build_stats(struct f2fs_sb_info *sbi) si->sbi = sbi; sbi->stat_info = si; + atomic_set(&sbi->total_hit_ext, 0); + atomic_set(&sbi->read_hit_rbtree, 0); + atomic_set(&sbi->read_hit_largest, 0); + atomic_set(&sbi->read_hit_cached, 0); + + atomic_set(&sbi->inline_xattr, 0); atomic_set(&sbi->inline_inode, 0); atomic_set(&sbi->inline_dir, 0); atomic_set(&sbi->inplace_count, 0); diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c index a34ebd8312ab89..8f15fc134040c5 100644 --- a/fs/f2fs/dir.c +++ b/fs/f2fs/dir.c @@ -718,8 +718,8 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page, if (inode) f2fs_drop_nlink(dir, inode, NULL); - if (bit_pos == NR_DENTRY_IN_BLOCK) { - truncate_hole(dir, page->index, page->index + 1); + if (bit_pos == NR_DENTRY_IN_BLOCK && + !truncate_hole(dir, page->index, page->index + 1)) { clear_page_dirty_for_io(page); ClearPagePrivate(page); ClearPageUptodate(page); diff --git a/fs/f2fs/extent_cache.c b/fs/f2fs/extent_cache.c new file mode 100644 index 00000000000000..997ac86f2a1d55 --- /dev/null +++ b/fs/f2fs/extent_cache.c @@ -0,0 +1,791 @@ +/* + * f2fs extent cache support + * + * Copyright (c) 2015 Motorola Mobility + * Copyright (c) 2015 Samsung Electronics + * Authors: Jaegeuk Kim + * Chao Yu + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#include +#include + +#include "f2fs.h" +#include "node.h" +#include + +static struct kmem_cache *extent_tree_slab; +static struct kmem_cache *extent_node_slab; + +static struct extent_node *__attach_extent_node(struct f2fs_sb_info *sbi, + struct extent_tree *et, struct extent_info *ei, + struct rb_node *parent, struct rb_node **p) +{ + struct extent_node *en; + + en = kmem_cache_alloc(extent_node_slab, GFP_ATOMIC); + if (!en) + return NULL; + + en->ei = *ei; + INIT_LIST_HEAD(&en->list); + + rb_link_node(&en->rb_node, parent, p); + rb_insert_color(&en->rb_node, &et->root); + et->count++; + atomic_inc(&sbi->total_ext_node); + return en; +} + +static void __detach_extent_node(struct f2fs_sb_info *sbi, + struct extent_tree *et, struct extent_node *en) +{ + rb_erase(&en->rb_node, &et->root); + et->count--; + atomic_dec(&sbi->total_ext_node); + + if (et->cached_en == en) + et->cached_en = NULL; +} + +static struct extent_tree *__grab_extent_tree(struct inode *inode) +{ + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + struct extent_tree *et; + nid_t ino = inode->i_ino; + + down_write(&sbi->extent_tree_lock); + et = radix_tree_lookup(&sbi->extent_tree_root, ino); + if (!et) { + et = f2fs_kmem_cache_alloc(extent_tree_slab, GFP_NOFS); + f2fs_radix_tree_insert(&sbi->extent_tree_root, ino, et); + memset(et, 0, sizeof(struct extent_tree)); + et->ino = ino; + et->root = RB_ROOT; + et->cached_en = NULL; + rwlock_init(&et->lock); + atomic_set(&et->refcount, 0); + et->count = 0; + sbi->total_ext_tree++; + } + atomic_inc(&et->refcount); + up_write(&sbi->extent_tree_lock); + + /* never died until evict_inode */ + F2FS_I(inode)->extent_tree = et; + + return et; +} + +static struct extent_node *__lookup_extent_tree(struct f2fs_sb_info *sbi, + struct extent_tree *et, unsigned int fofs) +{ + struct rb_node *node = et->root.rb_node; + struct extent_node *en = et->cached_en; + + if (en) { + struct extent_info *cei = &en->ei; + + if (cei->fofs <= fofs && cei->fofs + cei->len > fofs) { + stat_inc_cached_node_hit(sbi); + return en; + } + } + + while (node) { + en = rb_entry(node, struct extent_node, rb_node); + + if (fofs < en->ei.fofs) { + node = node->rb_left; + } else if (fofs >= en->ei.fofs + en->ei.len) { + node = node->rb_right; + } else { + stat_inc_rbtree_node_hit(sbi); + return en; + } + } + return NULL; +} + +static struct extent_node *__init_extent_tree(struct f2fs_sb_info *sbi, + struct extent_tree *et, struct extent_info *ei) +{ + struct rb_node **p = &et->root.rb_node; + struct extent_node *en; + + en = __attach_extent_node(sbi, et, ei, NULL, p); + if (!en) + return NULL; + + et->largest = en->ei; + et->cached_en = en; + return en; +} + +static unsigned int __free_extent_tree(struct f2fs_sb_info *sbi, + struct extent_tree *et, bool free_all) +{ + struct rb_node *node, *next; + struct extent_node *en; + unsigned int count = et->count; + + node = rb_first(&et->root); + while (node) { + next = rb_next(node); + en = rb_entry(node, struct extent_node, rb_node); + + if (free_all) { + spin_lock(&sbi->extent_lock); + if (!list_empty(&en->list)) + list_del_init(&en->list); + spin_unlock(&sbi->extent_lock); + } + + if (free_all || list_empty(&en->list)) { + __detach_extent_node(sbi, et, en); + kmem_cache_free(extent_node_slab, en); + } + node = next; + } + + return count - et->count; +} + +static void __drop_largest_extent(struct inode *inode, pgoff_t fofs) +{ + struct extent_info *largest = &F2FS_I(inode)->extent_tree->largest; + + if (largest->fofs <= fofs && largest->fofs + largest->len > fofs) + largest->len = 0; +} + +void f2fs_drop_largest_extent(struct inode *inode, pgoff_t fofs) +{ + if (!f2fs_may_extent_tree(inode)) + return; + + __drop_largest_extent(inode, fofs); +} + +void f2fs_init_extent_tree(struct inode *inode, struct f2fs_extent *i_ext) +{ + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + struct extent_tree *et; + struct extent_node *en; + struct extent_info ei; + + if (!f2fs_may_extent_tree(inode)) + return; + + et = __grab_extent_tree(inode); + + if (!i_ext || le32_to_cpu(i_ext->len) < F2FS_MIN_EXTENT_LEN) + return; + + set_extent_info(&ei, le32_to_cpu(i_ext->fofs), + le32_to_cpu(i_ext->blk), le32_to_cpu(i_ext->len)); + + write_lock(&et->lock); + if (et->count) + goto out; + + en = __init_extent_tree(sbi, et, &ei); + if (en) { + spin_lock(&sbi->extent_lock); + list_add_tail(&en->list, &sbi->extent_list); + spin_unlock(&sbi->extent_lock); + } +out: + write_unlock(&et->lock); +} + +static bool f2fs_lookup_extent_tree(struct inode *inode, pgoff_t pgofs, + struct extent_info *ei) +{ + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + struct extent_tree *et = F2FS_I(inode)->extent_tree; + struct extent_node *en; + bool ret = false; + + f2fs_bug_on(sbi, !et); + + trace_f2fs_lookup_extent_tree_start(inode, pgofs); + + read_lock(&et->lock); + + if (et->largest.fofs <= pgofs && + et->largest.fofs + et->largest.len > pgofs) { + *ei = et->largest; + ret = true; + stat_inc_largest_node_hit(sbi); + goto out; + } + + en = __lookup_extent_tree(sbi, et, pgofs); + if (en) { + *ei = en->ei; + spin_lock(&sbi->extent_lock); + if (!list_empty(&en->list)) + list_move_tail(&en->list, &sbi->extent_list); + et->cached_en = en; + spin_unlock(&sbi->extent_lock); + ret = true; + } +out: + stat_inc_total_hit(sbi); + read_unlock(&et->lock); + + trace_f2fs_lookup_extent_tree_end(inode, pgofs, ei); + return ret; +} + + +/* + * lookup extent at @fofs, if hit, return the extent + * if not, return NULL and + * @prev_ex: extent before fofs + * @next_ex: extent after fofs + * @insert_p: insert point for new extent at fofs + * in order to simpfy the insertion after. + * tree must stay unchanged between lookup and insertion. + */ +static struct extent_node *__lookup_extent_tree_ret(struct extent_tree *et, + unsigned int fofs, + struct extent_node **prev_ex, + struct extent_node **next_ex, + struct rb_node ***insert_p, + struct rb_node **insert_parent) +{ + struct rb_node **pnode = &et->root.rb_node; + struct rb_node *parent = NULL, *tmp_node; + struct extent_node *en = et->cached_en; + + *insert_p = NULL; + *insert_parent = NULL; + *prev_ex = NULL; + *next_ex = NULL; + + if (RB_EMPTY_ROOT(&et->root)) + return NULL; + + if (en) { + struct extent_info *cei = &en->ei; + + if (cei->fofs <= fofs && cei->fofs + cei->len > fofs) + goto lookup_neighbors; + } + + while (*pnode) { + parent = *pnode; + en = rb_entry(*pnode, struct extent_node, rb_node); + + if (fofs < en->ei.fofs) + pnode = &(*pnode)->rb_left; + else if (fofs >= en->ei.fofs + en->ei.len) + pnode = &(*pnode)->rb_right; + else + goto lookup_neighbors; + } + + *insert_p = pnode; + *insert_parent = parent; + + en = rb_entry(parent, struct extent_node, rb_node); + tmp_node = parent; + if (parent && fofs > en->ei.fofs) + tmp_node = rb_next(parent); + *next_ex = tmp_node ? + rb_entry(tmp_node, struct extent_node, rb_node) : NULL; + + tmp_node = parent; + if (parent && fofs < en->ei.fofs) + tmp_node = rb_prev(parent); + *prev_ex = tmp_node ? + rb_entry(tmp_node, struct extent_node, rb_node) : NULL; + return NULL; + +lookup_neighbors: + if (fofs == en->ei.fofs) { + /* lookup prev node for merging backward later */ + tmp_node = rb_prev(&en->rb_node); + *prev_ex = tmp_node ? + rb_entry(tmp_node, struct extent_node, rb_node) : NULL; + } + if (fofs == en->ei.fofs + en->ei.len - 1) { + /* lookup next node for merging frontward later */ + tmp_node = rb_next(&en->rb_node); + *next_ex = tmp_node ? + rb_entry(tmp_node, struct extent_node, rb_node) : NULL; + } + return en; +} + +static struct extent_node *__try_merge_extent_node(struct f2fs_sb_info *sbi, + struct extent_tree *et, struct extent_info *ei, + struct extent_node **den, + struct extent_node *prev_ex, + struct extent_node *next_ex) +{ + struct extent_node *en = NULL; + + if (prev_ex && __is_back_mergeable(ei, &prev_ex->ei)) { + prev_ex->ei.len += ei->len; + ei = &prev_ex->ei; + en = prev_ex; + } + + if (next_ex && __is_front_mergeable(ei, &next_ex->ei)) { + if (en) { + __detach_extent_node(sbi, et, prev_ex); + *den = prev_ex; + } + next_ex->ei.fofs = ei->fofs; + next_ex->ei.blk = ei->blk; + next_ex->ei.len += ei->len; + en = next_ex; + } + + if (en) { + if (en->ei.len > et->largest.len) + et->largest = en->ei; + et->cached_en = en; + } + return en; +} + +static struct extent_node *__insert_extent_tree(struct f2fs_sb_info *sbi, + struct extent_tree *et, struct extent_info *ei, + struct rb_node **insert_p, + struct rb_node *insert_parent) +{ + struct rb_node **p = &et->root.rb_node; + struct rb_node *parent = NULL; + struct extent_node *en = NULL; + + if (insert_p && insert_parent) { + parent = insert_parent; + p = insert_p; + goto do_insert; + } + + while (*p) { + parent = *p; + en = rb_entry(parent, struct extent_node, rb_node); + + if (ei->fofs < en->ei.fofs) + p = &(*p)->rb_left; + else if (ei->fofs >= en->ei.fofs + en->ei.len) + p = &(*p)->rb_right; + else + f2fs_bug_on(sbi, 1); + } +do_insert: + en = __attach_extent_node(sbi, et, ei, parent, p); + if (!en) + return NULL; + + if (en->ei.len > et->largest.len) + et->largest = en->ei; + et->cached_en = en; + return en; +} + +unsigned int f2fs_update_extent_tree_range(struct inode *inode, + pgoff_t fofs, block_t blkaddr, unsigned int len) +{ + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + struct extent_tree *et = F2FS_I(inode)->extent_tree; + struct extent_node *en = NULL, *en1 = NULL, *en2 = NULL, *en3 = NULL; + struct extent_node *prev_en = NULL, *next_en = NULL; + struct extent_info ei, dei, prev; + struct rb_node **insert_p = NULL, *insert_parent = NULL; + unsigned int end = fofs + len; + unsigned int pos = (unsigned int)fofs; + + if (!et) + return false; + + write_lock(&et->lock); + + if (is_inode_flag_set(F2FS_I(inode), FI_NO_EXTENT)) { + write_unlock(&et->lock); + return false; + } + + prev = et->largest; + dei.len = 0; + + /* we do not guarantee that the largest extent is cached all the time */ + __drop_largest_extent(inode, fofs); + + /* 1. lookup first extent node in range [fofs, fofs + len - 1] */ + en = __lookup_extent_tree_ret(et, fofs, &prev_en, &next_en, + &insert_p, &insert_parent); + if (!en) { + if (next_en) { + en = next_en; + f2fs_bug_on(sbi, en->ei.fofs <= pos); + pos = en->ei.fofs; + } else { + /* + * skip searching in the tree since there is no + * larger extent node in the cache. + */ + goto update_extent; + } + } + + /* 2. invlidate all extent nodes in range [fofs, fofs + len - 1] */ + while (en) { + struct rb_node *node; + + if (pos >= end) + break; + + dei = en->ei; + en1 = en2 = NULL; + + node = rb_next(&en->rb_node); + + /* + * 2.1 there are four cases when we invalidate blkaddr in extent + * node, |V: valid address, X: will be invalidated| + */ + /* case#1, invalidate right part of extent node |VVVVVXXXXX| */ + if (pos > dei.fofs && end >= dei.fofs + dei.len) { + en->ei.len = pos - dei.fofs; + + if (en->ei.len < F2FS_MIN_EXTENT_LEN) { + __detach_extent_node(sbi, et, en); + insert_p = NULL; + insert_parent = NULL; + goto update; + } + + if (__is_extent_same(&dei, &et->largest)) + et->largest = en->ei; + goto next; + } + + /* case#2, invalidate left part of extent node |XXXXXVVVVV| */ + if (pos <= dei.fofs && end < dei.fofs + dei.len) { + en->ei.fofs = end; + en->ei.blk += end - dei.fofs; + en->ei.len -= end - dei.fofs; + + if (en->ei.len < F2FS_MIN_EXTENT_LEN) { + __detach_extent_node(sbi, et, en); + insert_p = NULL; + insert_parent = NULL; + goto update; + } + + if (__is_extent_same(&dei, &et->largest)) + et->largest = en->ei; + goto next; + } + + __detach_extent_node(sbi, et, en); + + /* + * if we remove node in rb-tree, our parent node pointer may + * point the wrong place, discard them. + */ + insert_p = NULL; + insert_parent = NULL; + + /* case#3, invalidate entire extent node |XXXXXXXXXX| */ + if (pos <= dei.fofs && end >= dei.fofs + dei.len) { + if (__is_extent_same(&dei, &et->largest)) + et->largest.len = 0; + goto update; + } + + /* + * case#4, invalidate data in the middle of extent node + * |VVVXXXXVVV| + */ + if (dei.len > F2FS_MIN_EXTENT_LEN) { + unsigned int endofs; + + /* insert left part of split extent into cache */ + if (pos - dei.fofs >= F2FS_MIN_EXTENT_LEN) { + set_extent_info(&ei, dei.fofs, dei.blk, + pos - dei.fofs); + en1 = __insert_extent_tree(sbi, et, &ei, + NULL, NULL); + } + + /* insert right part of split extent into cache */ + endofs = dei.fofs + dei.len; + if (endofs - end >= F2FS_MIN_EXTENT_LEN) { + set_extent_info(&ei, end, + end - dei.fofs + dei.blk, + endofs - end); + en2 = __insert_extent_tree(sbi, et, &ei, + NULL, NULL); + } + } +update: + /* 2.2 update in global extent list */ + spin_lock(&sbi->extent_lock); + if (en && !list_empty(&en->list)) + list_del(&en->list); + if (en1) + list_add_tail(&en1->list, &sbi->extent_list); + if (en2) + list_add_tail(&en2->list, &sbi->extent_list); + spin_unlock(&sbi->extent_lock); + + /* 2.3 release extent node */ + if (en) + kmem_cache_free(extent_node_slab, en); +next: + en = node ? rb_entry(node, struct extent_node, rb_node) : NULL; + next_en = en; + if (en) + pos = en->ei.fofs; + } + +update_extent: + /* 3. update extent in extent cache */ + if (blkaddr) { + struct extent_node *den = NULL; + + set_extent_info(&ei, fofs, blkaddr, len); + en3 = __try_merge_extent_node(sbi, et, &ei, &den, + prev_en, next_en); + if (!en3) + en3 = __insert_extent_tree(sbi, et, &ei, + insert_p, insert_parent); + + /* give up extent_cache, if split and small updates happen */ + if (dei.len >= 1 && + prev.len < F2FS_MIN_EXTENT_LEN && + et->largest.len < F2FS_MIN_EXTENT_LEN) { + et->largest.len = 0; + set_inode_flag(F2FS_I(inode), FI_NO_EXTENT); + } + + spin_lock(&sbi->extent_lock); + if (en3) { + if (list_empty(&en3->list)) + list_add_tail(&en3->list, &sbi->extent_list); + else + list_move_tail(&en3->list, &sbi->extent_list); + } + if (den && !list_empty(&den->list)) + list_del(&den->list); + spin_unlock(&sbi->extent_lock); + + if (den) + kmem_cache_free(extent_node_slab, den); + } + + if (is_inode_flag_set(F2FS_I(inode), FI_NO_EXTENT)) + __free_extent_tree(sbi, et, true); + + write_unlock(&et->lock); + + return !__is_extent_same(&prev, &et->largest); +} + +unsigned int f2fs_shrink_extent_tree(struct f2fs_sb_info *sbi, int nr_shrink) +{ + struct extent_tree *treevec[EXT_TREE_VEC_SIZE]; + struct extent_node *en, *tmp; + unsigned long ino = F2FS_ROOT_INO(sbi); + struct radix_tree_root *root = &sbi->extent_tree_root; + unsigned int found; + unsigned int node_cnt = 0, tree_cnt = 0; + int remained; + + if (!test_opt(sbi, EXTENT_CACHE)) + return 0; + + if (!down_write_trylock(&sbi->extent_tree_lock)) + goto out; + + /* 1. remove unreferenced extent tree */ + while ((found = radix_tree_gang_lookup(root, + (void **)treevec, ino, EXT_TREE_VEC_SIZE))) { + unsigned i; + + ino = treevec[found - 1]->ino + 1; + for (i = 0; i < found; i++) { + struct extent_tree *et = treevec[i]; + + if (!atomic_read(&et->refcount)) { + write_lock(&et->lock); + node_cnt += __free_extent_tree(sbi, et, true); + write_unlock(&et->lock); + + radix_tree_delete(root, et->ino); + kmem_cache_free(extent_tree_slab, et); + sbi->total_ext_tree--; + tree_cnt++; + + if (node_cnt + tree_cnt >= nr_shrink) + goto unlock_out; + } + } + } + up_write(&sbi->extent_tree_lock); + + /* 2. remove LRU extent entries */ + if (!down_write_trylock(&sbi->extent_tree_lock)) + goto out; + + remained = nr_shrink - (node_cnt + tree_cnt); + + spin_lock(&sbi->extent_lock); + list_for_each_entry_safe(en, tmp, &sbi->extent_list, list) { + if (!remained--) + break; + list_del_init(&en->list); + } + spin_unlock(&sbi->extent_lock); + + while ((found = radix_tree_gang_lookup(root, + (void **)treevec, ino, EXT_TREE_VEC_SIZE))) { + unsigned i; + + ino = treevec[found - 1]->ino + 1; + for (i = 0; i < found; i++) { + struct extent_tree *et = treevec[i]; + + write_lock(&et->lock); + node_cnt += __free_extent_tree(sbi, et, false); + write_unlock(&et->lock); + + if (node_cnt + tree_cnt >= nr_shrink) + break; + } + } +unlock_out: + up_write(&sbi->extent_tree_lock); +out: + trace_f2fs_shrink_extent_tree(sbi, node_cnt, tree_cnt); + + return node_cnt + tree_cnt; +} + +unsigned int f2fs_destroy_extent_node(struct inode *inode) +{ + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + struct extent_tree *et = F2FS_I(inode)->extent_tree; + unsigned int node_cnt = 0; + + if (!et) + return 0; + + write_lock(&et->lock); + node_cnt = __free_extent_tree(sbi, et, true); + write_unlock(&et->lock); + + return node_cnt; +} + +void f2fs_destroy_extent_tree(struct inode *inode) +{ + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + struct extent_tree *et = F2FS_I(inode)->extent_tree; + unsigned int node_cnt = 0; + + if (!et) + return; + + if (inode->i_nlink && !is_bad_inode(inode) && et->count) { + atomic_dec(&et->refcount); + return; + } + + /* free all extent info belong to this extent tree */ + node_cnt = f2fs_destroy_extent_node(inode); + + /* delete extent tree entry in radix tree */ + down_write(&sbi->extent_tree_lock); + atomic_dec(&et->refcount); + f2fs_bug_on(sbi, atomic_read(&et->refcount) || et->count); + radix_tree_delete(&sbi->extent_tree_root, inode->i_ino); + kmem_cache_free(extent_tree_slab, et); + sbi->total_ext_tree--; + up_write(&sbi->extent_tree_lock); + + F2FS_I(inode)->extent_tree = NULL; + + trace_f2fs_destroy_extent_tree(inode, node_cnt); +} + +bool f2fs_lookup_extent_cache(struct inode *inode, pgoff_t pgofs, + struct extent_info *ei) +{ + if (!f2fs_may_extent_tree(inode)) + return false; + + return f2fs_lookup_extent_tree(inode, pgofs, ei); +} + +void f2fs_update_extent_cache(struct dnode_of_data *dn) +{ + struct f2fs_inode_info *fi = F2FS_I(dn->inode); + pgoff_t fofs; + + if (!f2fs_may_extent_tree(dn->inode)) + return; + + f2fs_bug_on(F2FS_I_SB(dn->inode), dn->data_blkaddr == NEW_ADDR); + + + fofs = start_bidx_of_node(ofs_of_node(dn->node_page), fi) + + dn->ofs_in_node; + + if (f2fs_update_extent_tree_range(dn->inode, fofs, dn->data_blkaddr, 1)) + sync_inode_page(dn); +} + +void f2fs_update_extent_cache_range(struct dnode_of_data *dn, + pgoff_t fofs, block_t blkaddr, unsigned int len) + +{ + if (!f2fs_may_extent_tree(dn->inode)) + return; + + if (f2fs_update_extent_tree_range(dn->inode, fofs, blkaddr, len)) + sync_inode_page(dn); +} + +void init_extent_cache_info(struct f2fs_sb_info *sbi) +{ + INIT_RADIX_TREE(&sbi->extent_tree_root, GFP_NOIO); + init_rwsem(&sbi->extent_tree_lock); + INIT_LIST_HEAD(&sbi->extent_list); + spin_lock_init(&sbi->extent_lock); + sbi->total_ext_tree = 0; + atomic_set(&sbi->total_ext_node, 0); +} + +int __init create_extent_cache(void) +{ + extent_tree_slab = f2fs_kmem_cache_create("f2fs_extent_tree", + sizeof(struct extent_tree)); + if (!extent_tree_slab) + return -ENOMEM; + extent_node_slab = f2fs_kmem_cache_create("f2fs_extent_node", + sizeof(struct extent_node)); + if (!extent_node_slab) { + kmem_cache_destroy(extent_tree_slab); + return -ENOMEM; + } + return 0; +} + +void destroy_extent_cache(void) +{ + kmem_cache_destroy(extent_node_slab); + kmem_cache_destroy(extent_tree_slab); +} diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index a8327ed7389873..f1a90ffd7cad1e 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -19,6 +19,7 @@ #include #include #include +#include #ifdef CONFIG_F2FS_CHECK_FS #define f2fs_bug_on(sbi, condition) BUG_ON(condition) @@ -228,6 +229,7 @@ static inline bool __has_cursum_space(struct f2fs_summary_block *sum, int size, #define F2FS_IOC_START_VOLATILE_WRITE _IO(F2FS_IOCTL_MAGIC, 3) #define F2FS_IOC_RELEASE_VOLATILE_WRITE _IO(F2FS_IOCTL_MAGIC, 4) #define F2FS_IOC_ABORT_VOLATILE_WRITE _IO(F2FS_IOCTL_MAGIC, 5) +#define F2FS_IOC_GARBAGE_COLLECT _IO(F2FS_IOCTL_MAGIC, 6) #define F2FS_IOC_SET_ENCRYPTION_POLICY \ _IOR('f', 19, struct f2fs_encryption_policy) @@ -320,7 +322,7 @@ enum { */ }; -#define F2FS_LINK_MAX 32000 /* maximum link count per file */ +#define F2FS_LINK_MAX 0xffffffff /* maximum link count per file */ #define MAX_DIR_RA_PAGES 4 /* maximum ra pages of dir */ @@ -349,6 +351,7 @@ struct extent_tree { nid_t ino; /* inode number */ struct rb_root root; /* root of extent info rb-tree */ struct extent_node *cached_en; /* recently accessed extent node */ + struct extent_info largest; /* largested extent info */ rwlock_t lock; /* protect extent info rb-tree */ atomic_t refcount; /* reference count of rb-tree */ unsigned int count; /* # of extent node in rb-tree*/ @@ -372,6 +375,12 @@ struct f2fs_map_blocks { unsigned int m_flags; }; +/* for flag in get_data_block */ +#define F2FS_GET_BLOCK_READ 0 +#define F2FS_GET_BLOCK_DIO 1 +#define F2FS_GET_BLOCK_FIEMAP 2 +#define F2FS_GET_BLOCK_BMAP 3 + /* * i_advise uses FADVISE_XXX_BIT. We can add additional hints later. */ @@ -420,14 +429,13 @@ struct f2fs_inode_info { unsigned int clevel; /* maximum level of given file name */ nid_t i_xattr_nid; /* node id that contains xattrs */ unsigned long long xattr_ver; /* cp version of xattr modification */ - struct extent_info ext; /* in-memory extent cache entry */ - rwlock_t ext_lock; /* rwlock for single extent cache */ struct inode_entry *dirty_dir; /* the pointer of dirty dir */ - struct radix_tree_root inmem_root; /* radix tree for inmem pages */ struct list_head inmem_pages; /* inmemory pages managed by f2fs */ struct mutex inmem_lock; /* lock for inmemory pages */ + struct extent_tree *extent_tree; /* cached extent_tree entry */ + #ifdef CONFIG_F2FS_FS_ENCRYPTION /* Encryption params */ struct f2fs_crypt_info *i_crypt_info; @@ -779,7 +787,11 @@ struct f2fs_sb_info { unsigned int segment_count[2]; /* # of allocated segments */ unsigned int block_count[2]; /* # of allocated blocks */ atomic_t inplace_count; /* # of inplace update */ - int total_hit_ext, read_hit_ext; /* extent cache hit ratio */ + atomic_t total_hit_ext; /* # of lookup extent cache */ + atomic_t read_hit_rbtree; /* # of hit rbtree extent node */ + atomic_t read_hit_largest; /* # of hit largest extent node */ + atomic_t read_hit_cached; /* # of hit cached extent node */ + atomic_t inline_xattr; /* # of inline_xattr inodes */ atomic_t inline_inode; /* # of inline_data inodes */ atomic_t inline_dir; /* # of inline_dentry inodes */ int bg_gc; /* background gc calls */ @@ -791,6 +803,11 @@ struct f2fs_sb_info { /* For sysfs suppport */ struct kobject s_kobj; struct completion s_kobj_unregister; + + /* For shrinker support */ + struct list_head s_list; + struct mutex umount_mutex; + unsigned int shrinker_run_no; }; /* @@ -1039,7 +1056,8 @@ static inline void dec_page_count(struct f2fs_sb_info *sbi, int count_type) static inline void inode_dec_dirty_pages(struct inode *inode) { - if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode)) + if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) && + !S_ISLNK(inode->i_mode)) return; atomic_dec(&F2FS_I(inode)->dirty_pages); @@ -1234,16 +1252,24 @@ static inline void *f2fs_kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { void *entry; -retry: - entry = kmem_cache_alloc(cachep, flags); - if (!entry) { - cond_resched(); - goto retry; - } + entry = kmem_cache_alloc(cachep, flags); + if (!entry) + entry = kmem_cache_alloc(cachep, flags | __GFP_NOFAIL); return entry; } +static inline struct bio *f2fs_bio_alloc(int npages) +{ + struct bio *bio; + + /* No failure on bio allocation */ + bio = bio_alloc(GFP_NOIO, npages); + if (!bio) + bio = bio_alloc(GFP_NOIO | __GFP_NOFAIL, npages); + return bio; +} + static inline void f2fs_radix_tree_insert(struct radix_tree_root *root, unsigned long index, void *item) { @@ -1342,6 +1368,7 @@ enum { FI_INC_LINK, /* need to increment i_nlink */ FI_ACL_MODE, /* indicate acl mode */ FI_NO_ALLOC, /* should not allocate any blocks */ + FI_FREE_NID, /* free allocated nide */ FI_UPDATE_DIR, /* should update inode block for consistency */ FI_DELAY_IPUT, /* used for the recovery */ FI_NO_EXTENT, /* not to use the extent cache */ @@ -1541,6 +1568,17 @@ static inline bool is_dot_dotdot(const struct qstr *str) return false; } +static inline bool f2fs_may_extent_tree(struct inode *inode) +{ + mode_t mode = inode->i_mode; + + if (!test_opt(F2FS_I_SB(inode), EXTENT_CACHE) || + is_inode_flag_set(F2FS_I(inode), FI_NO_EXTENT)) + return false; + + return S_ISREG(mode); +} + #define get_inode_mode(i) \ ((is_inode_flag_set(F2FS_I(i), FI_ACL_MODE)) ? \ (F2FS_I(i)->i_acl_mode) : ((i)->i_mode)) @@ -1557,7 +1595,7 @@ static inline bool is_dot_dotdot(const struct qstr *str) int f2fs_sync_file(struct file *, loff_t, loff_t, int); void truncate_data_blocks(struct dnode_of_data *); int truncate_blocks(struct inode *, u64, bool); -void f2fs_truncate(struct inode *); +int f2fs_truncate(struct inode *, bool); int f2fs_getattr(struct vfsmount *, struct dentry *, struct kstat *); int f2fs_setattr(struct dentry *, struct iattr *); int truncate_hole(struct inode *, pgoff_t, pgoff_t); @@ -1649,7 +1687,7 @@ int get_dnode_of_data(struct dnode_of_data *, pgoff_t, int); int truncate_inode_blocks(struct inode *, pgoff_t); int truncate_xattr_node(struct inode *, struct page *); int wait_on_node_pages_writeback(struct f2fs_sb_info *, nid_t); -void remove_inode_page(struct inode *); +int remove_inode_page(struct inode *); struct page *new_inode_page(struct inode *); struct page *new_node_page(struct dnode_of_data *, unsigned int, struct page *); void ra_node_page(struct f2fs_sb_info *, nid_t); @@ -1660,6 +1698,7 @@ int sync_node_pages(struct f2fs_sb_info *, nid_t, struct writeback_control *); bool alloc_nid(struct f2fs_sb_info *, nid_t *); void alloc_nid_done(struct f2fs_sb_info *, nid_t); void alloc_nid_failed(struct f2fs_sb_info *, nid_t); +int try_to_free_nids(struct f2fs_sb_info *, int); void recover_inline_xattr(struct inode *, struct page *); void recover_xattr_data(struct inode *, struct page *, block_t); int recover_inode_page(struct f2fs_sb_info *, struct page *); @@ -1675,7 +1714,7 @@ void destroy_node_manager_caches(void); * segment.c */ void register_inmem_page(struct inode *, struct page *); -void commit_inmem_pages(struct inode *, bool); +int commit_inmem_pages(struct inode *, bool); void f2fs_balance_fs(struct f2fs_sb_info *); void f2fs_balance_fs_bg(struct f2fs_sb_info *); int f2fs_issue_flush(struct f2fs_sb_info *); @@ -1685,7 +1724,7 @@ void invalidate_blocks(struct f2fs_sb_info *, block_t); void refresh_sit_entry(struct f2fs_sb_info *, block_t, block_t); void clear_prefree_segments(struct f2fs_sb_info *, struct cp_control *); void release_discard_addrs(struct f2fs_sb_info *); -void discard_next_dnode(struct f2fs_sb_info *, block_t); +bool discard_next_dnode(struct f2fs_sb_info *, block_t); int npages_for_summary_flush(struct f2fs_sb_info *, bool); void allocate_new_segments(struct f2fs_sb_info *); int f2fs_trim_fs(struct f2fs_sb_info *, struct fstrim_range *); @@ -1727,7 +1766,7 @@ int acquire_orphan_inode(struct f2fs_sb_info *); void release_orphan_inode(struct f2fs_sb_info *); void add_orphan_inode(struct f2fs_sb_info *, nid_t); void remove_orphan_inode(struct f2fs_sb_info *, nid_t); -void recover_orphan_inodes(struct f2fs_sb_info *); +int recover_orphan_inodes(struct f2fs_sb_info *); int get_valid_checkpoint(struct f2fs_sb_info *); void update_dirty_page(struct inode *, struct page *); void add_dirty_dir_inode(struct inode *); @@ -1746,21 +1785,14 @@ int f2fs_submit_page_bio(struct f2fs_io_info *); void f2fs_submit_page_mbio(struct f2fs_io_info *); void set_data_blkaddr(struct dnode_of_data *); int reserve_new_block(struct dnode_of_data *); +int f2fs_get_block(struct dnode_of_data *, pgoff_t); int f2fs_reserve_block(struct dnode_of_data *, pgoff_t); -void f2fs_shrink_extent_tree(struct f2fs_sb_info *, int); -void f2fs_destroy_extent_tree(struct inode *); -void f2fs_init_extent_cache(struct inode *, struct f2fs_extent *); -void f2fs_update_extent_cache(struct dnode_of_data *); -void f2fs_preserve_extent_tree(struct inode *); struct page *get_read_data_page(struct inode *, pgoff_t, int); struct page *find_data_page(struct inode *, pgoff_t); struct page *get_lock_data_page(struct inode *, pgoff_t); struct page *get_new_data_page(struct inode *, struct page *, pgoff_t, bool); int do_write_data_page(struct f2fs_io_info *); int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *, u64, u64); -void init_extent_cache_info(struct f2fs_sb_info *); -int __init create_extent_cache(void); -void destroy_extent_cache(void); void f2fs_invalidate_page(struct page *, unsigned int, unsigned int); int f2fs_release_page(struct page *, gfp_t); @@ -1788,11 +1820,13 @@ struct f2fs_stat_info { struct f2fs_sb_info *sbi; int all_area_segs, sit_area_segs, nat_area_segs, ssa_area_segs; int main_area_segs, main_area_sections, main_area_zones; - int hit_ext, total_ext, ext_tree, ext_node; + int hit_largest, hit_cached, hit_rbtree, hit_total, total_ext; + int ext_tree, ext_node; int ndirty_node, ndirty_dent, ndirty_dirs, ndirty_meta; int nats, dirty_nats, sits, dirty_sits, fnids; int total_count, utilization; - int bg_gc, inline_inode, inline_dir, inmem_pages, wb_pages; + int bg_gc, inmem_pages, wb_pages; + int inline_xattr, inline_inode, inline_dir; unsigned int valid_count, valid_node_count, valid_inode_count; unsigned int bimodal, avg_vblocks; int util_free, util_valid, util_invalid; @@ -1823,8 +1857,20 @@ static inline struct f2fs_stat_info *F2FS_STAT(struct f2fs_sb_info *sbi) #define stat_inc_bggc_count(sbi) ((sbi)->bg_gc++) #define stat_inc_dirty_dir(sbi) ((sbi)->n_dirty_dirs++) #define stat_dec_dirty_dir(sbi) ((sbi)->n_dirty_dirs--) -#define stat_inc_total_hit(sb) ((F2FS_SB(sb))->total_hit_ext++) -#define stat_inc_read_hit(sb) ((F2FS_SB(sb))->read_hit_ext++) +#define stat_inc_total_hit(sbi) (atomic_inc(&(sbi)->total_hit_ext)) +#define stat_inc_rbtree_node_hit(sbi) (atomic_inc(&(sbi)->read_hit_rbtree)) +#define stat_inc_largest_node_hit(sbi) (atomic_inc(&(sbi)->read_hit_largest)) +#define stat_inc_cached_node_hit(sbi) (atomic_inc(&(sbi)->read_hit_cached)) +#define stat_inc_inline_xattr(inode) \ + do { \ + if (f2fs_has_inline_xattr(inode)) \ + (atomic_inc(&F2FS_I_SB(inode)->inline_xattr)); \ + } while (0) +#define stat_dec_inline_xattr(inode) \ + do { \ + if (f2fs_has_inline_xattr(inode)) \ + (atomic_dec(&F2FS_I_SB(inode)->inline_xattr)); \ + } while (0) #define stat_inc_inline_inode(inode) \ do { \ if (f2fs_has_inline_data(inode)) \ @@ -1894,7 +1940,11 @@ void f2fs_destroy_root_stats(void); #define stat_inc_dirty_dir(sbi) #define stat_dec_dirty_dir(sbi) #define stat_inc_total_hit(sb) -#define stat_inc_read_hit(sb) +#define stat_inc_rbtree_node_hit(sb) +#define stat_inc_largest_node_hit(sbi) +#define stat_inc_cached_node_hit(sbi) +#define stat_inc_inline_xattr(inode) +#define stat_dec_inline_xattr(inode) #define stat_inc_inline_inode(inode) #define stat_dec_inline_inode(inode) #define stat_inc_inline_dir(inode) @@ -1949,6 +1999,30 @@ bool f2fs_empty_inline_dir(struct inode *); int f2fs_read_inline_dir(struct file *, struct dir_context *, struct f2fs_str *); +/* + * shrinker.c + */ +unsigned long f2fs_shrink_count(struct shrinker *, struct shrink_control *); +unsigned long f2fs_shrink_scan(struct shrinker *, struct shrink_control *); +void f2fs_join_shrinker(struct f2fs_sb_info *); +void f2fs_leave_shrinker(struct f2fs_sb_info *); + +/* + * extent_cache.c + */ +unsigned int f2fs_shrink_extent_tree(struct f2fs_sb_info *, int); +void f2fs_drop_largest_extent(struct inode *, pgoff_t); +void f2fs_init_extent_tree(struct inode *, struct f2fs_extent *); +unsigned int f2fs_destroy_extent_node(struct inode *); +void f2fs_destroy_extent_tree(struct inode *); +bool f2fs_lookup_extent_cache(struct inode *, pgoff_t, struct extent_info *); +void f2fs_update_extent_cache(struct dnode_of_data *); +void f2fs_update_extent_cache_range(struct dnode_of_data *dn, + pgoff_t, block_t, unsigned int); +void init_extent_cache_info(struct f2fs_sb_info *); +int __init create_extent_cache(void); +void destroy_extent_cache(void); + /* * crypto support */ diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c index b0f38c3b37f4d5..8120f868514196 100644 --- a/fs/f2fs/file.c +++ b/fs/f2fs/file.c @@ -27,6 +27,7 @@ #include "segment.h" #include "xattr.h" #include "acl.h" +#include "gc.h" #include "trace.h" #include @@ -85,6 +86,8 @@ static int f2fs_vm_page_mkwrite(struct vm_area_struct *vma, mapped: /* fill the page */ f2fs_wait_on_page_writeback(page, DATA); + /* if gced page is attached, don't write to cold segment */ + clear_cold_data(page); out: sb_end_pagefault(inode->i_sb); return block_page_mkwrite_return(err); @@ -203,8 +206,8 @@ int f2fs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) } /* if the inode is dirty, let's recover all the time */ - if (!datasync && is_inode_flag_set(fi, FI_DIRTY_INODE)) { - update_inode_page(inode); + if (!datasync) { + f2fs_write_inode(inode, NULL); goto go_write; } @@ -442,9 +445,9 @@ static int f2fs_file_open(struct inode *inode, struct file *filp) int truncate_data_blocks_range(struct dnode_of_data *dn, int count) { - int nr_free = 0, ofs = dn->ofs_in_node; struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode); struct f2fs_node *raw_node; + int nr_free = 0, ofs = dn->ofs_in_node, len = count; __le32 *addr; raw_node = F2FS_NODE(dn->node_page); @@ -457,14 +460,22 @@ int truncate_data_blocks_range(struct dnode_of_data *dn, int count) dn->data_blkaddr = NULL_ADDR; set_data_blkaddr(dn); - f2fs_update_extent_cache(dn); invalidate_blocks(sbi, blkaddr); if (dn->ofs_in_node == 0 && IS_INODE(dn->node_page)) clear_inode_flag(F2FS_I(dn->inode), FI_FIRST_BLOCK_WRITTEN); nr_free++; } + if (nr_free) { + pgoff_t fofs; + /* + * once we invalidate valid blkaddr in range [ofs, ofs + count], + * we will invalidate all blkaddr in the whole range. + */ + fofs = start_bidx_of_node(ofs_of_node(dn->node_page), + F2FS_I(dn->inode)) + ofs; + f2fs_update_extent_cache_range(dn, fofs, 0, len); dec_valid_block_count(sbi, dn->inode, nr_free); set_page_dirty(dn->node_page); sync_inode_page(dn); @@ -576,24 +587,30 @@ int truncate_blocks(struct inode *inode, u64 from, bool lock) return err; } -void f2fs_truncate(struct inode *inode) +int f2fs_truncate(struct inode *inode, bool lock) { + int err; + if (!(S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) || S_ISLNK(inode->i_mode))) - return; + return 0; trace_f2fs_truncate(inode); /* we should check inline_data size */ if (f2fs_has_inline_data(inode) && !f2fs_may_inline_data(inode)) { - if (f2fs_convert_inline_inode(inode)) - return; + err = f2fs_convert_inline_inode(inode); + if (err) + return err; } - if (!truncate_blocks(inode, i_size_read(inode), true)) { - inode->i_mtime = inode->i_ctime = CURRENT_TIME; - mark_inode_dirty(inode); - } + err = truncate_blocks(inode, i_size_read(inode), lock); + if (err) + return err; + + inode->i_mtime = inode->i_ctime = CURRENT_TIME; + mark_inode_dirty(inode); + return 0; } int f2fs_getattr(struct vfsmount *mnt, @@ -653,7 +670,9 @@ int f2fs_setattr(struct dentry *dentry, struct iattr *attr) if (attr->ia_size <= i_size_read(inode)) { truncate_setsize(inode, attr->ia_size); - f2fs_truncate(inode); + err = f2fs_truncate(inode, true); + if (err) + return err; f2fs_balance_fs(F2FS_I_SB(inode)); } else { /* @@ -692,14 +711,14 @@ const struct inode_operations f2fs_file_inode_operations = { .fiemap = f2fs_fiemap, }; -static void fill_zero(struct inode *inode, pgoff_t index, +static int fill_zero(struct inode *inode, pgoff_t index, loff_t start, loff_t len) { struct f2fs_sb_info *sbi = F2FS_I_SB(inode); struct page *page; if (!len) - return; + return 0; f2fs_balance_fs(sbi); @@ -707,12 +726,14 @@ static void fill_zero(struct inode *inode, pgoff_t index, page = get_new_data_page(inode, NULL, index, false); f2fs_unlock_op(sbi); - if (!IS_ERR(page)) { - f2fs_wait_on_page_writeback(page, DATA); - zero_user(page, start, len); - set_page_dirty(page); - f2fs_put_page(page, 1); - } + if (IS_ERR(page)) + return PTR_ERR(page); + + f2fs_wait_on_page_writeback(page, DATA); + zero_user(page, start, len); + set_page_dirty(page); + f2fs_put_page(page, 1); + return 0; } int truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end) @@ -760,14 +781,22 @@ static int punch_hole(struct inode *inode, loff_t offset, loff_t len) off_end = (offset + len) & (PAGE_CACHE_SIZE - 1); if (pg_start == pg_end) { - fill_zero(inode, pg_start, off_start, + ret = fill_zero(inode, pg_start, off_start, off_end - off_start); + if (ret) + return ret; } else { - if (off_start) - fill_zero(inode, pg_start++, off_start, - PAGE_CACHE_SIZE - off_start); - if (off_end) - fill_zero(inode, pg_end, 0, off_end); + if (off_start) { + ret = fill_zero(inode, pg_start++, off_start, + PAGE_CACHE_SIZE - off_start); + if (ret) + return ret; + } + if (off_end) { + ret = fill_zero(inode, pg_end, 0, off_end); + if (ret) + return ret; + } if (pg_start < pg_end) { struct address_space *mapping = inode->i_mapping; @@ -797,11 +826,11 @@ static int f2fs_do_collapse(struct inode *inode, pgoff_t start, pgoff_t end) pgoff_t nrpages = (i_size_read(inode) + PAGE_SIZE - 1) / PAGE_SIZE; int ret = 0; - f2fs_lock_op(sbi); - for (; end < nrpages; start++, end++) { block_t new_addr, old_addr; + f2fs_lock_op(sbi); + set_new_dnode(&dn, inode, NULL, NULL, 0); ret = get_dnode_of_data(&dn, end, LOOKUP_NODE_RA); if (ret && ret != -ENOENT) { @@ -817,13 +846,16 @@ static int f2fs_do_collapse(struct inode *inode, pgoff_t start, pgoff_t end) if (new_addr == NULL_ADDR) { set_new_dnode(&dn, inode, NULL, NULL, 0); ret = get_dnode_of_data(&dn, start, LOOKUP_NODE_RA); - if (ret && ret != -ENOENT) + if (ret && ret != -ENOENT) { goto out; - else if (ret == -ENOENT) + } else if (ret == -ENOENT) { + f2fs_unlock_op(sbi); continue; + } if (dn.data_blkaddr == NULL_ADDR) { f2fs_put_dnode(&dn); + f2fs_unlock_op(sbi); continue; } else { truncate_data_blocks_range(&dn, 1); @@ -862,8 +894,9 @@ static int f2fs_do_collapse(struct inode *inode, pgoff_t start, pgoff_t end) f2fs_put_dnode(&dn); } + f2fs_unlock_op(sbi); } - ret = 0; + return 0; out: f2fs_unlock_op(sbi); return ret; @@ -885,6 +918,14 @@ static int f2fs_collapse_range(struct inode *inode, loff_t offset, loff_t len) if (offset & (F2FS_BLKSIZE - 1) || len & (F2FS_BLKSIZE - 1)) return -EINVAL; + f2fs_balance_fs(F2FS_I_SB(inode)); + + if (f2fs_has_inline_data(inode)) { + ret = f2fs_convert_inline_inode(inode); + if (ret) + return ret; + } + pg_start = offset >> PAGE_CACHE_SHIFT; pg_end = (offset + len) >> PAGE_CACHE_SHIFT; @@ -946,14 +987,21 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len, off_end = (offset + len) & (PAGE_CACHE_SIZE - 1); if (pg_start == pg_end) { - fill_zero(inode, pg_start, off_start, off_end - off_start); + ret = fill_zero(inode, pg_start, off_start, + off_end - off_start); + if (ret) + return ret; + if (offset + len > new_size) new_size = offset + len; new_size = max_t(loff_t, new_size, offset + len); } else { if (off_start) { - fill_zero(inode, pg_start++, off_start, - PAGE_CACHE_SIZE - off_start); + ret = fill_zero(inode, pg_start++, off_start, + PAGE_CACHE_SIZE - off_start); + if (ret) + return ret; + new_size = max_t(loff_t, new_size, pg_start << PAGE_CACHE_SHIFT); } @@ -995,7 +1043,10 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len, } if (off_end) { - fill_zero(inode, pg_end, 0, off_end); + ret = fill_zero(inode, pg_end, 0, off_end); + if (ret) + goto out; + new_size = max_t(loff_t, new_size, offset + len); } } @@ -1033,6 +1084,12 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len) f2fs_balance_fs(sbi); + if (f2fs_has_inline_data(inode)) { + ret = f2fs_convert_inline_inode(inode); + if (ret) + return ret; + } + ret = truncate_blocks(inode, i_size_read(inode), true); if (ret) return ret; @@ -1302,6 +1359,7 @@ static int f2fs_ioc_getversion(struct file *filp, unsigned long arg) static int f2fs_ioc_start_atomic_write(struct file *filp) { struct inode *inode = file_inode(filp); + int ret; if (!inode_owner_or_capable(inode)) return -EACCES; @@ -1311,9 +1369,12 @@ static int f2fs_ioc_start_atomic_write(struct file *filp) if (f2fs_is_atomic_file(inode)) return 0; - set_inode_flag(F2FS_I(inode), FI_ATOMIC_FILE); + ret = f2fs_convert_inline_inode(inode); + if (ret) + return ret; - return f2fs_convert_inline_inode(inode); + set_inode_flag(F2FS_I(inode), FI_ATOMIC_FILE); + return 0; } static int f2fs_ioc_commit_atomic_write(struct file *filp) @@ -1333,10 +1394,13 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) if (f2fs_is_atomic_file(inode)) { clear_inode_flag(F2FS_I(inode), FI_ATOMIC_FILE); - commit_inmem_pages(inode, false); + ret = commit_inmem_pages(inode, false); + if (ret) + goto err_out; } - ret = f2fs_sync_file(filp, 0, LONG_MAX, 0); + ret = f2fs_sync_file(filp, 0, LLONG_MAX, 0); +err_out: mnt_drop_write_file(filp); return ret; } @@ -1344,6 +1408,7 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp) static int f2fs_ioc_start_volatile_write(struct file *filp) { struct inode *inode = file_inode(filp); + int ret; if (!inode_owner_or_capable(inode)) return -EACCES; @@ -1351,9 +1416,12 @@ static int f2fs_ioc_start_volatile_write(struct file *filp) if (f2fs_is_volatile_file(inode)) return 0; - set_inode_flag(F2FS_I(inode), FI_VOLATILE_FILE); + ret = f2fs_convert_inline_inode(inode); + if (ret) + return ret; - return f2fs_convert_inline_inode(inode); + set_inode_flag(F2FS_I(inode), FI_VOLATILE_FILE); + return 0; } static int f2fs_ioc_release_volatile_write(struct file *filp) @@ -1389,7 +1457,7 @@ static int f2fs_ioc_abort_volatile_write(struct file *filp) if (f2fs_is_atomic_file(inode)) { clear_inode_flag(F2FS_I(inode), FI_ATOMIC_FILE); - commit_inmem_pages(inode, false); + commit_inmem_pages(inode, true); } if (f2fs_is_volatile_file(inode)) @@ -1544,6 +1612,35 @@ static int f2fs_ioc_get_encryption_pwsalt(struct file *filp, unsigned long arg) return 0; } +static int f2fs_ioc_gc(struct file *filp, unsigned long arg) +{ + struct inode *inode = file_inode(filp); + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + __u32 i, count; + + if (!capable(CAP_SYS_ADMIN)) + return -EPERM; + + if (get_user(count, (__u32 __user *)arg)) + return -EFAULT; + + if (!count || count > F2FS_BATCH_GC_MAX_NUM) + return -EINVAL; + + for (i = 0; i < count; i++) { + if (!mutex_trylock(&sbi->gc_mutex)) + break; + + if (f2fs_gc(sbi)) + break; + } + + if (put_user(i, (__u32 __user *)arg)) + return -EFAULT; + + return 0; +} + long f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { switch (cmd) { @@ -1573,6 +1670,8 @@ long f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) return f2fs_ioc_get_encryption_policy(filp, arg); case F2FS_IOC_GET_ENCRYPTION_PWSALT: return f2fs_ioc_get_encryption_pwsalt(filp, arg); + case F2FS_IOC_GARBAGE_COLLECT: + return f2fs_ioc_gc(filp, arg); default: return -ENOTTY; } diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c index 22fb5ef3796621..782b8e72c094a8 100644 --- a/fs/f2fs/gc.c +++ b/fs/f2fs/gc.c @@ -391,23 +391,27 @@ static int check_valid_map(struct f2fs_sb_info *sbi, * On validity, copy that node with cold status, otherwise (invalid node) * ignore that. */ -static void gc_node_segment(struct f2fs_sb_info *sbi, +static int gc_node_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, unsigned int segno, int gc_type) { bool initial = true; struct f2fs_summary *entry; + block_t start_addr; int off; + start_addr = START_BLOCK(sbi, segno); + next_step: entry = sum; for (off = 0; off < sbi->blocks_per_seg; off++, entry++) { nid_t nid = le32_to_cpu(entry->nid); struct page *node_page; + struct node_info ni; /* stop BG_GC if there is not enough free sections. */ if (gc_type == BG_GC && has_not_enough_free_secs(sbi, 0)) - return; + return 0; if (check_valid_map(sbi, segno, off) == 0) continue; @@ -426,6 +430,12 @@ static void gc_node_segment(struct f2fs_sb_info *sbi, continue; } + get_node_info(sbi, nid, &ni); + if (ni.blk_addr != start_addr + off) { + f2fs_put_page(node_page, 1); + continue; + } + /* set page dirty and write it */ if (gc_type == FG_GC) { f2fs_wait_on_page_writeback(node_page, NODE); @@ -451,13 +461,11 @@ static void gc_node_segment(struct f2fs_sb_info *sbi, }; sync_node_pages(sbi, 0, &wbc); - /* - * In the case of FG_GC, it'd be better to reclaim this victim - * completely. - */ - if (get_valid_blocks(sbi, segno, 1) != 0) - goto next_step; + /* return 1 only if FG_GC succefully reclaimed one */ + if (get_valid_blocks(sbi, segno, 1) == 0) + return 1; } + return 0; } /* @@ -487,7 +495,7 @@ block_t start_bidx_of_node(unsigned int node_ofs, struct f2fs_inode_info *fi) return bidx * ADDRS_PER_BLOCK + ADDRS_PER_INODE(fi); } -static int check_dnode(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, +static bool is_alive(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, struct node_info *dni, block_t blkaddr, unsigned int *nofs) { struct page *node_page; @@ -500,13 +508,13 @@ static int check_dnode(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, node_page = get_node_page(sbi, nid); if (IS_ERR(node_page)) - return 0; + return false; get_node_info(sbi, nid, dni); if (sum->version != dni->version) { f2fs_put_page(node_page, 1); - return 0; + return false; } *nofs = ofs_of_node(node_page); @@ -514,8 +522,8 @@ static int check_dnode(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, f2fs_put_page(node_page, 1); if (source_blkaddr != blkaddr) - return 0; - return 1; + return false; + return true; } static void move_encrypted_block(struct inode *inode, block_t bidx) @@ -552,7 +560,10 @@ static void move_encrypted_block(struct inode *inode, block_t bidx) fio.page = page; fio.blk_addr = dn.data_blkaddr; - fio.encrypted_page = grab_cache_page(META_MAPPING(fio.sbi), fio.blk_addr); + fio.encrypted_page = pagecache_get_page(META_MAPPING(fio.sbi), + fio.blk_addr, + FGP_LOCK|FGP_CREAT, + GFP_NOFS); if (!fio.encrypted_page) goto put_out; @@ -636,7 +647,7 @@ static void move_data_page(struct inode *inode, block_t bidx, int gc_type) * If the parent node is not valid or the data block address is different, * the victim data block is ignored. */ -static void gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, +static int gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, struct gc_inode_list *gc_list, unsigned int segno, int gc_type) { struct super_block *sb = sbi->sb; @@ -659,7 +670,7 @@ static void gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, /* stop BG_GC if there is not enough free sections. */ if (gc_type == BG_GC && has_not_enough_free_secs(sbi, 0)) - return; + return 0; if (check_valid_map(sbi, segno, off) == 0) continue; @@ -670,7 +681,7 @@ static void gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, } /* Get an inode by ino with checking validity */ - if (check_dnode(sbi, entry, &dni, start_addr + off, &nofs) == 0) + if (!is_alive(sbi, entry, &dni, start_addr + off, &nofs)) continue; if (phase == 1) { @@ -724,15 +735,11 @@ static void gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum, if (gc_type == FG_GC) { f2fs_submit_merged_bio(sbi, DATA, WRITE); - /* - * In the case of FG_GC, it'd be better to reclaim this victim - * completely. - */ - if (get_valid_blocks(sbi, segno, 1) != 0) { - phase = 2; - goto next_step; - } + /* return 1 only if FG_GC succefully reclaimed one */ + if (get_valid_blocks(sbi, segno, 1) == 0) + return 1; } + return 0; } static int __get_victim(struct f2fs_sb_info *sbi, unsigned int *victim, @@ -748,12 +755,13 @@ static int __get_victim(struct f2fs_sb_info *sbi, unsigned int *victim, return ret; } -static void do_garbage_collect(struct f2fs_sb_info *sbi, unsigned int segno, +static int do_garbage_collect(struct f2fs_sb_info *sbi, unsigned int segno, struct gc_inode_list *gc_list, int gc_type) { struct page *sum_page; struct f2fs_summary_block *sum; struct blk_plug plug; + int nfree = 0; /* read segment summary of victim */ sum_page = get_sum_page(sbi, segno); @@ -773,10 +781,11 @@ static void do_garbage_collect(struct f2fs_sb_info *sbi, unsigned int segno, switch (GET_SUM_TYPE((&sum->footer))) { case SUM_TYPE_NODE: - gc_node_segment(sbi, sum->entries, segno, gc_type); + nfree = gc_node_segment(sbi, sum->entries, segno, gc_type); break; case SUM_TYPE_DATA: - gc_data_segment(sbi, sum->entries, gc_list, segno, gc_type); + nfree = gc_data_segment(sbi, sum->entries, gc_list, + segno, gc_type); break; } blk_finish_plug(&plug); @@ -785,11 +794,13 @@ static void do_garbage_collect(struct f2fs_sb_info *sbi, unsigned int segno, stat_inc_call_count(sbi->stat_info); f2fs_put_page(sum_page, 0); + return nfree; } int f2fs_gc(struct f2fs_sb_info *sbi) { - unsigned int segno, i; + unsigned int segno = NULL_SEGNO; + unsigned int i; int gc_type = BG_GC; int nfree = 0; int ret = -1; @@ -808,10 +819,11 @@ int f2fs_gc(struct f2fs_sb_info *sbi) if (gc_type == BG_GC && has_not_enough_free_secs(sbi, nfree)) { gc_type = FG_GC; - write_checkpoint(sbi, &cpc); + if (__get_victim(sbi, &segno, gc_type) || prefree_segments(sbi)) + write_checkpoint(sbi, &cpc); } - if (!__get_victim(sbi, &segno, gc_type)) + if (segno == NULL_SEGNO && !__get_victim(sbi, &segno, gc_type)) goto stop; ret = 0; @@ -821,13 +833,10 @@ int f2fs_gc(struct f2fs_sb_info *sbi) META_SSA); for (i = 0; i < sbi->segs_per_sec; i++) - do_garbage_collect(sbi, segno + i, &gc_list, gc_type); + nfree += do_garbage_collect(sbi, segno + i, &gc_list, gc_type); - if (gc_type == FG_GC) { + if (gc_type == FG_GC) sbi->cur_victim_sec = NULL_SEGNO; - nfree++; - WARN_ON(get_valid_blocks(sbi, segno, sbi->segs_per_sec)); - } if (has_not_enough_free_secs(sbi, nfree)) goto gc_more; diff --git a/fs/f2fs/gc.h b/fs/f2fs/gc.h index b4a65be9f7d3fc..c5a055b3376e6b 100644 --- a/fs/f2fs/gc.h +++ b/fs/f2fs/gc.h @@ -19,6 +19,12 @@ #define LIMIT_INVALID_BLOCK 40 /* percentage over total user space */ #define LIMIT_FREE_BLOCK 40 /* percentage over invalid + free space */ +/* + * with this macro, we can control the max time we do garbage collection, + * when user triggers batch mode gc by ioctl. + */ +#define F2FS_BATCH_GC_MAX_NUM 16 + /* Search max. number of dirty segments to select a victim segment */ #define DEF_MAX_VICTIM_SEARCH 4096 /* covers 8GB */ diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c index a13ffcc329923b..3d143be4289591 100644 --- a/fs/f2fs/inline.c +++ b/fs/f2fs/inline.c @@ -360,6 +360,10 @@ int make_empty_inline_dir(struct inode *inode, struct inode *parent, return 0; } +/* + * NOTE: ipage is grabbed by caller, but if any error occurs, we should + * release ipage in this function. + */ static int f2fs_convert_inline_dir(struct inode *dir, struct page *ipage, struct f2fs_inline_dentry *inline_dentry) { @@ -369,8 +373,10 @@ static int f2fs_convert_inline_dir(struct inode *dir, struct page *ipage, int err; page = grab_cache_page(dir->i_mapping, 0); - if (!page) + if (!page) { + f2fs_put_page(ipage, 1); return -ENOMEM; + } set_new_dnode(&dn, dir, ipage, NULL, 0); err = f2fs_reserve_block(&dn, 0); @@ -378,13 +384,21 @@ static int f2fs_convert_inline_dir(struct inode *dir, struct page *ipage, goto out; f2fs_wait_on_page_writeback(page, DATA); - zero_user_segment(page, 0, PAGE_CACHE_SIZE); + zero_user_segment(page, MAX_INLINE_DATA, PAGE_CACHE_SIZE); dentry_blk = kmap_atomic(page); /* copy data from inline dentry block to new dentry block */ memcpy(dentry_blk->dentry_bitmap, inline_dentry->dentry_bitmap, INLINE_DENTRY_BITMAP_SIZE); + memset(dentry_blk->dentry_bitmap + INLINE_DENTRY_BITMAP_SIZE, 0, + SIZE_OF_DENTRY_BITMAP - INLINE_DENTRY_BITMAP_SIZE); + /* + * we do not need to zero out remainder part of dentry and filename + * field, since we have used bitmap for marking the usage status of + * them, besides, we can also ignore copying/zeroing reserved space + * of dentry block, because them haven't been used so far. + */ memcpy(dentry_blk->dentry, inline_dentry->dentry, sizeof(struct f2fs_dir_entry) * NR_INLINE_DENTRY); memcpy(dentry_blk->filename, inline_dentry->filename, @@ -434,8 +448,9 @@ int f2fs_add_inline_entry(struct inode *dir, const struct qstr *name, slots, NR_INLINE_DENTRY); if (bit_pos >= NR_INLINE_DENTRY) { err = f2fs_convert_inline_dir(dir, ipage, dentry_blk); - if (!err) - err = -EAGAIN; + if (err) + return err; + err = -EAGAIN; goto out; } diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c index 2550868dc651ee..35aae65b3e5db6 100644 --- a/fs/f2fs/inode.c +++ b/fs/f2fs/inode.c @@ -12,7 +12,6 @@ #include #include #include -#include #include "f2fs.h" #include "node.h" @@ -34,8 +33,8 @@ void f2fs_set_inode_flags(struct inode *inode) new_fl |= S_NOATIME; if (flags & FS_DIRSYNC_FL) new_fl |= S_DIRSYNC; - set_mask_bits(&inode->i_flags, - S_SYNC|S_APPEND|S_IMMUTABLE|S_NOATIME|S_DIRSYNC, new_fl); + inode_set_flags(inode, new_fl, + S_SYNC|S_APPEND|S_IMMUTABLE|S_NOATIME|S_DIRSYNC); } static void __get_inode_rdev(struct inode *inode, struct f2fs_inode *ri) @@ -139,7 +138,7 @@ static int do_read_inode(struct inode *inode) fi->i_pino = le32_to_cpu(ri->i_pino); fi->i_dir_level = ri->i_dir_level; - f2fs_init_extent_cache(inode, &ri->i_ext); + f2fs_init_extent_tree(inode, &ri->i_ext); get_inline_info(fi, ri); @@ -155,6 +154,7 @@ static int do_read_inode(struct inode *inode) f2fs_put_page(node_page, 1); + stat_inc_inline_xattr(inode); stat_inc_inline_inode(inode); stat_inc_inline_dir(inode); @@ -237,10 +237,11 @@ void update_inode(struct inode *inode, struct page *node_page) ri->i_size = cpu_to_le64(i_size_read(inode)); ri->i_blocks = cpu_to_le64(inode->i_blocks); - read_lock(&F2FS_I(inode)->ext_lock); - set_raw_extent(&F2FS_I(inode)->ext, &ri->i_ext); - read_unlock(&F2FS_I(inode)->ext_lock); - + if (F2FS_I(inode)->extent_tree) + set_raw_extent(&F2FS_I(inode)->extent_tree->largest, + &ri->i_ext); + else + memset(&ri->i_ext, 0, sizeof(ri->i_ext)); set_raw_inline(F2FS_I(inode), ri); ri->i_atime = cpu_to_le64(inode->i_atime.tv_sec); @@ -314,7 +315,9 @@ int f2fs_write_inode(struct inode *inode, struct writeback_control *wbc) void f2fs_evict_inode(struct inode *inode) { struct f2fs_sb_info *sbi = F2FS_I_SB(inode); - nid_t xnid = F2FS_I(inode)->i_xattr_nid; + struct f2fs_inode_info *fi = F2FS_I(inode); + nid_t xnid = fi->i_xattr_nid; + int err = 0; /* some remained atomic pages should discarded */ if (f2fs_is_atomic_file(inode)) @@ -330,41 +333,62 @@ void f2fs_evict_inode(struct inode *inode) f2fs_bug_on(sbi, get_dirty_pages(inode)); remove_dirty_dir_inode(inode); + f2fs_destroy_extent_tree(inode); + if (inode->i_nlink || is_bad_inode(inode)) goto no_delete; sb_start_intwrite(inode->i_sb); - set_inode_flag(F2FS_I(inode), FI_NO_ALLOC); + set_inode_flag(fi, FI_NO_ALLOC); i_size_write(inode, 0); if (F2FS_HAS_BLOCKS(inode)) - f2fs_truncate(inode); + err = f2fs_truncate(inode, true); - f2fs_lock_op(sbi); - remove_inode_page(inode); - f2fs_unlock_op(sbi); + if (!err) { + f2fs_lock_op(sbi); + err = remove_inode_page(inode); + f2fs_unlock_op(sbi); + } sb_end_intwrite(inode->i_sb); no_delete: + stat_dec_inline_xattr(inode); stat_dec_inline_dir(inode); stat_dec_inline_inode(inode); - /* update extent info in inode */ - if (inode->i_nlink) - f2fs_preserve_extent_tree(inode); - f2fs_destroy_extent_tree(inode); - invalidate_mapping_pages(NODE_MAPPING(sbi), inode->i_ino, inode->i_ino); if (xnid) invalidate_mapping_pages(NODE_MAPPING(sbi), xnid, xnid); - if (is_inode_flag_set(F2FS_I(inode), FI_APPEND_WRITE)) + if (is_inode_flag_set(fi, FI_APPEND_WRITE)) add_dirty_inode(sbi, inode->i_ino, APPEND_INO); - if (is_inode_flag_set(F2FS_I(inode), FI_UPDATE_WRITE)) + if (is_inode_flag_set(fi, FI_UPDATE_WRITE)) add_dirty_inode(sbi, inode->i_ino, UPDATE_INO); + if (is_inode_flag_set(fi, FI_FREE_NID)) { + if (err && err != -ENOENT) + alloc_nid_done(sbi, inode->i_ino); + else + alloc_nid_failed(sbi, inode->i_ino); + clear_inode_flag(fi, FI_FREE_NID); + } + + if (err && err != -ENOENT) { + if (!exist_written_data(sbi, inode->i_ino, ORPHAN_INO)) { + /* + * get here because we failed to release resource + * of inode previously, reminder our user to run fsck + * for fixing. + */ + set_sbi_flag(sbi, SBI_NEED_FSCK); + f2fs_msg(sbi->sb, KERN_WARNING, + "inode (ino:%lu) resource leak, run fsck " + "to fix this issue!", inode->i_ino); + } + } out_clear: #ifdef CONFIG_F2FS_FS_ENCRYPTION - if (F2FS_I(inode)->i_crypt_info) - f2fs_free_encryption_info(inode, F2FS_I(inode)->i_crypt_info); + if (fi->i_crypt_info) + f2fs_free_encryption_info(inode, fi->i_crypt_info); #endif clear_inode(inode); } @@ -373,6 +397,7 @@ void f2fs_evict_inode(struct inode *inode) void handle_failed_inode(struct inode *inode) { struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + int err = 0; clear_nlink(inode); make_bad_inode(inode); @@ -380,13 +405,29 @@ void handle_failed_inode(struct inode *inode) i_size_write(inode, 0); if (F2FS_HAS_BLOCKS(inode)) - f2fs_truncate(inode); + err = f2fs_truncate(inode, false); - remove_inode_page(inode); + if (!err) + err = remove_inode_page(inode); + + /* + * if we skip truncate_node in remove_inode_page bacause we failed + * before, it's better to find another way to release resource of + * this inode (e.g. valid block count, node block or nid). Here we + * choose to add this inode to orphan list, so that we can call iput + * for releasing in orphan recovery flow. + * + * Note: we should add inode to orphan list before f2fs_unlock_op() + * so we can prevent losing this orphan when encoutering checkpoint + * and following suddenly power-off. + */ + if (err && err != -ENOENT) { + err = acquire_orphan_inode(sbi); + if (!err) + add_orphan_inode(sbi, inode->i_ino); + } - clear_inode_flag(F2FS_I(inode), FI_INLINE_DATA); - clear_inode_flag(F2FS_I(inode), FI_INLINE_DENTRY); - alloc_nid_failed(sbi, inode->i_ino); + set_inode_flag(F2FS_I(inode), FI_FREE_NID); f2fs_unlock_op(sbi); /* iput will drop the inode object */ diff --git a/fs/f2fs/namei.c b/fs/f2fs/namei.c index fdbae21ee8fb2c..a680bf38e4f0a7 100644 --- a/fs/f2fs/namei.c +++ b/fs/f2fs/namei.c @@ -53,7 +53,7 @@ static struct inode *f2fs_new_inode(struct inode *dir, umode_t mode) if (err) { err = -EINVAL; nid_free = true; - goto out; + goto fail; } /* If the directory encrypted, then we should encrypt the inode. */ @@ -65,6 +65,9 @@ static struct inode *f2fs_new_inode(struct inode *dir, umode_t mode) if (f2fs_may_inline_dentry(inode)) set_inode_flag(F2FS_I(inode), FI_INLINE_DENTRY); + f2fs_init_extent_tree(inode, NULL); + + stat_inc_inline_xattr(inode); stat_inc_inline_inode(inode); stat_inc_inline_dir(inode); @@ -72,15 +75,12 @@ static struct inode *f2fs_new_inode(struct inode *dir, umode_t mode) mark_inode_dirty(inode); return inode; -out: - clear_nlink(inode); - unlock_new_inode(inode); fail: trace_f2fs_new_inode(inode, err); make_bad_inode(inode); - iput(inode); if (nid_free) - alloc_nid_failed(sbi, ino); + set_inode_flag(F2FS_I(inode), FI_FREE_NID); + iput(inode); return ERR_PTR(err); } @@ -89,7 +89,14 @@ static int is_multimedia_file(const unsigned char *s, const char *sub) size_t slen = strlen(s); size_t sublen = strlen(sub); - if (sublen > slen) + /* + * filename format of multimedia file should be defined as: + * "filename + '.' + extension". + */ + if (slen < sublen + 2) + return 0; + + if (s[slen - sublen - 1] != '.') return 0; return !strncasecmp(s + slen - sublen, sub, sublen); diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c index 7dd63b794bfb5a..27d1a74dd6f34f 100644 --- a/fs/f2fs/node.c +++ b/fs/f2fs/node.c @@ -159,7 +159,7 @@ static void __set_nat_cache_dirty(struct f2fs_nm_info *nm_i, head = radix_tree_lookup(&nm_i->nat_set_root, set); if (!head) { - head = f2fs_kmem_cache_alloc(nat_entry_set_slab, GFP_ATOMIC); + head = f2fs_kmem_cache_alloc(nat_entry_set_slab, GFP_NOFS); INIT_LIST_HEAD(&head->entry_list); INIT_LIST_HEAD(&head->set_list); @@ -246,7 +246,7 @@ static struct nat_entry *grab_nat_entry(struct f2fs_nm_info *nm_i, nid_t nid) { struct nat_entry *new; - new = f2fs_kmem_cache_alloc(nat_entry_slab, GFP_ATOMIC); + new = f2fs_kmem_cache_alloc(nat_entry_slab, GFP_NOFS); f2fs_radix_tree_insert(&nm_i->nat_root, nid, new); memset(new, 0, sizeof(struct nat_entry)); nat_set_nid(new, nid); @@ -306,6 +306,10 @@ static void set_node_addr(struct f2fs_sb_info *sbi, struct node_info *ni, if (nat_get_blkaddr(e) != NEW_ADDR && new_blkaddr == NULL_ADDR) { unsigned char version = nat_get_version(e); nat_set_version(e, inc_node_version(version)); + + /* in order to reuse the nid */ + if (nm_i->next_scan_nid > ni->nid) + nm_i->next_scan_nid = ni->nid; } /* change address */ @@ -328,11 +332,11 @@ static void set_node_addr(struct f2fs_sb_info *sbi, struct node_info *ni, int try_to_free_nats(struct f2fs_sb_info *sbi, int nr_shrink) { struct f2fs_nm_info *nm_i = NM_I(sbi); + int nr = nr_shrink; - if (available_free_memory(sbi, NAT_ENTRIES)) + if (!down_write_trylock(&nm_i->nat_tree_lock)) return 0; - down_write(&nm_i->nat_tree_lock); while (nr_shrink && !list_empty(&nm_i->nat_entries)) { struct nat_entry *ne; ne = list_first_entry(&nm_i->nat_entries, @@ -341,7 +345,7 @@ int try_to_free_nats(struct f2fs_sb_info *sbi, int nr_shrink) nr_shrink--; } up_write(&nm_i->nat_tree_lock); - return nr_shrink; + return nr - nr_shrink; } /* @@ -898,17 +902,20 @@ int truncate_xattr_node(struct inode *inode, struct page *page) * Caller should grab and release a rwsem by calling f2fs_lock_op() and * f2fs_unlock_op(). */ -void remove_inode_page(struct inode *inode) +int remove_inode_page(struct inode *inode) { struct dnode_of_data dn; + int err; set_new_dnode(&dn, inode, NULL, NULL, inode->i_ino); - if (get_dnode_of_data(&dn, 0, LOOKUP_NODE)) - return; + err = get_dnode_of_data(&dn, 0, LOOKUP_NODE); + if (err) + return err; - if (truncate_xattr_node(inode, dn.inode_page)) { + err = truncate_xattr_node(inode, dn.inode_page); + if (err) { f2fs_put_dnode(&dn); - return; + return err; } /* remove potential inline_data blocks */ @@ -922,6 +929,7 @@ void remove_inode_page(struct inode *inode) /* will put inode & node pages */ truncate_node(&dn); + return 0; } struct page *new_inode_page(struct inode *inode) @@ -991,8 +999,7 @@ struct page *new_node_page(struct dnode_of_data *dn, /* * Caller should do after getting the following values. * 0: f2fs_put_page(page, 0) - * LOCKED_PAGE: f2fs_put_page(page, 1) - * error: nothing + * LOCKED_PAGE or error: f2fs_put_page(page, 1) */ static int read_node_page(struct page *page, int rw) { @@ -1010,7 +1017,6 @@ static int read_node_page(struct page *page, int rw) if (unlikely(ni.blk_addr == NULL_ADDR)) { ClearPageUptodate(page); - f2fs_put_page(page, 1); return -ENOENT; } @@ -1041,10 +1047,7 @@ void ra_node_page(struct f2fs_sb_info *sbi, nid_t nid) return; err = read_node_page(apage, READA); - if (err == 0) - f2fs_put_page(apage, 0); - else if (err == LOCKED_PAGE) - f2fs_put_page(apage, 1); + f2fs_put_page(apage, err ? 1 : 0); } struct page *get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid) @@ -1057,10 +1060,12 @@ struct page *get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid) return ERR_PTR(-ENOMEM); err = read_node_page(page, READ_SYNC); - if (err < 0) + if (err < 0) { + f2fs_put_page(page, 1); return ERR_PTR(err); - else if (err != LOCKED_PAGE) + } else if (err != LOCKED_PAGE) { lock_page(page); + } if (unlikely(!PageUptodate(page) || nid != nid_of_node(page))) { ClearPageUptodate(page); @@ -1096,10 +1101,12 @@ struct page *get_node_page_ra(struct page *parent, int start) return ERR_PTR(-ENOMEM); err = read_node_page(page, READ_SYNC); - if (err < 0) + if (err < 0) { + f2fs_put_page(page, 1); return ERR_PTR(err); - else if (err == LOCKED_PAGE) + } else if (err == LOCKED_PAGE) { goto page_hit; + } blk_start_plug(&plug); @@ -1533,7 +1540,7 @@ static void build_free_nids(struct f2fs_sb_info *sbi) if (unlikely(nid >= nm_i->max_nid)) nid = 0; - if (i++ == FREE_NID_PAGES) + if (++i >= FREE_NID_PAGES) break; } @@ -1570,6 +1577,8 @@ bool alloc_nid(struct f2fs_sb_info *sbi, nid_t *nid) /* We should not use stale free nids created by build_free_nids */ if (nm_i->fcnt && !on_build_free_nids(nm_i)) { + struct node_info ni; + f2fs_bug_on(sbi, list_empty(&nm_i->free_nid_list)); list_for_each_entry(i, &nm_i->free_nid_list, list) if (i->state == NID_NEW) @@ -1580,6 +1589,13 @@ bool alloc_nid(struct f2fs_sb_info *sbi, nid_t *nid) i->state = NID_ALLOC; nm_i->fcnt--; spin_unlock(&nm_i->free_nid_list_lock); + + /* check nid is allocated already */ + get_node_info(sbi, *nid, &ni); + if (ni.blk_addr != NULL_ADDR) { + alloc_nid_done(sbi, *nid); + goto retry; + } return true; } spin_unlock(&nm_i->free_nid_list_lock); @@ -1636,6 +1652,32 @@ void alloc_nid_failed(struct f2fs_sb_info *sbi, nid_t nid) kmem_cache_free(free_nid_slab, i); } +int try_to_free_nids(struct f2fs_sb_info *sbi, int nr_shrink) +{ + struct f2fs_nm_info *nm_i = NM_I(sbi); + struct free_nid *i, *next; + int nr = nr_shrink; + + if (!mutex_trylock(&nm_i->build_lock)) + return 0; + + spin_lock(&nm_i->free_nid_list_lock); + list_for_each_entry_safe(i, next, &nm_i->free_nid_list, list) { + if (nr_shrink <= 0 || nm_i->fcnt <= NAT_ENTRY_PER_BLOCK) + break; + if (i->state == NID_ALLOC) + continue; + __del_from_free_nid_list(nm_i, i); + kmem_cache_free(free_nid_slab, i); + nm_i->fcnt--; + nr_shrink--; + } + spin_unlock(&nm_i->free_nid_list_lock); + mutex_unlock(&nm_i->build_lock); + + return nr - nr_shrink; +} + void recover_inline_xattr(struct inode *inode, struct page *page) { void *src_addr, *dst_addr; diff --git a/fs/f2fs/recovery.c b/fs/f2fs/recovery.c index 24a8c1d4f45f28..faec2ca004b9e8 100644 --- a/fs/f2fs/recovery.c +++ b/fs/f2fs/recovery.c @@ -399,14 +399,35 @@ static int do_recover_data(struct f2fs_sb_info *sbi, struct inode *inode, f2fs_bug_on(sbi, ni.ino != ino_of_node(page)); f2fs_bug_on(sbi, ofs_of_node(dn.node_page) != ofs_of_node(page)); - for (; start < end; start++) { + for (; start < end; start++, dn.ofs_in_node++) { block_t src, dest; src = datablock_addr(dn.node_page, dn.ofs_in_node); dest = datablock_addr(page, dn.ofs_in_node); - if (src != dest && dest != NEW_ADDR && dest != NULL_ADDR && - is_valid_blkaddr(sbi, dest, META_POR)) { + /* skip recovering if dest is the same as src */ + if (src == dest) + continue; + + /* dest is invalid, just invalidate src block */ + if (dest == NULL_ADDR) { + truncate_data_blocks_range(&dn, 1); + continue; + } + + /* + * dest is reserved block, invalidate src block + * and then reserve one new block in dnode page. + */ + if (dest == NEW_ADDR) { + truncate_data_blocks_range(&dn, 1); + err = reserve_new_block(&dn); + f2fs_bug_on(sbi, err); + continue; + } + + /* dest is valid block, try to recover from src to dest */ + if (is_valid_blkaddr(sbi, dest, META_POR)) { if (src == NULL_ADDR) { err = reserve_new_block(&dn); @@ -424,7 +445,6 @@ static int do_recover_data(struct f2fs_sb_info *sbi, struct inode *inode, ni.version, false); recovered++; } - dn.ofs_in_node++; } if (IS_INODE(dn.node_page)) @@ -525,14 +545,12 @@ int recover_fsync_data(struct f2fs_sb_info *sbi) INIT_LIST_HEAD(&inode_list); - /* step #1: find fsynced inode numbers */ - set_sbi_flag(sbi, SBI_POR_DOING); - /* prevent checkpoint */ mutex_lock(&sbi->cp_mutex); blkaddr = NEXT_FREE_BLKADDR(sbi, curseg); + /* step #1: find fsynced inode numbers */ err = find_fsync_dnodes(sbi, &inode_list); if (err) goto out; @@ -561,11 +579,20 @@ int recover_fsync_data(struct f2fs_sb_info *sbi) clear_sbi_flag(sbi, SBI_POR_DOING); if (err) { - discard_next_dnode(sbi, blkaddr); + bool invalidate = false; + + if (discard_next_dnode(sbi, blkaddr)) + invalidate = true; /* Flush all the NAT/SIT pages */ while (get_pages(sbi, F2FS_DIRTY_META)) sync_meta_pages(sbi, META, LONG_MAX); + + /* invalidate temporary meta page */ + if (invalidate) + invalidate_mapping_pages(META_MAPPING(sbi), + blkaddr, blkaddr); + set_ckpt_flags(sbi->ckpt, CP_ERROR_FLAG); mutex_unlock(&sbi->cp_mutex); } else if (need_writecp) { diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 61b97f9cb9f657..78e6d06968477d 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -197,28 +197,20 @@ void register_inmem_page(struct inode *inode, struct page *page) { struct f2fs_inode_info *fi = F2FS_I(inode); struct inmem_pages *new; - int err; - SetPagePrivate(page); f2fs_trace_pid(page); + set_page_private(page, (unsigned long)ATOMIC_WRITTEN_PAGE); + SetPagePrivate(page); + new = f2fs_kmem_cache_alloc(inmem_entry_slab, GFP_NOFS); /* add atomic page indices to the list */ new->page = page; INIT_LIST_HEAD(&new->list); -retry: + /* increase reference count with clean state */ mutex_lock(&fi->inmem_lock); - err = radix_tree_insert(&fi->inmem_root, page->index, new); - if (err == -EEXIST) { - mutex_unlock(&fi->inmem_lock); - kmem_cache_free(inmem_entry_slab, new); - return; - } else if (err) { - mutex_unlock(&fi->inmem_lock); - goto retry; - } get_page(page); list_add_tail(&new->list, &fi->inmem_pages); inc_page_count(F2FS_I_SB(inode), F2FS_INMEM_PAGES); @@ -227,7 +219,7 @@ void register_inmem_page(struct inode *inode, struct page *page) trace_f2fs_register_inmem_page(page, INMEM); } -void commit_inmem_pages(struct inode *inode, bool abort) +int commit_inmem_pages(struct inode *inode, bool abort) { struct f2fs_sb_info *sbi = F2FS_I_SB(inode); struct f2fs_inode_info *fi = F2FS_I(inode); @@ -239,6 +231,7 @@ void commit_inmem_pages(struct inode *inode, bool abort) .rw = WRITE_SYNC | REQ_PRIO, .encrypted_page = NULL, }; + int err = 0; /* * The abort is true only when f2fs_evict_inode is called. @@ -254,8 +247,8 @@ void commit_inmem_pages(struct inode *inode, bool abort) mutex_lock(&fi->inmem_lock); list_for_each_entry_safe(cur, tmp, &fi->inmem_pages, list) { + lock_page(cur->page); if (!abort) { - lock_page(cur->page); if (cur->page->mapping == inode->i_mapping) { set_page_dirty(cur->page); f2fs_wait_on_page_writeback(cur->page, DATA); @@ -263,15 +256,20 @@ void commit_inmem_pages(struct inode *inode, bool abort) inode_dec_dirty_pages(inode); trace_f2fs_commit_inmem_page(cur->page, INMEM); fio.page = cur->page; - do_write_data_page(&fio); + err = do_write_data_page(&fio); submit_bio = true; + if (err) { + unlock_page(cur->page); + break; + } } - f2fs_put_page(cur->page, 1); } else { trace_f2fs_commit_inmem_page(cur->page, INMEM_DROP); - put_page(cur->page); } - radix_tree_delete(&fi->inmem_root, cur->page->index); + set_page_private(cur->page, 0); + ClearPagePrivate(cur->page); + f2fs_put_page(cur->page, 1); + list_del(&cur->list); kmem_cache_free(inmem_entry_slab, cur); dec_page_count(F2FS_I_SB(inode), F2FS_INMEM_PAGES); @@ -283,6 +281,7 @@ void commit_inmem_pages(struct inode *inode, bool abort) if (submit_bio) f2fs_submit_merged_bio(sbi, DATA, WRITE); } + return err; } /* @@ -304,10 +303,18 @@ void f2fs_balance_fs(struct f2fs_sb_info *sbi) void f2fs_balance_fs_bg(struct f2fs_sb_info *sbi) { /* try to shrink extent cache when there is no enough memory */ - f2fs_shrink_extent_tree(sbi, EXTENT_CACHE_SHRINK_NUMBER); + if (!available_free_memory(sbi, EXTENT_CACHE)) + f2fs_shrink_extent_tree(sbi, EXTENT_CACHE_SHRINK_NUMBER); + + /* check the # of cached NAT entries */ + if (!available_free_memory(sbi, NAT_ENTRIES)) + try_to_free_nats(sbi, NAT_ENTRY_PER_BLOCK); + + if (!available_free_memory(sbi, FREE_NIDS)) + try_to_free_nids(sbi, NAT_ENTRY_PER_BLOCK * FREE_NID_PAGES); - /* check the # of cached NAT entries and prefree segments */ - if (try_to_free_nats(sbi, NAT_ENTRY_PER_BLOCK) || + /* checkpoint is the only way to shrink partial cached entries */ + if (!available_free_memory(sbi, NAT_ENTRIES) || excess_prefree_segs(sbi) || !available_free_memory(sbi, INO_ENTRIES)) f2fs_sync_fs(sbi->sb, true); @@ -323,10 +330,12 @@ static int issue_flush_thread(void *data) return 0; if (!llist_empty(&fcc->issue_list)) { - struct bio *bio = bio_alloc(GFP_NOIO, 0); + struct bio *bio; struct flush_cmd *cmd, *next; int ret; + bio = f2fs_bio_alloc(0); + fcc->dispatch_list = llist_del_all(&fcc->issue_list); fcc->dispatch_list = llist_reverse_order(fcc->dispatch_list); @@ -358,8 +367,15 @@ int f2fs_issue_flush(struct f2fs_sb_info *sbi) if (test_opt(sbi, NOBARRIER)) return 0; - if (!test_opt(sbi, FLUSH_MERGE)) - return blkdev_issue_flush(sbi->sb->s_bdev, GFP_KERNEL, NULL); + if (!test_opt(sbi, FLUSH_MERGE)) { + struct bio *bio = f2fs_bio_alloc(0); + int ret; + + bio->bi_bdev = sbi->sb->s_bdev; + ret = submit_bio_wait(WRITE_FLUSH, bio); + bio_put(bio); + return ret; + } init_completion(&cmd.wait); @@ -503,7 +519,7 @@ static int f2fs_issue_discard(struct f2fs_sb_info *sbi, return blkdev_issue_discard(sbi->sb->s_bdev, start, len, GFP_NOFS, 0); } -void discard_next_dnode(struct f2fs_sb_info *sbi, block_t blkaddr) +bool discard_next_dnode(struct f2fs_sb_info *sbi, block_t blkaddr) { int err = -ENOTSUPP; @@ -513,13 +529,16 @@ void discard_next_dnode(struct f2fs_sb_info *sbi, block_t blkaddr) unsigned int offset = GET_BLKOFF_FROM_SEG0(sbi, blkaddr); if (f2fs_test_bit(offset, se->discard_map)) - return; + return false; err = f2fs_issue_discard(sbi, blkaddr, 1); } - if (err) + if (err) { update_meta_page(sbi, NULL, blkaddr); + return true; + } + return false; } static void __add_discard_entry(struct f2fs_sb_info *sbi, @@ -1218,7 +1237,8 @@ void allocate_data_block(struct f2fs_sb_info *sbi, struct page *page, mutex_lock(&sit_i->sentry_lock); /* direct_io'ed data is aligned to the segment for better performance */ - if (direct_io && curseg->next_blkoff) + if (direct_io && curseg->next_blkoff && + !has_not_enough_free_secs(sbi, 0)) __allocate_new_segments(sbi, type); *new_blkaddr = NEXT_FREE_BLKADDR(sbi, curseg); @@ -1733,7 +1753,7 @@ static struct page *get_next_sit_page(struct f2fs_sb_info *sbi, static struct sit_entry_set *grab_sit_entry_set(void) { struct sit_entry_set *ses = - f2fs_kmem_cache_alloc(sit_entry_set_slab, GFP_ATOMIC); + f2fs_kmem_cache_alloc(sit_entry_set_slab, GFP_NOFS); ses->entry_cnt = 0; INIT_LIST_HEAD(&ses->set_list); diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h index 79e7b879a75321..b6e4ed15c6988f 100644 --- a/fs/f2fs/segment.h +++ b/fs/f2fs/segment.h @@ -177,6 +177,15 @@ struct segment_allocation { void (*allocate_segment)(struct f2fs_sb_info *, int, bool); }; +/* + * this value is set in page as a private data which indicate that + * the page is atomically written, and it is in inmem_pages list. + */ +#define ATOMIC_WRITTEN_PAGE 0x0000ffff + +#define IS_ATOMIC_WRITTEN_PAGE(page) \ + (page_private(page) == (unsigned long)ATOMIC_WRITTEN_PAGE) + struct inmem_pages { struct list_head list; struct page *page; @@ -555,16 +564,15 @@ static inline unsigned short curseg_blkoff(struct f2fs_sb_info *sbi, int type) return curseg->next_blkoff; } -#ifdef CONFIG_F2FS_CHECK_FS static inline void check_seg_range(struct f2fs_sb_info *sbi, unsigned int segno) { - BUG_ON(segno > TOTAL_SEGS(sbi) - 1); + f2fs_bug_on(sbi, segno > TOTAL_SEGS(sbi) - 1); } static inline void verify_block_addr(struct f2fs_sb_info *sbi, block_t blk_addr) { - BUG_ON(blk_addr < SEG0_BLKADDR(sbi)); - BUG_ON(blk_addr >= MAX_BLKADDR(sbi)); + f2fs_bug_on(sbi, blk_addr < SEG0_BLKADDR(sbi) + || blk_addr >= MAX_BLKADDR(sbi)); } /* @@ -573,16 +581,11 @@ static inline void verify_block_addr(struct f2fs_sb_info *sbi, block_t blk_addr) static inline void check_block_count(struct f2fs_sb_info *sbi, int segno, struct f2fs_sit_entry *raw_sit) { +#ifdef CONFIG_F2FS_CHECK_FS bool is_valid = test_bit_le(0, raw_sit->valid_map) ? true : false; int valid_blocks = 0; int cur_pos = 0, next_pos; - /* check segment usage */ - BUG_ON(GET_SIT_VBLOCKS(raw_sit) > sbi->blocks_per_seg); - - /* check boundary of a given segment number */ - BUG_ON(segno > TOTAL_SEGS(sbi) - 1); - /* check bitmap with valid block count */ do { if (is_valid) { @@ -598,35 +601,11 @@ static inline void check_block_count(struct f2fs_sb_info *sbi, is_valid = !is_valid; } while (cur_pos < sbi->blocks_per_seg); BUG_ON(GET_SIT_VBLOCKS(raw_sit) != valid_blocks); -} -#else -static inline void check_seg_range(struct f2fs_sb_info *sbi, unsigned int segno) -{ - if (segno > TOTAL_SEGS(sbi) - 1) - set_sbi_flag(sbi, SBI_NEED_FSCK); -} - -static inline void verify_block_addr(struct f2fs_sb_info *sbi, block_t blk_addr) -{ - if (blk_addr < SEG0_BLKADDR(sbi) || blk_addr >= MAX_BLKADDR(sbi)) - set_sbi_flag(sbi, SBI_NEED_FSCK); -} - -/* - * Summary block is always treated as an invalid block - */ -static inline void check_block_count(struct f2fs_sb_info *sbi, - int segno, struct f2fs_sit_entry *raw_sit) -{ - /* check segment usage */ - if (GET_SIT_VBLOCKS(raw_sit) > sbi->blocks_per_seg) - set_sbi_flag(sbi, SBI_NEED_FSCK); - - /* check boundary of a given segment number */ - if (segno > TOTAL_SEGS(sbi) - 1) - set_sbi_flag(sbi, SBI_NEED_FSCK); -} #endif + /* check segment usage, and check boundary of a given segment number */ + f2fs_bug_on(sbi, GET_SIT_VBLOCKS(raw_sit) > sbi->blocks_per_seg + || segno > TOTAL_SEGS(sbi) - 1); +} static inline pgoff_t current_sit_addr(struct f2fs_sb_info *sbi, unsigned int start) diff --git a/fs/f2fs/shrinker.c b/fs/f2fs/shrinker.c new file mode 100644 index 00000000000000..da0d8e0b55a5d8 --- /dev/null +++ b/fs/f2fs/shrinker.c @@ -0,0 +1,139 @@ +/* + * f2fs shrinker support + * the basic infra was copied from fs/ubifs/shrinker.c + * + * Copyright (c) 2015 Motorola Mobility + * Copyright (c) 2015 Jaegeuk Kim + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ +#include +#include + +#include "f2fs.h" + +static LIST_HEAD(f2fs_list); +static DEFINE_SPINLOCK(f2fs_list_lock); +static unsigned int shrinker_run_no; + +static unsigned long __count_nat_entries(struct f2fs_sb_info *sbi) +{ + return NM_I(sbi)->nat_cnt - NM_I(sbi)->dirty_nat_cnt; +} + +static unsigned long __count_free_nids(struct f2fs_sb_info *sbi) +{ + if (NM_I(sbi)->fcnt > NAT_ENTRY_PER_BLOCK) + return NM_I(sbi)->fcnt - NAT_ENTRY_PER_BLOCK; + return 0; +} + +static unsigned long __count_extent_cache(struct f2fs_sb_info *sbi) +{ + return sbi->total_ext_tree + atomic_read(&sbi->total_ext_node); +} + +unsigned long f2fs_shrink_count(struct shrinker *shrink, + struct shrink_control *sc) +{ + struct f2fs_sb_info *sbi; + struct list_head *p; + unsigned long count = 0; + + spin_lock(&f2fs_list_lock); + p = f2fs_list.next; + while (p != &f2fs_list) { + sbi = list_entry(p, struct f2fs_sb_info, s_list); + + /* stop f2fs_put_super */ + if (!mutex_trylock(&sbi->umount_mutex)) { + p = p->next; + continue; + } + spin_unlock(&f2fs_list_lock); + + /* count extent cache entries */ + count += __count_extent_cache(sbi); + + /* shrink clean nat cache entries */ + count += __count_nat_entries(sbi); + + /* count free nids cache entries */ + count += __count_free_nids(sbi); + + spin_lock(&f2fs_list_lock); + p = p->next; + mutex_unlock(&sbi->umount_mutex); + } + spin_unlock(&f2fs_list_lock); + return count; +} + +unsigned long f2fs_shrink_scan(struct shrinker *shrink, + struct shrink_control *sc) +{ + unsigned long nr = sc->nr_to_scan; + struct f2fs_sb_info *sbi; + struct list_head *p; + unsigned int run_no; + unsigned long freed = 0; + + spin_lock(&f2fs_list_lock); + do { + run_no = ++shrinker_run_no; + } while (run_no == 0); + p = f2fs_list.next; + while (p != &f2fs_list) { + sbi = list_entry(p, struct f2fs_sb_info, s_list); + + if (sbi->shrinker_run_no == run_no) + break; + + /* stop f2fs_put_super */ + if (!mutex_trylock(&sbi->umount_mutex)) { + p = p->next; + continue; + } + spin_unlock(&f2fs_list_lock); + + sbi->shrinker_run_no = run_no; + + /* shrink extent cache entries */ + freed += f2fs_shrink_extent_tree(sbi, nr >> 1); + + /* shrink clean nat cache entries */ + if (freed < nr) + freed += try_to_free_nats(sbi, nr - freed); + + /* shrink free nids cache entries */ + if (freed < nr) + freed += try_to_free_nids(sbi, nr - freed); + + spin_lock(&f2fs_list_lock); + p = p->next; + list_move_tail(&sbi->s_list, &f2fs_list); + mutex_unlock(&sbi->umount_mutex); + if (freed >= nr) + break; + } + spin_unlock(&f2fs_list_lock); + return freed; +} + +void f2fs_join_shrinker(struct f2fs_sb_info *sbi) +{ + spin_lock(&f2fs_list_lock); + list_add_tail(&sbi->s_list, &f2fs_list); + spin_unlock(&f2fs_list_lock); +} + +void f2fs_leave_shrinker(struct f2fs_sb_info *sbi) +{ + f2fs_shrink_extent_tree(sbi, __count_extent_cache(sbi)); + + spin_lock(&f2fs_list_lock); + list_del(&sbi->s_list); + spin_unlock(&f2fs_list_lock); +} diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index a06b0b46fe6952..f79478115d379f 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -39,6 +39,13 @@ static struct proc_dir_entry *f2fs_proc_root; static struct kmem_cache *f2fs_inode_cachep; static struct kset *f2fs_kset; +/* f2fs-wide shrinker description */ +static struct shrinker f2fs_shrinker_info = { + .scan_objects = f2fs_shrink_scan, + .count_objects = f2fs_shrink_count, + .seeks = DEFAULT_SEEKS, +}; + enum { Opt_gc_background, Opt_disable_roll_forward, @@ -58,6 +65,7 @@ enum { Opt_nobarrier, Opt_fastboot, Opt_extent_cache, + Opt_noextent_cache, Opt_noinline_data, Opt_err, }; @@ -81,6 +89,7 @@ static match_table_t f2fs_tokens = { {Opt_nobarrier, "nobarrier"}, {Opt_fastboot, "fastboot"}, {Opt_extent_cache, "extent_cache"}, + {Opt_noextent_cache, "noextent_cache"}, {Opt_noinline_data, "noinline_data"}, {Opt_err, NULL}, }; @@ -382,6 +391,9 @@ static int parse_options(struct super_block *sb, char *options) case Opt_extent_cache: set_opt(sbi, EXTENT_CACHE); break; + case Opt_noextent_cache: + clear_opt(sbi, EXTENT_CACHE); + break; case Opt_noinline_data: clear_opt(sbi, INLINE_DATA); break; @@ -410,9 +422,7 @@ static struct inode *f2fs_alloc_inode(struct super_block *sb) atomic_set(&fi->dirty_pages, 0); fi->i_current_depth = 1; fi->i_advise = 0; - rwlock_init(&fi->ext_lock); init_rwsem(&fi->i_sem); - INIT_RADIX_TREE(&fi->inmem_root, GFP_NOFS); INIT_LIST_HEAD(&fi->inmem_pages); mutex_init(&fi->inmem_lock); @@ -441,17 +451,22 @@ static int f2fs_drop_inode(struct inode *inode) */ if (!inode_unhashed(inode) && inode->i_state & I_SYNC) { if (!inode->i_nlink && !is_bad_inode(inode)) { + /* to avoid evict_inode call simultaneously */ + atomic_inc(&inode->i_count); spin_unlock(&inode->i_lock); /* some remained atomic pages should discarded */ if (f2fs_is_atomic_file(inode)) commit_inmem_pages(inode, true); + /* should remain fi->extent_tree for writepage */ + f2fs_destroy_extent_node(inode); + sb_start_intwrite(inode->i_sb); i_size_write(inode, 0); if (F2FS_HAS_BLOCKS(inode)) - f2fs_truncate(inode); + f2fs_truncate(inode, true); sb_end_intwrite(inode->i_sb); @@ -461,6 +476,7 @@ static int f2fs_drop_inode(struct inode *inode) F2FS_I(inode)->i_crypt_info); #endif spin_lock(&inode->i_lock); + atomic_dec(&inode->i_count); } return 0; } @@ -498,9 +514,11 @@ static void f2fs_put_super(struct super_block *sb) } kobject_del(&sbi->s_kobj); - f2fs_destroy_stats(sbi); stop_gc_thread(sbi); + /* prevent remaining shrinker jobs */ + mutex_lock(&sbi->umount_mutex); + /* * We don't need to do checkpoint when superblock is clean. * But, the previous checkpoint was not done by umount, it needs to do @@ -514,6 +532,9 @@ static void f2fs_put_super(struct super_block *sb) write_checkpoint(sbi, &cpc); } + /* write_checkpoint can update stat informaion */ + f2fs_destroy_stats(sbi); + /* * normally superblock is clean, so we need to release this. * In addition, EIO will skip do checkpoint, we need this as well. @@ -521,6 +542,9 @@ static void f2fs_put_super(struct super_block *sb) release_dirty_inode(sbi); release_discard_addrs(sbi); + f2fs_leave_shrinker(sbi); + mutex_unlock(&sbi->umount_mutex); + iput(sbi->node_inode); iput(sbi->meta_inode); @@ -647,6 +671,8 @@ static int f2fs_show_options(struct seq_file *seq, struct dentry *root) seq_puts(seq, ",fastboot"); if (test_opt(sbi, EXTENT_CACHE)) seq_puts(seq, ",extent_cache"); + else + seq_puts(seq, ",noextent_cache"); seq_printf(seq, ",active_logs=%u", sbi->active_logs); return 0; @@ -667,7 +693,7 @@ static int segment_info_seq_show(struct seq_file *seq, void *offset) struct seg_entry *se = get_seg_entry(sbi, i); if ((i % 10) == 0) - seq_printf(seq, "%-5d", i); + seq_printf(seq, "%-10d", i); seq_printf(seq, "%d|%-3u", se->type, get_valid_blocks(sbi, i, 1)); if ((i % 10) == 9 || i == (total_segs - 1)) @@ -699,6 +725,7 @@ static void default_options(struct f2fs_sb_info *sbi) set_opt(sbi, BG_GC); set_opt(sbi, INLINE_DATA); + set_opt(sbi, EXTENT_CACHE); #ifdef CONFIG_F2FS_FS_XATTR set_opt(sbi, XATTR_USER); @@ -970,6 +997,9 @@ static void init_sb_info(struct f2fs_sb_info *sbi) sbi->dir_level = DEF_DIR_LEVEL; clear_sbi_flag(sbi, SBI_NEED_FSCK); + + INIT_LIST_HEAD(&sbi->s_list); + mutex_init(&sbi->umount_mutex); } /* @@ -1135,7 +1165,9 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) mutex_init(&sbi->writepages); mutex_init(&sbi->cp_mutex); init_rwsem(&sbi->node_write); - clear_sbi_flag(sbi, SBI_POR_DOING); + + /* disallow all the data/node/meta page writes */ + set_sbi_flag(sbi, SBI_POR_DOING); spin_lock_init(&sbi->stat_lock); init_rwsem(&sbi->read_io.io_rwsem); @@ -1212,8 +1244,12 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) goto free_nm; } + f2fs_join_shrinker(sbi); + /* if there are nt orphan nodes free them */ - recover_orphan_inodes(sbi); + err = recover_orphan_inodes(sbi); + if (err) + goto free_node_inode; /* read root inode and dentry */ root = f2fs_iget(sb, F2FS_ROOT_INO(sbi)); @@ -1275,6 +1311,8 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) goto free_kobj; } } + /* recover_fsync_data() cleared this already */ + clear_sbi_flag(sbi, SBI_POR_DOING); /* * If filesystem is not mounted as read-only then @@ -1308,7 +1346,10 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) dput(sb->s_root); sb->s_root = NULL; free_node_inode: + mutex_lock(&sbi->umount_mutex); + f2fs_leave_shrinker(sbi); iput(sbi->node_inode); + mutex_unlock(&sbi->umount_mutex); free_nm: destroy_node_manager(sbi); free_sm: @@ -1404,13 +1445,20 @@ static int __init init_f2fs_fs(void) err = f2fs_init_crypto(); if (err) goto free_kset; - err = register_filesystem(&f2fs_fs_type); + + err = register_shrinker(&f2fs_shrinker_info); if (err) goto free_crypto; + + err = register_filesystem(&f2fs_fs_type); + if (err) + goto free_shrinker; f2fs_create_root_stats(); f2fs_proc_root = proc_mkdir("fs/f2fs", NULL); return 0; +free_shrinker: + unregister_shrinker(&f2fs_shrinker_info); free_crypto: f2fs_exit_crypto(); free_kset: @@ -1433,6 +1481,7 @@ static void __exit exit_f2fs_fs(void) { remove_proc_entry("fs/f2fs", NULL); f2fs_destroy_root_stats(); + unregister_shrinker(&f2fs_shrinker_info); unregister_filesystem(&f2fs_fs_type); f2fs_exit_crypto(); destroy_extent_cache(); diff --git a/fs/f2fs/xattr.c b/fs/f2fs/xattr.c index 07449b980acb9a..4de2286c0e4df6 100644 --- a/fs/f2fs/xattr.c +++ b/fs/f2fs/xattr.c @@ -499,9 +499,12 @@ static int __f2fs_setxattr(struct inode *inode, int index, len = strlen(name); - if (len > F2FS_NAME_LEN || size > MAX_VALUE_LEN(inode)) + if (len > F2FS_NAME_LEN) return -ERANGE; + if (size > MAX_VALUE_LEN(inode)) + return -E2BIG; + base_addr = read_all_xattrs(inode, ipage); if (!base_addr) goto exit; diff --git a/fs/hpfs/buffer.c b/fs/hpfs/buffer.c index 8057fe4e657494..f626114449e4a7 100644 --- a/fs/hpfs/buffer.c +++ b/fs/hpfs/buffer.c @@ -10,6 +10,30 @@ #include #include "hpfs_fn.h" +secno hpfs_search_hotfix_map(struct super_block *s, secno sec) +{ + unsigned i; + struct hpfs_sb_info *sbi = hpfs_sb(s); + for (i = 0; unlikely(i < sbi->n_hotfixes); i++) { + if (sbi->hotfix_from[i] == sec) { + return sbi->hotfix_to[i]; + } + } + return sec; +} + +unsigned hpfs_search_hotfix_map_for_range(struct super_block *s, secno sec, unsigned n) +{ + unsigned i; + struct hpfs_sb_info *sbi = hpfs_sb(s); + for (i = 0; unlikely(i < sbi->n_hotfixes); i++) { + if (sbi->hotfix_from[i] >= sec && sbi->hotfix_from[i] < sec + n) { + n = sbi->hotfix_from[i] - sec; + } + } + return n; +} + void hpfs_prefetch_sectors(struct super_block *s, unsigned secno, int n) { struct buffer_head *bh; @@ -18,6 +42,9 @@ void hpfs_prefetch_sectors(struct super_block *s, unsigned secno, int n) if (n <= 0 || unlikely(secno >= hpfs_sb(s)->sb_fs_size)) return; + if (unlikely(hpfs_search_hotfix_map_for_range(s, secno, n) != n)) + return; + bh = sb_find_get_block(s, secno); if (bh) { if (buffer_uptodate(bh)) { @@ -51,7 +78,7 @@ void *hpfs_map_sector(struct super_block *s, unsigned secno, struct buffer_head cond_resched(); - *bhp = bh = sb_bread(s, secno); + *bhp = bh = sb_bread(s, hpfs_search_hotfix_map(s, secno)); if (bh != NULL) return bh->b_data; else { @@ -71,7 +98,7 @@ void *hpfs_get_sector(struct super_block *s, unsigned secno, struct buffer_head cond_resched(); - if ((*bhp = bh = sb_getblk(s, secno)) != NULL) { + if ((*bhp = bh = sb_getblk(s, hpfs_search_hotfix_map(s, secno))) != NULL) { if (!buffer_uptodate(bh)) wait_on_buffer(bh); set_buffer_uptodate(bh); return bh->b_data; @@ -99,10 +126,10 @@ void *hpfs_map_4sectors(struct super_block *s, unsigned secno, struct quad_buffe hpfs_prefetch_sectors(s, secno, 4 + ahead); - if (!(qbh->bh[0] = sb_bread(s, secno + 0))) goto bail0; - if (!(qbh->bh[1] = sb_bread(s, secno + 1))) goto bail1; - if (!(qbh->bh[2] = sb_bread(s, secno + 2))) goto bail2; - if (!(qbh->bh[3] = sb_bread(s, secno + 3))) goto bail3; + if (!hpfs_map_sector(s, secno + 0, &qbh->bh[0], 0)) goto bail0; + if (!hpfs_map_sector(s, secno + 1, &qbh->bh[1], 0)) goto bail1; + if (!hpfs_map_sector(s, secno + 2, &qbh->bh[2], 0)) goto bail2; + if (!hpfs_map_sector(s, secno + 3, &qbh->bh[3], 0)) goto bail3; if (likely(qbh->bh[1]->b_data == qbh->bh[0]->b_data + 1 * 512) && likely(qbh->bh[2]->b_data == qbh->bh[0]->b_data + 2 * 512) && diff --git a/fs/hpfs/file.c b/fs/hpfs/file.c index 7ca28d604bf767..d3bcdd97570057 100644 --- a/fs/hpfs/file.c +++ b/fs/hpfs/file.c @@ -83,6 +83,11 @@ static int hpfs_get_block(struct inode *inode, sector_t iblock, struct buffer_he if (s) { if (bh_result->b_size >> 9 < n_secs) n_secs = bh_result->b_size >> 9; + n_secs = hpfs_search_hotfix_map_for_range(inode->i_sb, s, n_secs); + if (unlikely(!n_secs)) { + s = hpfs_search_hotfix_map(inode->i_sb, s); + n_secs = 1; + } map_bh(bh_result, inode->i_sb, s); bh_result->b_size = n_secs << 9; goto ret_0; @@ -101,7 +106,7 @@ static int hpfs_get_block(struct inode *inode, sector_t iblock, struct buffer_he inode->i_blocks++; hpfs_i(inode)->mmu_private += 512; set_buffer_new(bh_result); - map_bh(bh_result, inode->i_sb, s); + map_bh(bh_result, inode->i_sb, hpfs_search_hotfix_map(inode->i_sb, s)); ret_0: r = 0; ret_r: @@ -181,7 +186,7 @@ static int hpfs_write_end(struct file *file, struct address_space *mapping, static sector_t _hpfs_bmap(struct address_space *mapping, sector_t block) { - return generic_block_bmap(mapping,block,hpfs_get_block); + return generic_block_bmap(mapping, block, hpfs_get_block); } const struct address_space_operations hpfs_aops = { diff --git a/fs/hpfs/hpfs_fn.h b/fs/hpfs/hpfs_fn.h index c4867b5116ddf7..975654a63c13d8 100644 --- a/fs/hpfs/hpfs_fn.h +++ b/fs/hpfs/hpfs_fn.h @@ -88,6 +88,10 @@ struct hpfs_sb_info { unsigned sb_max_fwd_alloc; /* max forwad allocation */ int sb_timeshift; struct rcu_head rcu; + + unsigned n_hotfixes; + secno hotfix_from[256]; + secno hotfix_to[256]; }; /* Four 512-byte buffers and the 2k block obtained by concatenating them */ @@ -217,6 +221,8 @@ void hpfs_remove_fnode(struct super_block *, fnode_secno fno); /* buffer.c */ +secno hpfs_search_hotfix_map(struct super_block *s, secno sec); +unsigned hpfs_search_hotfix_map_for_range(struct super_block *s, secno sec, unsigned n); void hpfs_prefetch_sectors(struct super_block *, unsigned, int); void *hpfs_map_sector(struct super_block *, unsigned, struct buffer_head **, int); void *hpfs_get_sector(struct super_block *, unsigned, struct buffer_head **); @@ -285,6 +291,7 @@ __le32 *hpfs_map_bitmap(struct super_block *, unsigned, struct quad_buffer_head void hpfs_prefetch_bitmap(struct super_block *, unsigned); unsigned char *hpfs_load_code_page(struct super_block *, secno); __le32 *hpfs_load_bitmap_directory(struct super_block *, secno bmp); +void hpfs_load_hotfix_map(struct super_block *s, struct hpfs_spare_block *spareblock); struct fnode *hpfs_map_fnode(struct super_block *s, ino_t, struct buffer_head **); struct anode *hpfs_map_anode(struct super_block *s, anode_secno, struct buffer_head **); struct dnode *hpfs_map_dnode(struct super_block *s, dnode_secno, struct quad_buffer_head *); diff --git a/fs/hpfs/map.c b/fs/hpfs/map.c index 442770edcdc704..a69bbc1e87f868 100644 --- a/fs/hpfs/map.c +++ b/fs/hpfs/map.c @@ -130,6 +130,32 @@ __le32 *hpfs_load_bitmap_directory(struct super_block *s, secno bmp) return b; } +void hpfs_load_hotfix_map(struct super_block *s, struct hpfs_spare_block *spareblock) +{ + struct quad_buffer_head qbh; + u32 *directory; + u32 n_hotfixes, n_used_hotfixes; + unsigned i; + + n_hotfixes = le32_to_cpu(spareblock->n_spares); + n_used_hotfixes = le32_to_cpu(spareblock->n_spares_used); + + if (n_hotfixes > 256 || n_used_hotfixes > n_hotfixes) { + hpfs_error(s, "invalid number of hotfixes: %u, used: %u", n_hotfixes, n_used_hotfixes); + return; + } + if (!(directory = hpfs_map_4sectors(s, le32_to_cpu(spareblock->hotfix_map), &qbh, 0))) { + hpfs_error(s, "can't load hotfix map"); + return; + } + for (i = 0; i < n_used_hotfixes; i++) { + hpfs_sb(s)->hotfix_from[i] = le32_to_cpu(directory[i]); + hpfs_sb(s)->hotfix_to[i] = le32_to_cpu(directory[n_hotfixes + i]); + } + hpfs_sb(s)->n_hotfixes = n_used_hotfixes; + hpfs_brelse4(&qbh); +} + /* * Load fnode to memory */ diff --git a/fs/hpfs/namei.c b/fs/hpfs/namei.c index a0872f239f04ff..9e92c9c2d31954 100644 --- a/fs/hpfs/namei.c +++ b/fs/hpfs/namei.c @@ -8,6 +8,17 @@ #include #include "hpfs_fn.h" +static void hpfs_update_directory_times(struct inode *dir) +{ + time_t t = get_seconds(); + if (t == dir->i_mtime.tv_sec && + t == dir->i_ctime.tv_sec) + return; + dir->i_mtime.tv_sec = dir->i_ctime.tv_sec = t; + dir->i_mtime.tv_nsec = dir->i_ctime.tv_nsec = 0; + hpfs_write_inode_nolock(dir); +} + static int hpfs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode) { const unsigned char *name = dentry->d_name.name; @@ -99,6 +110,7 @@ static int hpfs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode) result->i_mode = mode | S_IFDIR; hpfs_write_inode_nolock(result); } + hpfs_update_directory_times(dir); d_instantiate(dentry, result); hpfs_unlock(dir->i_sb); return 0; @@ -187,6 +199,7 @@ static int hpfs_create(struct inode *dir, struct dentry *dentry, umode_t mode, b result->i_mode = mode | S_IFREG; hpfs_write_inode_nolock(result); } + hpfs_update_directory_times(dir); d_instantiate(dentry, result); hpfs_unlock(dir->i_sb); return 0; @@ -262,6 +275,7 @@ static int hpfs_mknod(struct inode *dir, struct dentry *dentry, umode_t mode, de insert_inode_hash(result); hpfs_write_inode_nolock(result); + hpfs_update_directory_times(dir); d_instantiate(dentry, result); brelse(bh); hpfs_unlock(dir->i_sb); @@ -340,6 +354,7 @@ static int hpfs_symlink(struct inode *dir, struct dentry *dentry, const char *sy insert_inode_hash(result); hpfs_write_inode_nolock(result); + hpfs_update_directory_times(dir); d_instantiate(dentry, result); hpfs_unlock(dir->i_sb); return 0; @@ -423,6 +438,8 @@ static int hpfs_unlink(struct inode *dir, struct dentry *dentry) out1: hpfs_brelse4(&qbh); out: + if (!err) + hpfs_update_directory_times(dir); hpfs_unlock(dir->i_sb); return err; } @@ -477,6 +494,8 @@ static int hpfs_rmdir(struct inode *dir, struct dentry *dentry) out1: hpfs_brelse4(&qbh); out: + if (!err) + hpfs_update_directory_times(dir); hpfs_unlock(dir->i_sb); return err; } @@ -595,7 +614,7 @@ static int hpfs_rename(struct inode *old_dir, struct dentry *old_dentry, goto end1; } - end: +end: hpfs_i(i)->i_parent_dir = new_dir->i_ino; if (S_ISDIR(i->i_mode)) { inc_nlink(new_dir); @@ -610,6 +629,10 @@ static int hpfs_rename(struct inode *old_dir, struct dentry *old_dentry, brelse(bh); } end1: + if (!err) { + hpfs_update_directory_times(old_dir); + hpfs_update_directory_times(new_dir); + } hpfs_unlock(i->i_sb); return err; } diff --git a/fs/hpfs/super.c b/fs/hpfs/super.c index 68a9bed056284c..a561591896bd34 100644 --- a/fs/hpfs/super.c +++ b/fs/hpfs/super.c @@ -628,6 +628,9 @@ static int hpfs_fill_super(struct super_block *s, void *options, int silent) goto bail4; } + if (spareblock->n_spares_used) + hpfs_load_hotfix_map(s, spareblock); + /* Load bitmap directory */ if (!(sbi->sb_bmp_dir = hpfs_load_bitmap_directory(s, le32_to_cpu(superblock->bitmaps)))) goto bail4; @@ -647,18 +650,6 @@ static int hpfs_fill_super(struct super_block *s, void *options, int silent) mark_buffer_dirty(bh2); } - if (spareblock->hotfixes_used || spareblock->n_spares_used) { - if (errs >= 2) { - pr_err("Hotfixes not supported here, try chkdsk\n"); - mark_dirty(s, 0); - goto bail4; - } - hpfs_error(s, "hotfixes not supported here, try chkdsk"); - if (errs == 0) - pr_err("Proceeding, but your filesystem will be probably corrupted by this driver...\n"); - else - pr_err("This driver may read bad files or crash when operating on disk with hotfixes.\n"); - } if (le32_to_cpu(spareblock->n_dnode_spares) != le32_to_cpu(spareblock->n_dnode_spares_free)) { if (errs >= 2) { pr_err("Spare dnodes used, try chkdsk\n"); diff --git a/fs/jbd/Kconfig b/fs/jbd/Kconfig deleted file mode 100644 index 4e28beeed15723..00000000000000 --- a/fs/jbd/Kconfig +++ /dev/null @@ -1,30 +0,0 @@ -config JBD - tristate - help - This is a generic journalling layer for block devices. It is - currently used by the ext3 file system, but it could also be - used to add journal support to other file systems or block - devices such as RAID or LVM. - - If you are using the ext3 file system, you need to say Y here. - If you are not using ext3 then you will probably want to say N. - - To compile this device as a module, choose M here: the module will be - called jbd. If you are compiling ext3 into the kernel, you - cannot compile this code as a module. - -config JBD_DEBUG - bool "JBD (ext3) debugging support" - depends on JBD && DEBUG_FS - help - If you are using the ext3 journaled file system (or potentially any - other file system/device using JBD), this option allows you to - enable debugging output while the system is running, in order to - help track down any problems you are having. By default the - debugging output will be turned off. - - If you select Y here, then you will be able to turn on debugging - with "echo N > /sys/kernel/debug/jbd/jbd-debug", where N is a - number between 1 and 5, the higher the number, the more debugging - output is generated. To turn debugging off again, do - "echo 0 > /sys/kernel/debug/jbd/jbd-debug". diff --git a/fs/jbd/Makefile b/fs/jbd/Makefile deleted file mode 100644 index 54aca4868a3632..00000000000000 --- a/fs/jbd/Makefile +++ /dev/null @@ -1,7 +0,0 @@ -# -# Makefile for the linux journaling routines. -# - -obj-$(CONFIG_JBD) += jbd.o - -jbd-objs := transaction.o commit.o recovery.o checkpoint.o revoke.o journal.o diff --git a/fs/jbd/checkpoint.c b/fs/jbd/checkpoint.c deleted file mode 100644 index 08c03044abddbc..00000000000000 --- a/fs/jbd/checkpoint.c +++ /dev/null @@ -1,782 +0,0 @@ -/* - * linux/fs/jbd/checkpoint.c - * - * Written by Stephen C. Tweedie , 1999 - * - * Copyright 1999 Red Hat Software --- All Rights Reserved - * - * This file is part of the Linux kernel and is made available under - * the terms of the GNU General Public License, version 2, or at your - * option, any later version, incorporated herein by reference. - * - * Checkpoint routines for the generic filesystem journaling code. - * Part of the ext2fs journaling system. - * - * Checkpointing is the process of ensuring that a section of the log is - * committed fully to disk, so that that portion of the log can be - * reused. - */ - -#include -#include -#include -#include -#include -#include -#include - -/* - * Unlink a buffer from a transaction checkpoint list. - * - * Called with j_list_lock held. - */ -static inline void __buffer_unlink_first(struct journal_head *jh) -{ - transaction_t *transaction = jh->b_cp_transaction; - - jh->b_cpnext->b_cpprev = jh->b_cpprev; - jh->b_cpprev->b_cpnext = jh->b_cpnext; - if (transaction->t_checkpoint_list == jh) { - transaction->t_checkpoint_list = jh->b_cpnext; - if (transaction->t_checkpoint_list == jh) - transaction->t_checkpoint_list = NULL; - } -} - -/* - * Unlink a buffer from a transaction checkpoint(io) list. - * - * Called with j_list_lock held. - */ -static inline void __buffer_unlink(struct journal_head *jh) -{ - transaction_t *transaction = jh->b_cp_transaction; - - __buffer_unlink_first(jh); - if (transaction->t_checkpoint_io_list == jh) { - transaction->t_checkpoint_io_list = jh->b_cpnext; - if (transaction->t_checkpoint_io_list == jh) - transaction->t_checkpoint_io_list = NULL; - } -} - -/* - * Move a buffer from the checkpoint list to the checkpoint io list - * - * Called with j_list_lock held - */ -static inline void __buffer_relink_io(struct journal_head *jh) -{ - transaction_t *transaction = jh->b_cp_transaction; - - __buffer_unlink_first(jh); - - if (!transaction->t_checkpoint_io_list) { - jh->b_cpnext = jh->b_cpprev = jh; - } else { - jh->b_cpnext = transaction->t_checkpoint_io_list; - jh->b_cpprev = transaction->t_checkpoint_io_list->b_cpprev; - jh->b_cpprev->b_cpnext = jh; - jh->b_cpnext->b_cpprev = jh; - } - transaction->t_checkpoint_io_list = jh; -} - -/* - * Try to release a checkpointed buffer from its transaction. - * Returns 1 if we released it and 2 if we also released the - * whole transaction. - * - * Requires j_list_lock - * Called under jbd_lock_bh_state(jh2bh(jh)), and drops it - */ -static int __try_to_free_cp_buf(struct journal_head *jh) -{ - int ret = 0; - struct buffer_head *bh = jh2bh(jh); - - if (jh->b_jlist == BJ_None && !buffer_locked(bh) && - !buffer_dirty(bh) && !buffer_write_io_error(bh)) { - /* - * Get our reference so that bh cannot be freed before - * we unlock it - */ - get_bh(bh); - JBUFFER_TRACE(jh, "remove from checkpoint list"); - ret = __journal_remove_checkpoint(jh) + 1; - jbd_unlock_bh_state(bh); - BUFFER_TRACE(bh, "release"); - __brelse(bh); - } else { - jbd_unlock_bh_state(bh); - } - return ret; -} - -/* - * __log_wait_for_space: wait until there is space in the journal. - * - * Called under j-state_lock *only*. It will be unlocked if we have to wait - * for a checkpoint to free up some space in the log. - */ -void __log_wait_for_space(journal_t *journal) -{ - int nblocks, space_left; - assert_spin_locked(&journal->j_state_lock); - - nblocks = jbd_space_needed(journal); - while (__log_space_left(journal) < nblocks) { - if (journal->j_flags & JFS_ABORT) - return; - spin_unlock(&journal->j_state_lock); - mutex_lock(&journal->j_checkpoint_mutex); - - /* - * Test again, another process may have checkpointed while we - * were waiting for the checkpoint lock. If there are no - * transactions ready to be checkpointed, try to recover - * journal space by calling cleanup_journal_tail(), and if - * that doesn't work, by waiting for the currently committing - * transaction to complete. If there is absolutely no way - * to make progress, this is either a BUG or corrupted - * filesystem, so abort the journal and leave a stack - * trace for forensic evidence. - */ - spin_lock(&journal->j_state_lock); - spin_lock(&journal->j_list_lock); - nblocks = jbd_space_needed(journal); - space_left = __log_space_left(journal); - if (space_left < nblocks) { - int chkpt = journal->j_checkpoint_transactions != NULL; - tid_t tid = 0; - - if (journal->j_committing_transaction) - tid = journal->j_committing_transaction->t_tid; - spin_unlock(&journal->j_list_lock); - spin_unlock(&journal->j_state_lock); - if (chkpt) { - log_do_checkpoint(journal); - } else if (cleanup_journal_tail(journal) == 0) { - /* We were able to recover space; yay! */ - ; - } else if (tid) { - log_wait_commit(journal, tid); - } else { - printk(KERN_ERR "%s: needed %d blocks and " - "only had %d space available\n", - __func__, nblocks, space_left); - printk(KERN_ERR "%s: no way to get more " - "journal space\n", __func__); - WARN_ON(1); - journal_abort(journal, 0); - } - spin_lock(&journal->j_state_lock); - } else { - spin_unlock(&journal->j_list_lock); - } - mutex_unlock(&journal->j_checkpoint_mutex); - } -} - -/* - * We were unable to perform jbd_trylock_bh_state() inside j_list_lock. - * The caller must restart a list walk. Wait for someone else to run - * jbd_unlock_bh_state(). - */ -static void jbd_sync_bh(journal_t *journal, struct buffer_head *bh) - __releases(journal->j_list_lock) -{ - get_bh(bh); - spin_unlock(&journal->j_list_lock); - jbd_lock_bh_state(bh); - jbd_unlock_bh_state(bh); - put_bh(bh); -} - -/* - * Clean up transaction's list of buffers submitted for io. - * We wait for any pending IO to complete and remove any clean - * buffers. Note that we take the buffers in the opposite ordering - * from the one in which they were submitted for IO. - * - * Return 0 on success, and return <0 if some buffers have failed - * to be written out. - * - * Called with j_list_lock held. - */ -static int __wait_cp_io(journal_t *journal, transaction_t *transaction) -{ - struct journal_head *jh; - struct buffer_head *bh; - tid_t this_tid; - int released = 0; - int ret = 0; - - this_tid = transaction->t_tid; -restart: - /* Did somebody clean up the transaction in the meanwhile? */ - if (journal->j_checkpoint_transactions != transaction || - transaction->t_tid != this_tid) - return ret; - while (!released && transaction->t_checkpoint_io_list) { - jh = transaction->t_checkpoint_io_list; - bh = jh2bh(jh); - if (!jbd_trylock_bh_state(bh)) { - jbd_sync_bh(journal, bh); - spin_lock(&journal->j_list_lock); - goto restart; - } - get_bh(bh); - if (buffer_locked(bh)) { - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - wait_on_buffer(bh); - /* the journal_head may have gone by now */ - BUFFER_TRACE(bh, "brelse"); - __brelse(bh); - spin_lock(&journal->j_list_lock); - goto restart; - } - if (unlikely(buffer_write_io_error(bh))) - ret = -EIO; - - /* - * Now in whatever state the buffer currently is, we know that - * it has been written out and so we can drop it from the list - */ - released = __journal_remove_checkpoint(jh); - jbd_unlock_bh_state(bh); - __brelse(bh); - } - - return ret; -} - -#define NR_BATCH 64 - -static void -__flush_batch(journal_t *journal, struct buffer_head **bhs, int *batch_count) -{ - int i; - struct blk_plug plug; - - blk_start_plug(&plug); - for (i = 0; i < *batch_count; i++) - write_dirty_buffer(bhs[i], WRITE_SYNC); - blk_finish_plug(&plug); - - for (i = 0; i < *batch_count; i++) { - struct buffer_head *bh = bhs[i]; - clear_buffer_jwrite(bh); - BUFFER_TRACE(bh, "brelse"); - __brelse(bh); - } - *batch_count = 0; -} - -/* - * Try to flush one buffer from the checkpoint list to disk. - * - * Return 1 if something happened which requires us to abort the current - * scan of the checkpoint list. Return <0 if the buffer has failed to - * be written out. - * - * Called with j_list_lock held and drops it if 1 is returned - * Called under jbd_lock_bh_state(jh2bh(jh)), and drops it - */ -static int __process_buffer(journal_t *journal, struct journal_head *jh, - struct buffer_head **bhs, int *batch_count) -{ - struct buffer_head *bh = jh2bh(jh); - int ret = 0; - - if (buffer_locked(bh)) { - get_bh(bh); - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - wait_on_buffer(bh); - /* the journal_head may have gone by now */ - BUFFER_TRACE(bh, "brelse"); - __brelse(bh); - ret = 1; - } else if (jh->b_transaction != NULL) { - transaction_t *t = jh->b_transaction; - tid_t tid = t->t_tid; - - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - log_start_commit(journal, tid); - log_wait_commit(journal, tid); - ret = 1; - } else if (!buffer_dirty(bh)) { - ret = 1; - if (unlikely(buffer_write_io_error(bh))) - ret = -EIO; - get_bh(bh); - J_ASSERT_JH(jh, !buffer_jbddirty(bh)); - BUFFER_TRACE(bh, "remove from checkpoint"); - __journal_remove_checkpoint(jh); - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - __brelse(bh); - } else { - /* - * Important: we are about to write the buffer, and - * possibly block, while still holding the journal lock. - * We cannot afford to let the transaction logic start - * messing around with this buffer before we write it to - * disk, as that would break recoverability. - */ - BUFFER_TRACE(bh, "queue"); - get_bh(bh); - J_ASSERT_BH(bh, !buffer_jwrite(bh)); - set_buffer_jwrite(bh); - bhs[*batch_count] = bh; - __buffer_relink_io(jh); - jbd_unlock_bh_state(bh); - (*batch_count)++; - if (*batch_count == NR_BATCH) { - spin_unlock(&journal->j_list_lock); - __flush_batch(journal, bhs, batch_count); - ret = 1; - } - } - return ret; -} - -/* - * Perform an actual checkpoint. We take the first transaction on the - * list of transactions to be checkpointed and send all its buffers - * to disk. We submit larger chunks of data at once. - * - * The journal should be locked before calling this function. - * Called with j_checkpoint_mutex held. - */ -int log_do_checkpoint(journal_t *journal) -{ - transaction_t *transaction; - tid_t this_tid; - int result; - - jbd_debug(1, "Start checkpoint\n"); - - /* - * First thing: if there are any transactions in the log which - * don't need checkpointing, just eliminate them from the - * journal straight away. - */ - result = cleanup_journal_tail(journal); - trace_jbd_checkpoint(journal, result); - jbd_debug(1, "cleanup_journal_tail returned %d\n", result); - if (result <= 0) - return result; - - /* - * OK, we need to start writing disk blocks. Take one transaction - * and write it. - */ - result = 0; - spin_lock(&journal->j_list_lock); - if (!journal->j_checkpoint_transactions) - goto out; - transaction = journal->j_checkpoint_transactions; - this_tid = transaction->t_tid; -restart: - /* - * If someone cleaned up this transaction while we slept, we're - * done (maybe it's a new transaction, but it fell at the same - * address). - */ - if (journal->j_checkpoint_transactions == transaction && - transaction->t_tid == this_tid) { - int batch_count = 0; - struct buffer_head *bhs[NR_BATCH]; - struct journal_head *jh; - int retry = 0, err; - - while (!retry && transaction->t_checkpoint_list) { - struct buffer_head *bh; - - jh = transaction->t_checkpoint_list; - bh = jh2bh(jh); - if (!jbd_trylock_bh_state(bh)) { - jbd_sync_bh(journal, bh); - retry = 1; - break; - } - retry = __process_buffer(journal, jh, bhs,&batch_count); - if (retry < 0 && !result) - result = retry; - if (!retry && (need_resched() || - spin_needbreak(&journal->j_list_lock))) { - spin_unlock(&journal->j_list_lock); - retry = 1; - break; - } - } - - if (batch_count) { - if (!retry) { - spin_unlock(&journal->j_list_lock); - retry = 1; - } - __flush_batch(journal, bhs, &batch_count); - } - - if (retry) { - spin_lock(&journal->j_list_lock); - goto restart; - } - /* - * Now we have cleaned up the first transaction's checkpoint - * list. Let's clean up the second one - */ - err = __wait_cp_io(journal, transaction); - if (!result) - result = err; - } -out: - spin_unlock(&journal->j_list_lock); - if (result < 0) - journal_abort(journal, result); - else - result = cleanup_journal_tail(journal); - - return (result < 0) ? result : 0; -} - -/* - * Check the list of checkpoint transactions for the journal to see if - * we have already got rid of any since the last update of the log tail - * in the journal superblock. If so, we can instantly roll the - * superblock forward to remove those transactions from the log. - * - * Return <0 on error, 0 on success, 1 if there was nothing to clean up. - * - * This is the only part of the journaling code which really needs to be - * aware of transaction aborts. Checkpointing involves writing to the - * main filesystem area rather than to the journal, so it can proceed - * even in abort state, but we must not update the super block if - * checkpointing may have failed. Otherwise, we would lose some metadata - * buffers which should be written-back to the filesystem. - */ - -int cleanup_journal_tail(journal_t *journal) -{ - transaction_t * transaction; - tid_t first_tid; - unsigned int blocknr, freed; - - if (is_journal_aborted(journal)) - return 1; - - /* - * OK, work out the oldest transaction remaining in the log, and - * the log block it starts at. - * - * If the log is now empty, we need to work out which is the - * next transaction ID we will write, and where it will - * start. - */ - spin_lock(&journal->j_state_lock); - spin_lock(&journal->j_list_lock); - transaction = journal->j_checkpoint_transactions; - if (transaction) { - first_tid = transaction->t_tid; - blocknr = transaction->t_log_start; - } else if ((transaction = journal->j_committing_transaction) != NULL) { - first_tid = transaction->t_tid; - blocknr = transaction->t_log_start; - } else if ((transaction = journal->j_running_transaction) != NULL) { - first_tid = transaction->t_tid; - blocknr = journal->j_head; - } else { - first_tid = journal->j_transaction_sequence; - blocknr = journal->j_head; - } - spin_unlock(&journal->j_list_lock); - J_ASSERT(blocknr != 0); - - /* If the oldest pinned transaction is at the tail of the log - already then there's not much we can do right now. */ - if (journal->j_tail_sequence == first_tid) { - spin_unlock(&journal->j_state_lock); - return 1; - } - spin_unlock(&journal->j_state_lock); - - /* - * We need to make sure that any blocks that were recently written out - * --- perhaps by log_do_checkpoint() --- are flushed out before we - * drop the transactions from the journal. Similarly we need to be sure - * superblock makes it to disk before next transaction starts reusing - * freed space (otherwise we could replay some blocks of the new - * transaction thinking they belong to the old one). So we use - * WRITE_FLUSH_FUA. It's unlikely this will be necessary, especially - * with an appropriately sized journal, but we need this to guarantee - * correctness. Fortunately cleanup_journal_tail() doesn't get called - * all that often. - */ - journal_update_sb_log_tail(journal, first_tid, blocknr, - WRITE_FLUSH_FUA); - - spin_lock(&journal->j_state_lock); - /* OK, update the superblock to recover the freed space. - * Physical blocks come first: have we wrapped beyond the end of - * the log? */ - freed = blocknr - journal->j_tail; - if (blocknr < journal->j_tail) - freed = freed + journal->j_last - journal->j_first; - - trace_jbd_cleanup_journal_tail(journal, first_tid, blocknr, freed); - jbd_debug(1, - "Cleaning journal tail from %d to %d (offset %u), " - "freeing %u\n", - journal->j_tail_sequence, first_tid, blocknr, freed); - - journal->j_free += freed; - journal->j_tail_sequence = first_tid; - journal->j_tail = blocknr; - spin_unlock(&journal->j_state_lock); - return 0; -} - - -/* Checkpoint list management */ - -/* - * journal_clean_one_cp_list - * - * Find all the written-back checkpoint buffers in the given list and release - * them. - * - * Called with j_list_lock held. - * Returns number of buffers reaped (for debug) - */ - -static int journal_clean_one_cp_list(struct journal_head *jh, int *released) -{ - struct journal_head *last_jh; - struct journal_head *next_jh = jh; - int ret, freed = 0; - - *released = 0; - if (!jh) - return 0; - - last_jh = jh->b_cpprev; - do { - jh = next_jh; - next_jh = jh->b_cpnext; - /* Use trylock because of the ranking */ - if (jbd_trylock_bh_state(jh2bh(jh))) { - ret = __try_to_free_cp_buf(jh); - if (ret) { - freed++; - if (ret == 2) { - *released = 1; - return freed; - } - } - } - /* - * This function only frees up some memory - * if possible so we dont have an obligation - * to finish processing. Bail out if preemption - * requested: - */ - if (need_resched()) - return freed; - } while (jh != last_jh); - - return freed; -} - -/* - * journal_clean_checkpoint_list - * - * Find all the written-back checkpoint buffers in the journal and release them. - * - * Called with the journal locked. - * Called with j_list_lock held. - * Returns number of buffers reaped (for debug) - */ - -int __journal_clean_checkpoint_list(journal_t *journal) -{ - transaction_t *transaction, *last_transaction, *next_transaction; - int ret = 0; - int released; - - transaction = journal->j_checkpoint_transactions; - if (!transaction) - goto out; - - last_transaction = transaction->t_cpprev; - next_transaction = transaction; - do { - transaction = next_transaction; - next_transaction = transaction->t_cpnext; - ret += journal_clean_one_cp_list(transaction-> - t_checkpoint_list, &released); - /* - * This function only frees up some memory if possible so we - * dont have an obligation to finish processing. Bail out if - * preemption requested: - */ - if (need_resched()) - goto out; - if (released) - continue; - /* - * It is essential that we are as careful as in the case of - * t_checkpoint_list with removing the buffer from the list as - * we can possibly see not yet submitted buffers on io_list - */ - ret += journal_clean_one_cp_list(transaction-> - t_checkpoint_io_list, &released); - if (need_resched()) - goto out; - } while (transaction != last_transaction); -out: - return ret; -} - -/* - * journal_remove_checkpoint: called after a buffer has been committed - * to disk (either by being write-back flushed to disk, or being - * committed to the log). - * - * We cannot safely clean a transaction out of the log until all of the - * buffer updates committed in that transaction have safely been stored - * elsewhere on disk. To achieve this, all of the buffers in a - * transaction need to be maintained on the transaction's checkpoint - * lists until they have been rewritten, at which point this function is - * called to remove the buffer from the existing transaction's - * checkpoint lists. - * - * The function returns 1 if it frees the transaction, 0 otherwise. - * The function can free jh and bh. - * - * This function is called with j_list_lock held. - * This function is called with jbd_lock_bh_state(jh2bh(jh)) - */ - -int __journal_remove_checkpoint(struct journal_head *jh) -{ - transaction_t *transaction; - journal_t *journal; - int ret = 0; - - JBUFFER_TRACE(jh, "entry"); - - if ((transaction = jh->b_cp_transaction) == NULL) { - JBUFFER_TRACE(jh, "not on transaction"); - goto out; - } - journal = transaction->t_journal; - - JBUFFER_TRACE(jh, "removing from transaction"); - __buffer_unlink(jh); - jh->b_cp_transaction = NULL; - journal_put_journal_head(jh); - - if (transaction->t_checkpoint_list != NULL || - transaction->t_checkpoint_io_list != NULL) - goto out; - - /* - * There is one special case to worry about: if we have just pulled the - * buffer off a running or committing transaction's checkpoing list, - * then even if the checkpoint list is empty, the transaction obviously - * cannot be dropped! - * - * The locking here around t_state is a bit sleazy. - * See the comment at the end of journal_commit_transaction(). - */ - if (transaction->t_state != T_FINISHED) - goto out; - - /* OK, that was the last buffer for the transaction: we can now - safely remove this transaction from the log */ - - __journal_drop_transaction(journal, transaction); - - /* Just in case anybody was waiting for more transactions to be - checkpointed... */ - wake_up(&journal->j_wait_logspace); - ret = 1; -out: - return ret; -} - -/* - * journal_insert_checkpoint: put a committed buffer onto a checkpoint - * list so that we know when it is safe to clean the transaction out of - * the log. - * - * Called with the journal locked. - * Called with j_list_lock held. - */ -void __journal_insert_checkpoint(struct journal_head *jh, - transaction_t *transaction) -{ - JBUFFER_TRACE(jh, "entry"); - J_ASSERT_JH(jh, buffer_dirty(jh2bh(jh)) || buffer_jbddirty(jh2bh(jh))); - J_ASSERT_JH(jh, jh->b_cp_transaction == NULL); - - /* Get reference for checkpointing transaction */ - journal_grab_journal_head(jh2bh(jh)); - jh->b_cp_transaction = transaction; - - if (!transaction->t_checkpoint_list) { - jh->b_cpnext = jh->b_cpprev = jh; - } else { - jh->b_cpnext = transaction->t_checkpoint_list; - jh->b_cpprev = transaction->t_checkpoint_list->b_cpprev; - jh->b_cpprev->b_cpnext = jh; - jh->b_cpnext->b_cpprev = jh; - } - transaction->t_checkpoint_list = jh; -} - -/* - * We've finished with this transaction structure: adios... - * - * The transaction must have no links except for the checkpoint by this - * point. - * - * Called with the journal locked. - * Called with j_list_lock held. - */ - -void __journal_drop_transaction(journal_t *journal, transaction_t *transaction) -{ - assert_spin_locked(&journal->j_list_lock); - if (transaction->t_cpnext) { - transaction->t_cpnext->t_cpprev = transaction->t_cpprev; - transaction->t_cpprev->t_cpnext = transaction->t_cpnext; - if (journal->j_checkpoint_transactions == transaction) - journal->j_checkpoint_transactions = - transaction->t_cpnext; - if (journal->j_checkpoint_transactions == transaction) - journal->j_checkpoint_transactions = NULL; - } - - J_ASSERT(transaction->t_state == T_FINISHED); - J_ASSERT(transaction->t_buffers == NULL); - J_ASSERT(transaction->t_sync_datalist == NULL); - J_ASSERT(transaction->t_forget == NULL); - J_ASSERT(transaction->t_iobuf_list == NULL); - J_ASSERT(transaction->t_shadow_list == NULL); - J_ASSERT(transaction->t_log_list == NULL); - J_ASSERT(transaction->t_checkpoint_list == NULL); - J_ASSERT(transaction->t_checkpoint_io_list == NULL); - J_ASSERT(transaction->t_updates == 0); - J_ASSERT(journal->j_committing_transaction != transaction); - J_ASSERT(journal->j_running_transaction != transaction); - - trace_jbd_drop_transaction(journal, transaction); - jbd_debug(1, "Dropping transaction %d, all done\n", transaction->t_tid); - kfree(transaction); -} diff --git a/fs/jbd/commit.c b/fs/jbd/commit.c deleted file mode 100644 index bb217dcb41affb..00000000000000 --- a/fs/jbd/commit.c +++ /dev/null @@ -1,1021 +0,0 @@ -/* - * linux/fs/jbd/commit.c - * - * Written by Stephen C. Tweedie , 1998 - * - * Copyright 1998 Red Hat corp --- All Rights Reserved - * - * This file is part of the Linux kernel and is made available under - * the terms of the GNU General Public License, version 2, or at your - * option, any later version, incorporated herein by reference. - * - * Journal commit routines for the generic filesystem journaling code; - * part of the ext2fs journaling system. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -/* - * Default IO end handler for temporary BJ_IO buffer_heads. - */ -static void journal_end_buffer_io_sync(struct buffer_head *bh, int uptodate) -{ - BUFFER_TRACE(bh, ""); - if (uptodate) - set_buffer_uptodate(bh); - else - clear_buffer_uptodate(bh); - unlock_buffer(bh); -} - -/* - * When an ext3-ordered file is truncated, it is possible that many pages are - * not successfully freed, because they are attached to a committing transaction. - * After the transaction commits, these pages are left on the LRU, with no - * ->mapping, and with attached buffers. These pages are trivially reclaimable - * by the VM, but their apparent absence upsets the VM accounting, and it makes - * the numbers in /proc/meminfo look odd. - * - * So here, we have a buffer which has just come off the forget list. Look to - * see if we can strip all buffers from the backing page. - * - * Called under journal->j_list_lock. The caller provided us with a ref - * against the buffer, and we drop that here. - */ -static void release_buffer_page(struct buffer_head *bh) -{ - struct page *page; - - if (buffer_dirty(bh)) - goto nope; - if (atomic_read(&bh->b_count) != 1) - goto nope; - page = bh->b_page; - if (!page) - goto nope; - if (page->mapping) - goto nope; - - /* OK, it's a truncated page */ - if (!trylock_page(page)) - goto nope; - - page_cache_get(page); - __brelse(bh); - try_to_free_buffers(page); - unlock_page(page); - page_cache_release(page); - return; - -nope: - __brelse(bh); -} - -/* - * Decrement reference counter for data buffer. If it has been marked - * 'BH_Freed', release it and the page to which it belongs if possible. - */ -static void release_data_buffer(struct buffer_head *bh) -{ - if (buffer_freed(bh)) { - WARN_ON_ONCE(buffer_dirty(bh)); - clear_buffer_freed(bh); - clear_buffer_mapped(bh); - clear_buffer_new(bh); - clear_buffer_req(bh); - bh->b_bdev = NULL; - release_buffer_page(bh); - } else - put_bh(bh); -} - -/* - * Try to acquire jbd_lock_bh_state() against the buffer, when j_list_lock is - * held. For ranking reasons we must trylock. If we lose, schedule away and - * return 0. j_list_lock is dropped in this case. - */ -static int inverted_lock(journal_t *journal, struct buffer_head *bh) -{ - if (!jbd_trylock_bh_state(bh)) { - spin_unlock(&journal->j_list_lock); - schedule(); - return 0; - } - return 1; -} - -/* Done it all: now write the commit record. We should have - * cleaned up our previous buffers by now, so if we are in abort - * mode we can now just skip the rest of the journal write - * entirely. - * - * Returns 1 if the journal needs to be aborted or 0 on success - */ -static int journal_write_commit_record(journal_t *journal, - transaction_t *commit_transaction) -{ - struct journal_head *descriptor; - struct buffer_head *bh; - journal_header_t *header; - int ret; - - if (is_journal_aborted(journal)) - return 0; - - descriptor = journal_get_descriptor_buffer(journal); - if (!descriptor) - return 1; - - bh = jh2bh(descriptor); - - header = (journal_header_t *)(bh->b_data); - header->h_magic = cpu_to_be32(JFS_MAGIC_NUMBER); - header->h_blocktype = cpu_to_be32(JFS_COMMIT_BLOCK); - header->h_sequence = cpu_to_be32(commit_transaction->t_tid); - - JBUFFER_TRACE(descriptor, "write commit block"); - set_buffer_dirty(bh); - - if (journal->j_flags & JFS_BARRIER) - ret = __sync_dirty_buffer(bh, WRITE_SYNC | WRITE_FLUSH_FUA); - else - ret = sync_dirty_buffer(bh); - - put_bh(bh); /* One for getblk() */ - journal_put_journal_head(descriptor); - - return (ret == -EIO); -} - -static void journal_do_submit_data(struct buffer_head **wbuf, int bufs, - int write_op) -{ - int i; - - for (i = 0; i < bufs; i++) { - wbuf[i]->b_end_io = end_buffer_write_sync; - /* - * Here we write back pagecache data that may be mmaped. Since - * we cannot afford to clean the page and set PageWriteback - * here due to lock ordering (page lock ranks above transaction - * start), the data can change while IO is in flight. Tell the - * block layer it should bounce the bio pages if stable data - * during write is required. - * - * We use up our safety reference in submit_bh(). - */ - _submit_bh(write_op, wbuf[i], 1 << BIO_SNAP_STABLE); - } -} - -/* - * Submit all the data buffers to disk - */ -static int journal_submit_data_buffers(journal_t *journal, - transaction_t *commit_transaction, - int write_op) -{ - struct journal_head *jh; - struct buffer_head *bh; - int locked; - int bufs = 0; - struct buffer_head **wbuf = journal->j_wbuf; - int err = 0; - - /* - * Whenever we unlock the journal and sleep, things can get added - * onto ->t_sync_datalist, so we have to keep looping back to - * write_out_data until we *know* that the list is empty. - * - * Cleanup any flushed data buffers from the data list. Even in - * abort mode, we want to flush this out as soon as possible. - */ -write_out_data: - cond_resched(); - spin_lock(&journal->j_list_lock); - - while (commit_transaction->t_sync_datalist) { - jh = commit_transaction->t_sync_datalist; - bh = jh2bh(jh); - locked = 0; - - /* Get reference just to make sure buffer does not disappear - * when we are forced to drop various locks */ - get_bh(bh); - /* If the buffer is dirty, we need to submit IO and hence - * we need the buffer lock. We try to lock the buffer without - * blocking. If we fail, we need to drop j_list_lock and do - * blocking lock_buffer(). - */ - if (buffer_dirty(bh)) { - if (!trylock_buffer(bh)) { - BUFFER_TRACE(bh, "needs blocking lock"); - spin_unlock(&journal->j_list_lock); - trace_jbd_do_submit_data(journal, - commit_transaction); - /* Write out all data to prevent deadlocks */ - journal_do_submit_data(wbuf, bufs, write_op); - bufs = 0; - lock_buffer(bh); - spin_lock(&journal->j_list_lock); - } - locked = 1; - } - /* We have to get bh_state lock. Again out of order, sigh. */ - if (!inverted_lock(journal, bh)) { - jbd_lock_bh_state(bh); - spin_lock(&journal->j_list_lock); - } - /* Someone already cleaned up the buffer? */ - if (!buffer_jbd(bh) || bh2jh(bh) != jh - || jh->b_transaction != commit_transaction - || jh->b_jlist != BJ_SyncData) { - jbd_unlock_bh_state(bh); - if (locked) - unlock_buffer(bh); - BUFFER_TRACE(bh, "already cleaned up"); - release_data_buffer(bh); - continue; - } - if (locked && test_clear_buffer_dirty(bh)) { - BUFFER_TRACE(bh, "needs writeout, adding to array"); - wbuf[bufs++] = bh; - __journal_file_buffer(jh, commit_transaction, - BJ_Locked); - jbd_unlock_bh_state(bh); - if (bufs == journal->j_wbufsize) { - spin_unlock(&journal->j_list_lock); - trace_jbd_do_submit_data(journal, - commit_transaction); - journal_do_submit_data(wbuf, bufs, write_op); - bufs = 0; - goto write_out_data; - } - } else if (!locked && buffer_locked(bh)) { - __journal_file_buffer(jh, commit_transaction, - BJ_Locked); - jbd_unlock_bh_state(bh); - put_bh(bh); - } else { - BUFFER_TRACE(bh, "writeout complete: unfile"); - if (unlikely(!buffer_uptodate(bh))) - err = -EIO; - __journal_unfile_buffer(jh); - jbd_unlock_bh_state(bh); - if (locked) - unlock_buffer(bh); - release_data_buffer(bh); - } - - if (need_resched() || spin_needbreak(&journal->j_list_lock)) { - spin_unlock(&journal->j_list_lock); - goto write_out_data; - } - } - spin_unlock(&journal->j_list_lock); - trace_jbd_do_submit_data(journal, commit_transaction); - journal_do_submit_data(wbuf, bufs, write_op); - - return err; -} - -/* - * journal_commit_transaction - * - * The primary function for committing a transaction to the log. This - * function is called by the journal thread to begin a complete commit. - */ -void journal_commit_transaction(journal_t *journal) -{ - transaction_t *commit_transaction; - struct journal_head *jh, *new_jh, *descriptor; - struct buffer_head **wbuf = journal->j_wbuf; - int bufs; - int flags; - int err; - unsigned int blocknr; - ktime_t start_time; - u64 commit_time; - char *tagp = NULL; - journal_header_t *header; - journal_block_tag_t *tag = NULL; - int space_left = 0; - int first_tag = 0; - int tag_flag; - int i; - struct blk_plug plug; - int write_op = WRITE; - - /* - * First job: lock down the current transaction and wait for - * all outstanding updates to complete. - */ - - /* Do we need to erase the effects of a prior journal_flush? */ - if (journal->j_flags & JFS_FLUSHED) { - jbd_debug(3, "super block updated\n"); - mutex_lock(&journal->j_checkpoint_mutex); - /* - * We hold j_checkpoint_mutex so tail cannot change under us. - * We don't need any special data guarantees for writing sb - * since journal is empty and it is ok for write to be - * flushed only with transaction commit. - */ - journal_update_sb_log_tail(journal, journal->j_tail_sequence, - journal->j_tail, WRITE_SYNC); - mutex_unlock(&journal->j_checkpoint_mutex); - } else { - jbd_debug(3, "superblock not updated\n"); - } - - J_ASSERT(journal->j_running_transaction != NULL); - J_ASSERT(journal->j_committing_transaction == NULL); - - commit_transaction = journal->j_running_transaction; - - trace_jbd_start_commit(journal, commit_transaction); - jbd_debug(1, "JBD: starting commit of transaction %d\n", - commit_transaction->t_tid); - - spin_lock(&journal->j_state_lock); - J_ASSERT(commit_transaction->t_state == T_RUNNING); - commit_transaction->t_state = T_LOCKED; - - trace_jbd_commit_locking(journal, commit_transaction); - spin_lock(&commit_transaction->t_handle_lock); - while (commit_transaction->t_updates) { - DEFINE_WAIT(wait); - - prepare_to_wait(&journal->j_wait_updates, &wait, - TASK_UNINTERRUPTIBLE); - if (commit_transaction->t_updates) { - spin_unlock(&commit_transaction->t_handle_lock); - spin_unlock(&journal->j_state_lock); - schedule(); - spin_lock(&journal->j_state_lock); - spin_lock(&commit_transaction->t_handle_lock); - } - finish_wait(&journal->j_wait_updates, &wait); - } - spin_unlock(&commit_transaction->t_handle_lock); - - J_ASSERT (commit_transaction->t_outstanding_credits <= - journal->j_max_transaction_buffers); - - /* - * First thing we are allowed to do is to discard any remaining - * BJ_Reserved buffers. Note, it is _not_ permissible to assume - * that there are no such buffers: if a large filesystem - * operation like a truncate needs to split itself over multiple - * transactions, then it may try to do a journal_restart() while - * there are still BJ_Reserved buffers outstanding. These must - * be released cleanly from the current transaction. - * - * In this case, the filesystem must still reserve write access - * again before modifying the buffer in the new transaction, but - * we do not require it to remember exactly which old buffers it - * has reserved. This is consistent with the existing behaviour - * that multiple journal_get_write_access() calls to the same - * buffer are perfectly permissible. - */ - while (commit_transaction->t_reserved_list) { - jh = commit_transaction->t_reserved_list; - JBUFFER_TRACE(jh, "reserved, unused: refile"); - /* - * A journal_get_undo_access()+journal_release_buffer() may - * leave undo-committed data. - */ - if (jh->b_committed_data) { - struct buffer_head *bh = jh2bh(jh); - - jbd_lock_bh_state(bh); - jbd_free(jh->b_committed_data, bh->b_size); - jh->b_committed_data = NULL; - jbd_unlock_bh_state(bh); - } - journal_refile_buffer(journal, jh); - } - - /* - * Now try to drop any written-back buffers from the journal's - * checkpoint lists. We do this *before* commit because it potentially - * frees some memory - */ - spin_lock(&journal->j_list_lock); - __journal_clean_checkpoint_list(journal); - spin_unlock(&journal->j_list_lock); - - jbd_debug (3, "JBD: commit phase 1\n"); - - /* - * Clear revoked flag to reflect there is no revoked buffers - * in the next transaction which is going to be started. - */ - journal_clear_buffer_revoked_flags(journal); - - /* - * Switch to a new revoke table. - */ - journal_switch_revoke_table(journal); - - trace_jbd_commit_flushing(journal, commit_transaction); - commit_transaction->t_state = T_FLUSH; - journal->j_committing_transaction = commit_transaction; - journal->j_running_transaction = NULL; - start_time = ktime_get(); - commit_transaction->t_log_start = journal->j_head; - wake_up(&journal->j_wait_transaction_locked); - spin_unlock(&journal->j_state_lock); - - jbd_debug (3, "JBD: commit phase 2\n"); - - if (tid_geq(journal->j_commit_waited, commit_transaction->t_tid)) - write_op = WRITE_SYNC; - - /* - * Now start flushing things to disk, in the order they appear - * on the transaction lists. Data blocks go first. - */ - blk_start_plug(&plug); - err = journal_submit_data_buffers(journal, commit_transaction, - write_op); - blk_finish_plug(&plug); - - /* - * Wait for all previously submitted IO to complete. - */ - spin_lock(&journal->j_list_lock); - while (commit_transaction->t_locked_list) { - struct buffer_head *bh; - - jh = commit_transaction->t_locked_list->b_tprev; - bh = jh2bh(jh); - get_bh(bh); - if (buffer_locked(bh)) { - spin_unlock(&journal->j_list_lock); - wait_on_buffer(bh); - spin_lock(&journal->j_list_lock); - } - if (unlikely(!buffer_uptodate(bh))) { - if (!trylock_page(bh->b_page)) { - spin_unlock(&journal->j_list_lock); - lock_page(bh->b_page); - spin_lock(&journal->j_list_lock); - } - if (bh->b_page->mapping) - set_bit(AS_EIO, &bh->b_page->mapping->flags); - - unlock_page(bh->b_page); - SetPageError(bh->b_page); - err = -EIO; - } - if (!inverted_lock(journal, bh)) { - put_bh(bh); - spin_lock(&journal->j_list_lock); - continue; - } - if (buffer_jbd(bh) && bh2jh(bh) == jh && - jh->b_transaction == commit_transaction && - jh->b_jlist == BJ_Locked) - __journal_unfile_buffer(jh); - jbd_unlock_bh_state(bh); - release_data_buffer(bh); - cond_resched_lock(&journal->j_list_lock); - } - spin_unlock(&journal->j_list_lock); - - if (err) { - char b[BDEVNAME_SIZE]; - - printk(KERN_WARNING - "JBD: Detected IO errors while flushing file data " - "on %s\n", bdevname(journal->j_fs_dev, b)); - if (journal->j_flags & JFS_ABORT_ON_SYNCDATA_ERR) - journal_abort(journal, err); - err = 0; - } - - blk_start_plug(&plug); - - journal_write_revoke_records(journal, commit_transaction, write_op); - - /* - * If we found any dirty or locked buffers, then we should have - * looped back up to the write_out_data label. If there weren't - * any then journal_clean_data_list should have wiped the list - * clean by now, so check that it is in fact empty. - */ - J_ASSERT (commit_transaction->t_sync_datalist == NULL); - - jbd_debug (3, "JBD: commit phase 3\n"); - - /* - * Way to go: we have now written out all of the data for a - * transaction! Now comes the tricky part: we need to write out - * metadata. Loop over the transaction's entire buffer list: - */ - spin_lock(&journal->j_state_lock); - commit_transaction->t_state = T_COMMIT; - spin_unlock(&journal->j_state_lock); - - trace_jbd_commit_logging(journal, commit_transaction); - J_ASSERT(commit_transaction->t_nr_buffers <= - commit_transaction->t_outstanding_credits); - - descriptor = NULL; - bufs = 0; - while (commit_transaction->t_buffers) { - - /* Find the next buffer to be journaled... */ - - jh = commit_transaction->t_buffers; - - /* If we're in abort mode, we just un-journal the buffer and - release it. */ - - if (is_journal_aborted(journal)) { - clear_buffer_jbddirty(jh2bh(jh)); - JBUFFER_TRACE(jh, "journal is aborting: refile"); - journal_refile_buffer(journal, jh); - /* If that was the last one, we need to clean up - * any descriptor buffers which may have been - * already allocated, even if we are now - * aborting. */ - if (!commit_transaction->t_buffers) - goto start_journal_io; - continue; - } - - /* Make sure we have a descriptor block in which to - record the metadata buffer. */ - - if (!descriptor) { - struct buffer_head *bh; - - J_ASSERT (bufs == 0); - - jbd_debug(4, "JBD: get descriptor\n"); - - descriptor = journal_get_descriptor_buffer(journal); - if (!descriptor) { - journal_abort(journal, -EIO); - continue; - } - - bh = jh2bh(descriptor); - jbd_debug(4, "JBD: got buffer %llu (%p)\n", - (unsigned long long)bh->b_blocknr, bh->b_data); - header = (journal_header_t *)&bh->b_data[0]; - header->h_magic = cpu_to_be32(JFS_MAGIC_NUMBER); - header->h_blocktype = cpu_to_be32(JFS_DESCRIPTOR_BLOCK); - header->h_sequence = cpu_to_be32(commit_transaction->t_tid); - - tagp = &bh->b_data[sizeof(journal_header_t)]; - space_left = bh->b_size - sizeof(journal_header_t); - first_tag = 1; - set_buffer_jwrite(bh); - set_buffer_dirty(bh); - wbuf[bufs++] = bh; - - /* Record it so that we can wait for IO - completion later */ - BUFFER_TRACE(bh, "ph3: file as descriptor"); - journal_file_buffer(descriptor, commit_transaction, - BJ_LogCtl); - } - - /* Where is the buffer to be written? */ - - err = journal_next_log_block(journal, &blocknr); - /* If the block mapping failed, just abandon the buffer - and repeat this loop: we'll fall into the - refile-on-abort condition above. */ - if (err) { - journal_abort(journal, err); - continue; - } - - /* - * start_this_handle() uses t_outstanding_credits to determine - * the free space in the log, but this counter is changed - * by journal_next_log_block() also. - */ - commit_transaction->t_outstanding_credits--; - - /* Bump b_count to prevent truncate from stumbling over - the shadowed buffer! @@@ This can go if we ever get - rid of the BJ_IO/BJ_Shadow pairing of buffers. */ - get_bh(jh2bh(jh)); - - /* Make a temporary IO buffer with which to write it out - (this will requeue both the metadata buffer and the - temporary IO buffer). new_bh goes on BJ_IO*/ - - set_buffer_jwrite(jh2bh(jh)); - /* - * akpm: journal_write_metadata_buffer() sets - * new_bh->b_transaction to commit_transaction. - * We need to clean this up before we release new_bh - * (which is of type BJ_IO) - */ - JBUFFER_TRACE(jh, "ph3: write metadata"); - flags = journal_write_metadata_buffer(commit_transaction, - jh, &new_jh, blocknr); - set_buffer_jwrite(jh2bh(new_jh)); - wbuf[bufs++] = jh2bh(new_jh); - - /* Record the new block's tag in the current descriptor - buffer */ - - tag_flag = 0; - if (flags & 1) - tag_flag |= JFS_FLAG_ESCAPE; - if (!first_tag) - tag_flag |= JFS_FLAG_SAME_UUID; - - tag = (journal_block_tag_t *) tagp; - tag->t_blocknr = cpu_to_be32(jh2bh(jh)->b_blocknr); - tag->t_flags = cpu_to_be32(tag_flag); - tagp += sizeof(journal_block_tag_t); - space_left -= sizeof(journal_block_tag_t); - - if (first_tag) { - memcpy (tagp, journal->j_uuid, 16); - tagp += 16; - space_left -= 16; - first_tag = 0; - } - - /* If there's no more to do, or if the descriptor is full, - let the IO rip! */ - - if (bufs == journal->j_wbufsize || - commit_transaction->t_buffers == NULL || - space_left < sizeof(journal_block_tag_t) + 16) { - - jbd_debug(4, "JBD: Submit %d IOs\n", bufs); - - /* Write an end-of-descriptor marker before - submitting the IOs. "tag" still points to - the last tag we set up. */ - - tag->t_flags |= cpu_to_be32(JFS_FLAG_LAST_TAG); - -start_journal_io: - for (i = 0; i < bufs; i++) { - struct buffer_head *bh = wbuf[i]; - lock_buffer(bh); - clear_buffer_dirty(bh); - set_buffer_uptodate(bh); - bh->b_end_io = journal_end_buffer_io_sync; - /* - * In data=journal mode, here we can end up - * writing pagecache data that might be - * mmapped. Since we can't afford to clean the - * page and set PageWriteback (see the comment - * near the other use of _submit_bh()), the - * data can change while the write is in - * flight. Tell the block layer to bounce the - * bio pages if stable pages are required. - */ - _submit_bh(write_op, bh, 1 << BIO_SNAP_STABLE); - } - cond_resched(); - - /* Force a new descriptor to be generated next - time round the loop. */ - descriptor = NULL; - bufs = 0; - } - } - - blk_finish_plug(&plug); - - /* Lo and behold: we have just managed to send a transaction to - the log. Before we can commit it, wait for the IO so far to - complete. Control buffers being written are on the - transaction's t_log_list queue, and metadata buffers are on - the t_iobuf_list queue. - - Wait for the buffers in reverse order. That way we are - less likely to be woken up until all IOs have completed, and - so we incur less scheduling load. - */ - - jbd_debug(3, "JBD: commit phase 4\n"); - - /* - * akpm: these are BJ_IO, and j_list_lock is not needed. - * See __journal_try_to_free_buffer. - */ -wait_for_iobuf: - while (commit_transaction->t_iobuf_list != NULL) { - struct buffer_head *bh; - - jh = commit_transaction->t_iobuf_list->b_tprev; - bh = jh2bh(jh); - if (buffer_locked(bh)) { - wait_on_buffer(bh); - goto wait_for_iobuf; - } - if (cond_resched()) - goto wait_for_iobuf; - - if (unlikely(!buffer_uptodate(bh))) - err = -EIO; - - clear_buffer_jwrite(bh); - - JBUFFER_TRACE(jh, "ph4: unfile after journal write"); - journal_unfile_buffer(journal, jh); - - /* - * ->t_iobuf_list should contain only dummy buffer_heads - * which were created by journal_write_metadata_buffer(). - */ - BUFFER_TRACE(bh, "dumping temporary bh"); - journal_put_journal_head(jh); - __brelse(bh); - J_ASSERT_BH(bh, atomic_read(&bh->b_count) == 0); - free_buffer_head(bh); - - /* We also have to unlock and free the corresponding - shadowed buffer */ - jh = commit_transaction->t_shadow_list->b_tprev; - bh = jh2bh(jh); - clear_buffer_jwrite(bh); - J_ASSERT_BH(bh, buffer_jbddirty(bh)); - - /* The metadata is now released for reuse, but we need - to remember it against this transaction so that when - we finally commit, we can do any checkpointing - required. */ - JBUFFER_TRACE(jh, "file as BJ_Forget"); - journal_file_buffer(jh, commit_transaction, BJ_Forget); - /* - * Wake up any transactions which were waiting for this - * IO to complete. The barrier must be here so that changes - * by journal_file_buffer() take effect before wake_up_bit() - * does the waitqueue check. - */ - smp_mb(); - wake_up_bit(&bh->b_state, BH_Unshadow); - JBUFFER_TRACE(jh, "brelse shadowed buffer"); - __brelse(bh); - } - - J_ASSERT (commit_transaction->t_shadow_list == NULL); - - jbd_debug(3, "JBD: commit phase 5\n"); - - /* Here we wait for the revoke record and descriptor record buffers */ - wait_for_ctlbuf: - while (commit_transaction->t_log_list != NULL) { - struct buffer_head *bh; - - jh = commit_transaction->t_log_list->b_tprev; - bh = jh2bh(jh); - if (buffer_locked(bh)) { - wait_on_buffer(bh); - goto wait_for_ctlbuf; - } - if (cond_resched()) - goto wait_for_ctlbuf; - - if (unlikely(!buffer_uptodate(bh))) - err = -EIO; - - BUFFER_TRACE(bh, "ph5: control buffer writeout done: unfile"); - clear_buffer_jwrite(bh); - journal_unfile_buffer(journal, jh); - journal_put_journal_head(jh); - __brelse(bh); /* One for getblk */ - /* AKPM: bforget here */ - } - - if (err) - journal_abort(journal, err); - - jbd_debug(3, "JBD: commit phase 6\n"); - - /* All metadata is written, now write commit record and do cleanup */ - spin_lock(&journal->j_state_lock); - J_ASSERT(commit_transaction->t_state == T_COMMIT); - commit_transaction->t_state = T_COMMIT_RECORD; - spin_unlock(&journal->j_state_lock); - - if (journal_write_commit_record(journal, commit_transaction)) - err = -EIO; - - if (err) - journal_abort(journal, err); - - /* End of a transaction! Finally, we can do checkpoint - processing: any buffers committed as a result of this - transaction can be removed from any checkpoint list it was on - before. */ - - jbd_debug(3, "JBD: commit phase 7\n"); - - J_ASSERT(commit_transaction->t_sync_datalist == NULL); - J_ASSERT(commit_transaction->t_buffers == NULL); - J_ASSERT(commit_transaction->t_checkpoint_list == NULL); - J_ASSERT(commit_transaction->t_iobuf_list == NULL); - J_ASSERT(commit_transaction->t_shadow_list == NULL); - J_ASSERT(commit_transaction->t_log_list == NULL); - -restart_loop: - /* - * As there are other places (journal_unmap_buffer()) adding buffers - * to this list we have to be careful and hold the j_list_lock. - */ - spin_lock(&journal->j_list_lock); - while (commit_transaction->t_forget) { - transaction_t *cp_transaction; - struct buffer_head *bh; - int try_to_free = 0; - - jh = commit_transaction->t_forget; - spin_unlock(&journal->j_list_lock); - bh = jh2bh(jh); - /* - * Get a reference so that bh cannot be freed before we are - * done with it. - */ - get_bh(bh); - jbd_lock_bh_state(bh); - J_ASSERT_JH(jh, jh->b_transaction == commit_transaction || - jh->b_transaction == journal->j_running_transaction); - - /* - * If there is undo-protected committed data against - * this buffer, then we can remove it now. If it is a - * buffer needing such protection, the old frozen_data - * field now points to a committed version of the - * buffer, so rotate that field to the new committed - * data. - * - * Otherwise, we can just throw away the frozen data now. - */ - if (jh->b_committed_data) { - jbd_free(jh->b_committed_data, bh->b_size); - jh->b_committed_data = NULL; - if (jh->b_frozen_data) { - jh->b_committed_data = jh->b_frozen_data; - jh->b_frozen_data = NULL; - } - } else if (jh->b_frozen_data) { - jbd_free(jh->b_frozen_data, bh->b_size); - jh->b_frozen_data = NULL; - } - - spin_lock(&journal->j_list_lock); - cp_transaction = jh->b_cp_transaction; - if (cp_transaction) { - JBUFFER_TRACE(jh, "remove from old cp transaction"); - __journal_remove_checkpoint(jh); - } - - /* Only re-checkpoint the buffer_head if it is marked - * dirty. If the buffer was added to the BJ_Forget list - * by journal_forget, it may no longer be dirty and - * there's no point in keeping a checkpoint record for - * it. */ - - /* - * A buffer which has been freed while still being journaled by - * a previous transaction. - */ - if (buffer_freed(bh)) { - /* - * If the running transaction is the one containing - * "add to orphan" operation (b_next_transaction != - * NULL), we have to wait for that transaction to - * commit before we can really get rid of the buffer. - * So just clear b_modified to not confuse transaction - * credit accounting and refile the buffer to - * BJ_Forget of the running transaction. If the just - * committed transaction contains "add to orphan" - * operation, we can completely invalidate the buffer - * now. We are rather throughout in that since the - * buffer may be still accessible when blocksize < - * pagesize and it is attached to the last partial - * page. - */ - jh->b_modified = 0; - if (!jh->b_next_transaction) { - clear_buffer_freed(bh); - clear_buffer_jbddirty(bh); - clear_buffer_mapped(bh); - clear_buffer_new(bh); - clear_buffer_req(bh); - bh->b_bdev = NULL; - } - } - - if (buffer_jbddirty(bh)) { - JBUFFER_TRACE(jh, "add to new checkpointing trans"); - __journal_insert_checkpoint(jh, commit_transaction); - if (is_journal_aborted(journal)) - clear_buffer_jbddirty(bh); - } else { - J_ASSERT_BH(bh, !buffer_dirty(bh)); - /* - * The buffer on BJ_Forget list and not jbddirty means - * it has been freed by this transaction and hence it - * could not have been reallocated until this - * transaction has committed. *BUT* it could be - * reallocated once we have written all the data to - * disk and before we process the buffer on BJ_Forget - * list. - */ - if (!jh->b_next_transaction) - try_to_free = 1; - } - JBUFFER_TRACE(jh, "refile or unfile freed buffer"); - __journal_refile_buffer(jh); - jbd_unlock_bh_state(bh); - if (try_to_free) - release_buffer_page(bh); - else - __brelse(bh); - cond_resched_lock(&journal->j_list_lock); - } - spin_unlock(&journal->j_list_lock); - /* - * This is a bit sleazy. We use j_list_lock to protect transition - * of a transaction into T_FINISHED state and calling - * __journal_drop_transaction(). Otherwise we could race with - * other checkpointing code processing the transaction... - */ - spin_lock(&journal->j_state_lock); - spin_lock(&journal->j_list_lock); - /* - * Now recheck if some buffers did not get attached to the transaction - * while the lock was dropped... - */ - if (commit_transaction->t_forget) { - spin_unlock(&journal->j_list_lock); - spin_unlock(&journal->j_state_lock); - goto restart_loop; - } - - /* Done with this transaction! */ - - jbd_debug(3, "JBD: commit phase 8\n"); - - J_ASSERT(commit_transaction->t_state == T_COMMIT_RECORD); - - commit_transaction->t_state = T_FINISHED; - J_ASSERT(commit_transaction == journal->j_committing_transaction); - journal->j_commit_sequence = commit_transaction->t_tid; - journal->j_committing_transaction = NULL; - commit_time = ktime_to_ns(ktime_sub(ktime_get(), start_time)); - - /* - * weight the commit time higher than the average time so we don't - * react too strongly to vast changes in commit time - */ - if (likely(journal->j_average_commit_time)) - journal->j_average_commit_time = (commit_time*3 + - journal->j_average_commit_time) / 4; - else - journal->j_average_commit_time = commit_time; - - spin_unlock(&journal->j_state_lock); - - if (commit_transaction->t_checkpoint_list == NULL && - commit_transaction->t_checkpoint_io_list == NULL) { - __journal_drop_transaction(journal, commit_transaction); - } else { - if (journal->j_checkpoint_transactions == NULL) { - journal->j_checkpoint_transactions = commit_transaction; - commit_transaction->t_cpnext = commit_transaction; - commit_transaction->t_cpprev = commit_transaction; - } else { - commit_transaction->t_cpnext = - journal->j_checkpoint_transactions; - commit_transaction->t_cpprev = - commit_transaction->t_cpnext->t_cpprev; - commit_transaction->t_cpnext->t_cpprev = - commit_transaction; - commit_transaction->t_cpprev->t_cpnext = - commit_transaction; - } - } - spin_unlock(&journal->j_list_lock); - - trace_jbd_end_commit(journal, commit_transaction); - jbd_debug(1, "JBD: commit %d complete, head %d\n", - journal->j_commit_sequence, journal->j_tail_sequence); - - wake_up(&journal->j_wait_done_commit); -} diff --git a/fs/jbd/journal.c b/fs/jbd/journal.c deleted file mode 100644 index c46a79adb6adf0..00000000000000 --- a/fs/jbd/journal.c +++ /dev/null @@ -1,2145 +0,0 @@ -/* - * linux/fs/jbd/journal.c - * - * Written by Stephen C. Tweedie , 1998 - * - * Copyright 1998 Red Hat corp --- All Rights Reserved - * - * This file is part of the Linux kernel and is made available under - * the terms of the GNU General Public License, version 2, or at your - * option, any later version, incorporated herein by reference. - * - * Generic filesystem journal-writing code; part of the ext2fs - * journaling system. - * - * This file manages journals: areas of disk reserved for logging - * transactional updates. This includes the kernel journaling thread - * which is responsible for scheduling updates to the log. - * - * We do not actually manage the physical storage of the journal in this - * file: that is left to a per-journal policy function, which allows us - * to store the journal within a filesystem-specified area for ext2 - * journaling (ext2 can use a reserved inode for storing the log). - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define CREATE_TRACE_POINTS -#include - -#include -#include - -EXPORT_SYMBOL(journal_start); -EXPORT_SYMBOL(journal_restart); -EXPORT_SYMBOL(journal_extend); -EXPORT_SYMBOL(journal_stop); -EXPORT_SYMBOL(journal_lock_updates); -EXPORT_SYMBOL(journal_unlock_updates); -EXPORT_SYMBOL(journal_get_write_access); -EXPORT_SYMBOL(journal_get_create_access); -EXPORT_SYMBOL(journal_get_undo_access); -EXPORT_SYMBOL(journal_dirty_data); -EXPORT_SYMBOL(journal_dirty_metadata); -EXPORT_SYMBOL(journal_release_buffer); -EXPORT_SYMBOL(journal_forget); -#if 0 -EXPORT_SYMBOL(journal_sync_buffer); -#endif -EXPORT_SYMBOL(journal_flush); -EXPORT_SYMBOL(journal_revoke); - -EXPORT_SYMBOL(journal_init_dev); -EXPORT_SYMBOL(journal_init_inode); -EXPORT_SYMBOL(journal_update_format); -EXPORT_SYMBOL(journal_check_used_features); -EXPORT_SYMBOL(journal_check_available_features); -EXPORT_SYMBOL(journal_set_features); -EXPORT_SYMBOL(journal_create); -EXPORT_SYMBOL(journal_load); -EXPORT_SYMBOL(journal_destroy); -EXPORT_SYMBOL(journal_abort); -EXPORT_SYMBOL(journal_errno); -EXPORT_SYMBOL(journal_ack_err); -EXPORT_SYMBOL(journal_clear_err); -EXPORT_SYMBOL(log_wait_commit); -EXPORT_SYMBOL(log_start_commit); -EXPORT_SYMBOL(journal_start_commit); -EXPORT_SYMBOL(journal_force_commit_nested); -EXPORT_SYMBOL(journal_wipe); -EXPORT_SYMBOL(journal_blocks_per_page); -EXPORT_SYMBOL(journal_invalidatepage); -EXPORT_SYMBOL(journal_try_to_free_buffers); -EXPORT_SYMBOL(journal_force_commit); - -static int journal_convert_superblock_v1(journal_t *, journal_superblock_t *); -static void __journal_abort_soft (journal_t *journal, int errno); -static const char *journal_dev_name(journal_t *journal, char *buffer); - -#ifdef CONFIG_JBD_DEBUG -void __jbd_debug(int level, const char *file, const char *func, - unsigned int line, const char *fmt, ...) -{ - struct va_format vaf; - va_list args; - - if (level > journal_enable_debug) - return; - va_start(args, fmt); - vaf.fmt = fmt; - vaf.va = &args; - printk(KERN_DEBUG "%s: (%s, %u): %pV\n", file, func, line, &vaf); - va_end(args); -} -EXPORT_SYMBOL(__jbd_debug); -#endif - -/* - * Helper function used to manage commit timeouts - */ - -static void commit_timeout(unsigned long __data) -{ - struct task_struct * p = (struct task_struct *) __data; - - wake_up_process(p); -} - -/* - * kjournald: The main thread function used to manage a logging device - * journal. - * - * This kernel thread is responsible for two things: - * - * 1) COMMIT: Every so often we need to commit the current state of the - * filesystem to disk. The journal thread is responsible for writing - * all of the metadata buffers to disk. - * - * 2) CHECKPOINT: We cannot reuse a used section of the log file until all - * of the data in that part of the log has been rewritten elsewhere on - * the disk. Flushing these old buffers to reclaim space in the log is - * known as checkpointing, and this thread is responsible for that job. - */ - -static int kjournald(void *arg) -{ - journal_t *journal = arg; - transaction_t *transaction; - - /* - * Set up an interval timer which can be used to trigger a commit wakeup - * after the commit interval expires - */ - setup_timer(&journal->j_commit_timer, commit_timeout, - (unsigned long)current); - - set_freezable(); - - /* Record that the journal thread is running */ - journal->j_task = current; - wake_up(&journal->j_wait_done_commit); - - printk(KERN_INFO "kjournald starting. Commit interval %ld seconds\n", - journal->j_commit_interval / HZ); - - /* - * And now, wait forever for commit wakeup events. - */ - spin_lock(&journal->j_state_lock); - -loop: - if (journal->j_flags & JFS_UNMOUNT) - goto end_loop; - - jbd_debug(1, "commit_sequence=%d, commit_request=%d\n", - journal->j_commit_sequence, journal->j_commit_request); - - if (journal->j_commit_sequence != journal->j_commit_request) { - jbd_debug(1, "OK, requests differ\n"); - spin_unlock(&journal->j_state_lock); - del_timer_sync(&journal->j_commit_timer); - journal_commit_transaction(journal); - spin_lock(&journal->j_state_lock); - goto loop; - } - - wake_up(&journal->j_wait_done_commit); - if (freezing(current)) { - /* - * The simpler the better. Flushing journal isn't a - * good idea, because that depends on threads that may - * be already stopped. - */ - jbd_debug(1, "Now suspending kjournald\n"); - spin_unlock(&journal->j_state_lock); - try_to_freeze(); - spin_lock(&journal->j_state_lock); - } else { - /* - * We assume on resume that commits are already there, - * so we don't sleep - */ - DEFINE_WAIT(wait); - int should_sleep = 1; - - prepare_to_wait(&journal->j_wait_commit, &wait, - TASK_INTERRUPTIBLE); - if (journal->j_commit_sequence != journal->j_commit_request) - should_sleep = 0; - transaction = journal->j_running_transaction; - if (transaction && time_after_eq(jiffies, - transaction->t_expires)) - should_sleep = 0; - if (journal->j_flags & JFS_UNMOUNT) - should_sleep = 0; - if (should_sleep) { - spin_unlock(&journal->j_state_lock); - schedule(); - spin_lock(&journal->j_state_lock); - } - finish_wait(&journal->j_wait_commit, &wait); - } - - jbd_debug(1, "kjournald wakes\n"); - - /* - * Were we woken up by a commit wakeup event? - */ - transaction = journal->j_running_transaction; - if (transaction && time_after_eq(jiffies, transaction->t_expires)) { - journal->j_commit_request = transaction->t_tid; - jbd_debug(1, "woke because of timeout\n"); - } - goto loop; - -end_loop: - spin_unlock(&journal->j_state_lock); - del_timer_sync(&journal->j_commit_timer); - journal->j_task = NULL; - wake_up(&journal->j_wait_done_commit); - jbd_debug(1, "Journal thread exiting.\n"); - return 0; -} - -static int journal_start_thread(journal_t *journal) -{ - struct task_struct *t; - - t = kthread_run(kjournald, journal, "kjournald"); - if (IS_ERR(t)) - return PTR_ERR(t); - - wait_event(journal->j_wait_done_commit, journal->j_task != NULL); - return 0; -} - -static void journal_kill_thread(journal_t *journal) -{ - spin_lock(&journal->j_state_lock); - journal->j_flags |= JFS_UNMOUNT; - - while (journal->j_task) { - wake_up(&journal->j_wait_commit); - spin_unlock(&journal->j_state_lock); - wait_event(journal->j_wait_done_commit, - journal->j_task == NULL); - spin_lock(&journal->j_state_lock); - } - spin_unlock(&journal->j_state_lock); -} - -/* - * journal_write_metadata_buffer: write a metadata buffer to the journal. - * - * Writes a metadata buffer to a given disk block. The actual IO is not - * performed but a new buffer_head is constructed which labels the data - * to be written with the correct destination disk block. - * - * Any magic-number escaping which needs to be done will cause a - * copy-out here. If the buffer happens to start with the - * JFS_MAGIC_NUMBER, then we can't write it to the log directly: the - * magic number is only written to the log for descripter blocks. In - * this case, we copy the data and replace the first word with 0, and we - * return a result code which indicates that this buffer needs to be - * marked as an escaped buffer in the corresponding log descriptor - * block. The missing word can then be restored when the block is read - * during recovery. - * - * If the source buffer has already been modified by a new transaction - * since we took the last commit snapshot, we use the frozen copy of - * that data for IO. If we end up using the existing buffer_head's data - * for the write, then we *have* to lock the buffer to prevent anyone - * else from using and possibly modifying it while the IO is in - * progress. - * - * The function returns a pointer to the buffer_heads to be used for IO. - * - * We assume that the journal has already been locked in this function. - * - * Return value: - * <0: Error - * >=0: Finished OK - * - * On success: - * Bit 0 set == escape performed on the data - * Bit 1 set == buffer copy-out performed (kfree the data after IO) - */ - -int journal_write_metadata_buffer(transaction_t *transaction, - struct journal_head *jh_in, - struct journal_head **jh_out, - unsigned int blocknr) -{ - int need_copy_out = 0; - int done_copy_out = 0; - int do_escape = 0; - char *mapped_data; - struct buffer_head *new_bh; - struct journal_head *new_jh; - struct page *new_page; - unsigned int new_offset; - struct buffer_head *bh_in = jh2bh(jh_in); - journal_t *journal = transaction->t_journal; - - /* - * The buffer really shouldn't be locked: only the current committing - * transaction is allowed to write it, so nobody else is allowed - * to do any IO. - * - * akpm: except if we're journalling data, and write() output is - * also part of a shared mapping, and another thread has - * decided to launch a writepage() against this buffer. - */ - J_ASSERT_BH(bh_in, buffer_jbddirty(bh_in)); - - new_bh = alloc_buffer_head(GFP_NOFS|__GFP_NOFAIL); - /* keep subsequent assertions sane */ - atomic_set(&new_bh->b_count, 1); - new_jh = journal_add_journal_head(new_bh); /* This sleeps */ - - /* - * If a new transaction has already done a buffer copy-out, then - * we use that version of the data for the commit. - */ - jbd_lock_bh_state(bh_in); -repeat: - if (jh_in->b_frozen_data) { - done_copy_out = 1; - new_page = virt_to_page(jh_in->b_frozen_data); - new_offset = offset_in_page(jh_in->b_frozen_data); - } else { - new_page = jh2bh(jh_in)->b_page; - new_offset = offset_in_page(jh2bh(jh_in)->b_data); - } - - mapped_data = kmap_atomic(new_page); - /* - * Check for escaping - */ - if (*((__be32 *)(mapped_data + new_offset)) == - cpu_to_be32(JFS_MAGIC_NUMBER)) { - need_copy_out = 1; - do_escape = 1; - } - kunmap_atomic(mapped_data); - - /* - * Do we need to do a data copy? - */ - if (need_copy_out && !done_copy_out) { - char *tmp; - - jbd_unlock_bh_state(bh_in); - tmp = jbd_alloc(bh_in->b_size, GFP_NOFS); - jbd_lock_bh_state(bh_in); - if (jh_in->b_frozen_data) { - jbd_free(tmp, bh_in->b_size); - goto repeat; - } - - jh_in->b_frozen_data = tmp; - mapped_data = kmap_atomic(new_page); - memcpy(tmp, mapped_data + new_offset, jh2bh(jh_in)->b_size); - kunmap_atomic(mapped_data); - - new_page = virt_to_page(tmp); - new_offset = offset_in_page(tmp); - done_copy_out = 1; - } - - /* - * Did we need to do an escaping? Now we've done all the - * copying, we can finally do so. - */ - if (do_escape) { - mapped_data = kmap_atomic(new_page); - *((unsigned int *)(mapped_data + new_offset)) = 0; - kunmap_atomic(mapped_data); - } - - set_bh_page(new_bh, new_page, new_offset); - new_jh->b_transaction = NULL; - new_bh->b_size = jh2bh(jh_in)->b_size; - new_bh->b_bdev = transaction->t_journal->j_dev; - new_bh->b_blocknr = blocknr; - set_buffer_mapped(new_bh); - set_buffer_dirty(new_bh); - - *jh_out = new_jh; - - /* - * The to-be-written buffer needs to get moved to the io queue, - * and the original buffer whose contents we are shadowing or - * copying is moved to the transaction's shadow queue. - */ - JBUFFER_TRACE(jh_in, "file as BJ_Shadow"); - spin_lock(&journal->j_list_lock); - __journal_file_buffer(jh_in, transaction, BJ_Shadow); - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh_in); - - JBUFFER_TRACE(new_jh, "file as BJ_IO"); - journal_file_buffer(new_jh, transaction, BJ_IO); - - return do_escape | (done_copy_out << 1); -} - -/* - * Allocation code for the journal file. Manage the space left in the - * journal, so that we can begin checkpointing when appropriate. - */ - -/* - * __log_space_left: Return the number of free blocks left in the journal. - * - * Called with the journal already locked. - * - * Called under j_state_lock - */ - -int __log_space_left(journal_t *journal) -{ - int left = journal->j_free; - - assert_spin_locked(&journal->j_state_lock); - - /* - * Be pessimistic here about the number of those free blocks which - * might be required for log descriptor control blocks. - */ - -#define MIN_LOG_RESERVED_BLOCKS 32 /* Allow for rounding errors */ - - left -= MIN_LOG_RESERVED_BLOCKS; - - if (left <= 0) - return 0; - left -= (left >> 3); - return left; -} - -/* - * Called under j_state_lock. Returns true if a transaction commit was started. - */ -int __log_start_commit(journal_t *journal, tid_t target) -{ - /* - * The only transaction we can possibly wait upon is the - * currently running transaction (if it exists). Otherwise, - * the target tid must be an old one. - */ - if (journal->j_commit_request != target && - journal->j_running_transaction && - journal->j_running_transaction->t_tid == target) { - /* - * We want a new commit: OK, mark the request and wakeup the - * commit thread. We do _not_ do the commit ourselves. - */ - - journal->j_commit_request = target; - jbd_debug(1, "JBD: requesting commit %d/%d\n", - journal->j_commit_request, - journal->j_commit_sequence); - wake_up(&journal->j_wait_commit); - return 1; - } else if (!tid_geq(journal->j_commit_request, target)) - /* This should never happen, but if it does, preserve - the evidence before kjournald goes into a loop and - increments j_commit_sequence beyond all recognition. */ - WARN_ONCE(1, "jbd: bad log_start_commit: %u %u %u %u\n", - journal->j_commit_request, journal->j_commit_sequence, - target, journal->j_running_transaction ? - journal->j_running_transaction->t_tid : 0); - return 0; -} - -int log_start_commit(journal_t *journal, tid_t tid) -{ - int ret; - - spin_lock(&journal->j_state_lock); - ret = __log_start_commit(journal, tid); - spin_unlock(&journal->j_state_lock); - return ret; -} - -/* - * Force and wait upon a commit if the calling process is not within - * transaction. This is used for forcing out undo-protected data which contains - * bitmaps, when the fs is running out of space. - * - * We can only force the running transaction if we don't have an active handle; - * otherwise, we will deadlock. - * - * Returns true if a transaction was started. - */ -int journal_force_commit_nested(journal_t *journal) -{ - transaction_t *transaction = NULL; - tid_t tid; - - spin_lock(&journal->j_state_lock); - if (journal->j_running_transaction && !current->journal_info) { - transaction = journal->j_running_transaction; - __log_start_commit(journal, transaction->t_tid); - } else if (journal->j_committing_transaction) - transaction = journal->j_committing_transaction; - - if (!transaction) { - spin_unlock(&journal->j_state_lock); - return 0; /* Nothing to retry */ - } - - tid = transaction->t_tid; - spin_unlock(&journal->j_state_lock); - log_wait_commit(journal, tid); - return 1; -} - -/* - * Start a commit of the current running transaction (if any). Returns true - * if a transaction is going to be committed (or is currently already - * committing), and fills its tid in at *ptid - */ -int journal_start_commit(journal_t *journal, tid_t *ptid) -{ - int ret = 0; - - spin_lock(&journal->j_state_lock); - if (journal->j_running_transaction) { - tid_t tid = journal->j_running_transaction->t_tid; - - __log_start_commit(journal, tid); - /* There's a running transaction and we've just made sure - * it's commit has been scheduled. */ - if (ptid) - *ptid = tid; - ret = 1; - } else if (journal->j_committing_transaction) { - /* - * If commit has been started, then we have to wait for - * completion of that transaction. - */ - if (ptid) - *ptid = journal->j_committing_transaction->t_tid; - ret = 1; - } - spin_unlock(&journal->j_state_lock); - return ret; -} - -/* - * Wait for a specified commit to complete. - * The caller may not hold the journal lock. - */ -int log_wait_commit(journal_t *journal, tid_t tid) -{ - int err = 0; - -#ifdef CONFIG_JBD_DEBUG - spin_lock(&journal->j_state_lock); - if (!tid_geq(journal->j_commit_request, tid)) { - printk(KERN_ERR - "%s: error: j_commit_request=%d, tid=%d\n", - __func__, journal->j_commit_request, tid); - } - spin_unlock(&journal->j_state_lock); -#endif - spin_lock(&journal->j_state_lock); - /* - * Not running or committing trans? Must be already committed. This - * saves us from waiting for a *long* time when tid overflows. - */ - if (!((journal->j_running_transaction && - journal->j_running_transaction->t_tid == tid) || - (journal->j_committing_transaction && - journal->j_committing_transaction->t_tid == tid))) - goto out_unlock; - - if (!tid_geq(journal->j_commit_waited, tid)) - journal->j_commit_waited = tid; - while (tid_gt(tid, journal->j_commit_sequence)) { - jbd_debug(1, "JBD: want %d, j_commit_sequence=%d\n", - tid, journal->j_commit_sequence); - wake_up(&journal->j_wait_commit); - spin_unlock(&journal->j_state_lock); - wait_event(journal->j_wait_done_commit, - !tid_gt(tid, journal->j_commit_sequence)); - spin_lock(&journal->j_state_lock); - } -out_unlock: - spin_unlock(&journal->j_state_lock); - - if (unlikely(is_journal_aborted(journal))) - err = -EIO; - return err; -} - -/* - * Return 1 if a given transaction has not yet sent barrier request - * connected with a transaction commit. If 0 is returned, transaction - * may or may not have sent the barrier. Used to avoid sending barrier - * twice in common cases. - */ -int journal_trans_will_send_data_barrier(journal_t *journal, tid_t tid) -{ - int ret = 0; - transaction_t *commit_trans; - - if (!(journal->j_flags & JFS_BARRIER)) - return 0; - spin_lock(&journal->j_state_lock); - /* Transaction already committed? */ - if (tid_geq(journal->j_commit_sequence, tid)) - goto out; - /* - * Transaction is being committed and we already proceeded to - * writing commit record? - */ - commit_trans = journal->j_committing_transaction; - if (commit_trans && commit_trans->t_tid == tid && - commit_trans->t_state >= T_COMMIT_RECORD) - goto out; - ret = 1; -out: - spin_unlock(&journal->j_state_lock); - return ret; -} -EXPORT_SYMBOL(journal_trans_will_send_data_barrier); - -/* - * Log buffer allocation routines: - */ - -int journal_next_log_block(journal_t *journal, unsigned int *retp) -{ - unsigned int blocknr; - - spin_lock(&journal->j_state_lock); - J_ASSERT(journal->j_free > 1); - - blocknr = journal->j_head; - journal->j_head++; - journal->j_free--; - if (journal->j_head == journal->j_last) - journal->j_head = journal->j_first; - spin_unlock(&journal->j_state_lock); - return journal_bmap(journal, blocknr, retp); -} - -/* - * Conversion of logical to physical block numbers for the journal - * - * On external journals the journal blocks are identity-mapped, so - * this is a no-op. If needed, we can use j_blk_offset - everything is - * ready. - */ -int journal_bmap(journal_t *journal, unsigned int blocknr, - unsigned int *retp) -{ - int err = 0; - unsigned int ret; - - if (journal->j_inode) { - ret = bmap(journal->j_inode, blocknr); - if (ret) - *retp = ret; - else { - char b[BDEVNAME_SIZE]; - - printk(KERN_ALERT "%s: journal block not found " - "at offset %u on %s\n", - __func__, - blocknr, - bdevname(journal->j_dev, b)); - err = -EIO; - __journal_abort_soft(journal, err); - } - } else { - *retp = blocknr; /* +journal->j_blk_offset */ - } - return err; -} - -/* - * We play buffer_head aliasing tricks to write data/metadata blocks to - * the journal without copying their contents, but for journal - * descriptor blocks we do need to generate bona fide buffers. - * - * After the caller of journal_get_descriptor_buffer() has finished modifying - * the buffer's contents they really should run flush_dcache_page(bh->b_page). - * But we don't bother doing that, so there will be coherency problems with - * mmaps of blockdevs which hold live JBD-controlled filesystems. - */ -struct journal_head *journal_get_descriptor_buffer(journal_t *journal) -{ - struct buffer_head *bh; - unsigned int blocknr; - int err; - - err = journal_next_log_block(journal, &blocknr); - - if (err) - return NULL; - - bh = __getblk(journal->j_dev, blocknr, journal->j_blocksize); - if (!bh) - return NULL; - lock_buffer(bh); - memset(bh->b_data, 0, journal->j_blocksize); - set_buffer_uptodate(bh); - unlock_buffer(bh); - BUFFER_TRACE(bh, "return this buffer"); - return journal_add_journal_head(bh); -} - -/* - * Management for journal control blocks: functions to create and - * destroy journal_t structures, and to initialise and read existing - * journal blocks from disk. */ - -/* First: create and setup a journal_t object in memory. We initialise - * very few fields yet: that has to wait until we have created the - * journal structures from from scratch, or loaded them from disk. */ - -static journal_t * journal_init_common (void) -{ - journal_t *journal; - int err; - - journal = kzalloc(sizeof(*journal), GFP_KERNEL); - if (!journal) - goto fail; - - init_waitqueue_head(&journal->j_wait_transaction_locked); - init_waitqueue_head(&journal->j_wait_logspace); - init_waitqueue_head(&journal->j_wait_done_commit); - init_waitqueue_head(&journal->j_wait_checkpoint); - init_waitqueue_head(&journal->j_wait_commit); - init_waitqueue_head(&journal->j_wait_updates); - mutex_init(&journal->j_checkpoint_mutex); - spin_lock_init(&journal->j_revoke_lock); - spin_lock_init(&journal->j_list_lock); - spin_lock_init(&journal->j_state_lock); - - journal->j_commit_interval = (HZ * JBD_DEFAULT_MAX_COMMIT_AGE); - - /* The journal is marked for error until we succeed with recovery! */ - journal->j_flags = JFS_ABORT; - - /* Set up a default-sized revoke table for the new mount. */ - err = journal_init_revoke(journal, JOURNAL_REVOKE_DEFAULT_HASH); - if (err) { - kfree(journal); - goto fail; - } - return journal; -fail: - return NULL; -} - -/* journal_init_dev and journal_init_inode: - * - * Create a journal structure assigned some fixed set of disk blocks to - * the journal. We don't actually touch those disk blocks yet, but we - * need to set up all of the mapping information to tell the journaling - * system where the journal blocks are. - * - */ - -/** - * journal_t * journal_init_dev() - creates and initialises a journal structure - * @bdev: Block device on which to create the journal - * @fs_dev: Device which hold journalled filesystem for this journal. - * @start: Block nr Start of journal. - * @len: Length of the journal in blocks. - * @blocksize: blocksize of journalling device - * - * Returns: a newly created journal_t * - * - * journal_init_dev creates a journal which maps a fixed contiguous - * range of blocks on an arbitrary block device. - * - */ -journal_t * journal_init_dev(struct block_device *bdev, - struct block_device *fs_dev, - int start, int len, int blocksize) -{ - journal_t *journal = journal_init_common(); - struct buffer_head *bh; - int n; - - if (!journal) - return NULL; - - /* journal descriptor can store up to n blocks -bzzz */ - journal->j_blocksize = blocksize; - n = journal->j_blocksize / sizeof(journal_block_tag_t); - journal->j_wbufsize = n; - journal->j_wbuf = kmalloc(n * sizeof(struct buffer_head*), GFP_KERNEL); - if (!journal->j_wbuf) { - printk(KERN_ERR "%s: Can't allocate bhs for commit thread\n", - __func__); - goto out_err; - } - journal->j_dev = bdev; - journal->j_fs_dev = fs_dev; - journal->j_blk_offset = start; - journal->j_maxlen = len; - - bh = __getblk(journal->j_dev, start, journal->j_blocksize); - if (!bh) { - printk(KERN_ERR - "%s: Cannot get buffer for journal superblock\n", - __func__); - goto out_err; - } - journal->j_sb_buffer = bh; - journal->j_superblock = (journal_superblock_t *)bh->b_data; - - return journal; -out_err: - kfree(journal->j_wbuf); - kfree(journal); - return NULL; -} - -/** - * journal_t * journal_init_inode () - creates a journal which maps to a inode. - * @inode: An inode to create the journal in - * - * journal_init_inode creates a journal which maps an on-disk inode as - * the journal. The inode must exist already, must support bmap() and - * must have all data blocks preallocated. - */ -journal_t * journal_init_inode (struct inode *inode) -{ - struct buffer_head *bh; - journal_t *journal = journal_init_common(); - int err; - int n; - unsigned int blocknr; - - if (!journal) - return NULL; - - journal->j_dev = journal->j_fs_dev = inode->i_sb->s_bdev; - journal->j_inode = inode; - jbd_debug(1, - "journal %p: inode %s/%ld, size %Ld, bits %d, blksize %ld\n", - journal, inode->i_sb->s_id, inode->i_ino, - (long long) inode->i_size, - inode->i_sb->s_blocksize_bits, inode->i_sb->s_blocksize); - - journal->j_maxlen = inode->i_size >> inode->i_sb->s_blocksize_bits; - journal->j_blocksize = inode->i_sb->s_blocksize; - - /* journal descriptor can store up to n blocks -bzzz */ - n = journal->j_blocksize / sizeof(journal_block_tag_t); - journal->j_wbufsize = n; - journal->j_wbuf = kmalloc(n * sizeof(struct buffer_head*), GFP_KERNEL); - if (!journal->j_wbuf) { - printk(KERN_ERR "%s: Can't allocate bhs for commit thread\n", - __func__); - goto out_err; - } - - err = journal_bmap(journal, 0, &blocknr); - /* If that failed, give up */ - if (err) { - printk(KERN_ERR "%s: Cannot locate journal superblock\n", - __func__); - goto out_err; - } - - bh = getblk_unmovable(journal->j_dev, blocknr, journal->j_blocksize); - if (!bh) { - printk(KERN_ERR - "%s: Cannot get buffer for journal superblock\n", - __func__); - goto out_err; - } - journal->j_sb_buffer = bh; - journal->j_superblock = (journal_superblock_t *)bh->b_data; - - return journal; -out_err: - kfree(journal->j_wbuf); - kfree(journal); - return NULL; -} - -/* - * If the journal init or create aborts, we need to mark the journal - * superblock as being NULL to prevent the journal destroy from writing - * back a bogus superblock. - */ -static void journal_fail_superblock (journal_t *journal) -{ - struct buffer_head *bh = journal->j_sb_buffer; - brelse(bh); - journal->j_sb_buffer = NULL; -} - -/* - * Given a journal_t structure, initialise the various fields for - * startup of a new journaling session. We use this both when creating - * a journal, and after recovering an old journal to reset it for - * subsequent use. - */ - -static int journal_reset(journal_t *journal) -{ - journal_superblock_t *sb = journal->j_superblock; - unsigned int first, last; - - first = be32_to_cpu(sb->s_first); - last = be32_to_cpu(sb->s_maxlen); - if (first + JFS_MIN_JOURNAL_BLOCKS > last + 1) { - printk(KERN_ERR "JBD: Journal too short (blocks %u-%u).\n", - first, last); - journal_fail_superblock(journal); - return -EINVAL; - } - - journal->j_first = first; - journal->j_last = last; - - journal->j_head = first; - journal->j_tail = first; - journal->j_free = last - first; - - journal->j_tail_sequence = journal->j_transaction_sequence; - journal->j_commit_sequence = journal->j_transaction_sequence - 1; - journal->j_commit_request = journal->j_commit_sequence; - - journal->j_max_transaction_buffers = journal->j_maxlen / 4; - - /* - * As a special case, if the on-disk copy is already marked as needing - * no recovery (s_start == 0), then we can safely defer the superblock - * update until the next commit by setting JFS_FLUSHED. This avoids - * attempting a write to a potential-readonly device. - */ - if (sb->s_start == 0) { - jbd_debug(1,"JBD: Skipping superblock update on recovered sb " - "(start %u, seq %d, errno %d)\n", - journal->j_tail, journal->j_tail_sequence, - journal->j_errno); - journal->j_flags |= JFS_FLUSHED; - } else { - /* Lock here to make assertions happy... */ - mutex_lock(&journal->j_checkpoint_mutex); - /* - * Update log tail information. We use WRITE_FUA since new - * transaction will start reusing journal space and so we - * must make sure information about current log tail is on - * disk before that. - */ - journal_update_sb_log_tail(journal, - journal->j_tail_sequence, - journal->j_tail, - WRITE_FUA); - mutex_unlock(&journal->j_checkpoint_mutex); - } - return journal_start_thread(journal); -} - -/** - * int journal_create() - Initialise the new journal file - * @journal: Journal to create. This structure must have been initialised - * - * Given a journal_t structure which tells us which disk blocks we can - * use, create a new journal superblock and initialise all of the - * journal fields from scratch. - **/ -int journal_create(journal_t *journal) -{ - unsigned int blocknr; - struct buffer_head *bh; - journal_superblock_t *sb; - int i, err; - - if (journal->j_maxlen < JFS_MIN_JOURNAL_BLOCKS) { - printk (KERN_ERR "Journal length (%d blocks) too short.\n", - journal->j_maxlen); - journal_fail_superblock(journal); - return -EINVAL; - } - - if (journal->j_inode == NULL) { - /* - * We don't know what block to start at! - */ - printk(KERN_EMERG - "%s: creation of journal on external device!\n", - __func__); - BUG(); - } - - /* Zero out the entire journal on disk. We cannot afford to - have any blocks on disk beginning with JFS_MAGIC_NUMBER. */ - jbd_debug(1, "JBD: Zeroing out journal blocks...\n"); - for (i = 0; i < journal->j_maxlen; i++) { - err = journal_bmap(journal, i, &blocknr); - if (err) - return err; - bh = __getblk(journal->j_dev, blocknr, journal->j_blocksize); - if (unlikely(!bh)) - return -ENOMEM; - lock_buffer(bh); - memset (bh->b_data, 0, journal->j_blocksize); - BUFFER_TRACE(bh, "marking dirty"); - mark_buffer_dirty(bh); - BUFFER_TRACE(bh, "marking uptodate"); - set_buffer_uptodate(bh); - unlock_buffer(bh); - __brelse(bh); - } - - sync_blockdev(journal->j_dev); - jbd_debug(1, "JBD: journal cleared.\n"); - - /* OK, fill in the initial static fields in the new superblock */ - sb = journal->j_superblock; - - sb->s_header.h_magic = cpu_to_be32(JFS_MAGIC_NUMBER); - sb->s_header.h_blocktype = cpu_to_be32(JFS_SUPERBLOCK_V2); - - sb->s_blocksize = cpu_to_be32(journal->j_blocksize); - sb->s_maxlen = cpu_to_be32(journal->j_maxlen); - sb->s_first = cpu_to_be32(1); - - journal->j_transaction_sequence = 1; - - journal->j_flags &= ~JFS_ABORT; - journal->j_format_version = 2; - - return journal_reset(journal); -} - -static void journal_write_superblock(journal_t *journal, int write_op) -{ - struct buffer_head *bh = journal->j_sb_buffer; - int ret; - - trace_journal_write_superblock(journal, write_op); - if (!(journal->j_flags & JFS_BARRIER)) - write_op &= ~(REQ_FUA | REQ_FLUSH); - lock_buffer(bh); - if (buffer_write_io_error(bh)) { - char b[BDEVNAME_SIZE]; - /* - * Oh, dear. A previous attempt to write the journal - * superblock failed. This could happen because the - * USB device was yanked out. Or it could happen to - * be a transient write error and maybe the block will - * be remapped. Nothing we can do but to retry the - * write and hope for the best. - */ - printk(KERN_ERR "JBD: previous I/O error detected " - "for journal superblock update for %s.\n", - journal_dev_name(journal, b)); - clear_buffer_write_io_error(bh); - set_buffer_uptodate(bh); - } - - get_bh(bh); - bh->b_end_io = end_buffer_write_sync; - ret = submit_bh(write_op, bh); - wait_on_buffer(bh); - if (buffer_write_io_error(bh)) { - clear_buffer_write_io_error(bh); - set_buffer_uptodate(bh); - ret = -EIO; - } - if (ret) { - char b[BDEVNAME_SIZE]; - printk(KERN_ERR "JBD: Error %d detected " - "when updating journal superblock for %s.\n", - ret, journal_dev_name(journal, b)); - } -} - -/** - * journal_update_sb_log_tail() - Update log tail in journal sb on disk. - * @journal: The journal to update. - * @tail_tid: TID of the new transaction at the tail of the log - * @tail_block: The first block of the transaction at the tail of the log - * @write_op: With which operation should we write the journal sb - * - * Update a journal's superblock information about log tail and write it to - * disk, waiting for the IO to complete. - */ -void journal_update_sb_log_tail(journal_t *journal, tid_t tail_tid, - unsigned int tail_block, int write_op) -{ - journal_superblock_t *sb = journal->j_superblock; - - BUG_ON(!mutex_is_locked(&journal->j_checkpoint_mutex)); - jbd_debug(1,"JBD: updating superblock (start %u, seq %u)\n", - tail_block, tail_tid); - - sb->s_sequence = cpu_to_be32(tail_tid); - sb->s_start = cpu_to_be32(tail_block); - - journal_write_superblock(journal, write_op); - - /* Log is no longer empty */ - spin_lock(&journal->j_state_lock); - WARN_ON(!sb->s_sequence); - journal->j_flags &= ~JFS_FLUSHED; - spin_unlock(&journal->j_state_lock); -} - -/** - * mark_journal_empty() - Mark on disk journal as empty. - * @journal: The journal to update. - * - * Update a journal's dynamic superblock fields to show that journal is empty. - * Write updated superblock to disk waiting for IO to complete. - */ -static void mark_journal_empty(journal_t *journal) -{ - journal_superblock_t *sb = journal->j_superblock; - - BUG_ON(!mutex_is_locked(&journal->j_checkpoint_mutex)); - spin_lock(&journal->j_state_lock); - /* Is it already empty? */ - if (sb->s_start == 0) { - spin_unlock(&journal->j_state_lock); - return; - } - jbd_debug(1, "JBD: Marking journal as empty (seq %d)\n", - journal->j_tail_sequence); - - sb->s_sequence = cpu_to_be32(journal->j_tail_sequence); - sb->s_start = cpu_to_be32(0); - spin_unlock(&journal->j_state_lock); - - journal_write_superblock(journal, WRITE_FUA); - - spin_lock(&journal->j_state_lock); - /* Log is empty */ - journal->j_flags |= JFS_FLUSHED; - spin_unlock(&journal->j_state_lock); -} - -/** - * journal_update_sb_errno() - Update error in the journal. - * @journal: The journal to update. - * - * Update a journal's errno. Write updated superblock to disk waiting for IO - * to complete. - */ -static void journal_update_sb_errno(journal_t *journal) -{ - journal_superblock_t *sb = journal->j_superblock; - - spin_lock(&journal->j_state_lock); - jbd_debug(1, "JBD: updating superblock error (errno %d)\n", - journal->j_errno); - sb->s_errno = cpu_to_be32(journal->j_errno); - spin_unlock(&journal->j_state_lock); - - journal_write_superblock(journal, WRITE_SYNC); -} - -/* - * Read the superblock for a given journal, performing initial - * validation of the format. - */ - -static int journal_get_superblock(journal_t *journal) -{ - struct buffer_head *bh; - journal_superblock_t *sb; - int err = -EIO; - - bh = journal->j_sb_buffer; - - J_ASSERT(bh != NULL); - if (!buffer_uptodate(bh)) { - ll_rw_block(READ, 1, &bh); - wait_on_buffer(bh); - if (!buffer_uptodate(bh)) { - printk (KERN_ERR - "JBD: IO error reading journal superblock\n"); - goto out; - } - } - - sb = journal->j_superblock; - - err = -EINVAL; - - if (sb->s_header.h_magic != cpu_to_be32(JFS_MAGIC_NUMBER) || - sb->s_blocksize != cpu_to_be32(journal->j_blocksize)) { - printk(KERN_WARNING "JBD: no valid journal superblock found\n"); - goto out; - } - - switch(be32_to_cpu(sb->s_header.h_blocktype)) { - case JFS_SUPERBLOCK_V1: - journal->j_format_version = 1; - break; - case JFS_SUPERBLOCK_V2: - journal->j_format_version = 2; - break; - default: - printk(KERN_WARNING "JBD: unrecognised superblock format ID\n"); - goto out; - } - - if (be32_to_cpu(sb->s_maxlen) < journal->j_maxlen) - journal->j_maxlen = be32_to_cpu(sb->s_maxlen); - else if (be32_to_cpu(sb->s_maxlen) > journal->j_maxlen) { - printk (KERN_WARNING "JBD: journal file too short\n"); - goto out; - } - - if (be32_to_cpu(sb->s_first) == 0 || - be32_to_cpu(sb->s_first) >= journal->j_maxlen) { - printk(KERN_WARNING - "JBD: Invalid start block of journal: %u\n", - be32_to_cpu(sb->s_first)); - goto out; - } - - return 0; - -out: - journal_fail_superblock(journal); - return err; -} - -/* - * Load the on-disk journal superblock and read the key fields into the - * journal_t. - */ - -static int load_superblock(journal_t *journal) -{ - int err; - journal_superblock_t *sb; - - err = journal_get_superblock(journal); - if (err) - return err; - - sb = journal->j_superblock; - - journal->j_tail_sequence = be32_to_cpu(sb->s_sequence); - journal->j_tail = be32_to_cpu(sb->s_start); - journal->j_first = be32_to_cpu(sb->s_first); - journal->j_last = be32_to_cpu(sb->s_maxlen); - journal->j_errno = be32_to_cpu(sb->s_errno); - - return 0; -} - - -/** - * int journal_load() - Read journal from disk. - * @journal: Journal to act on. - * - * Given a journal_t structure which tells us which disk blocks contain - * a journal, read the journal from disk to initialise the in-memory - * structures. - */ -int journal_load(journal_t *journal) -{ - int err; - journal_superblock_t *sb; - - err = load_superblock(journal); - if (err) - return err; - - sb = journal->j_superblock; - /* If this is a V2 superblock, then we have to check the - * features flags on it. */ - - if (journal->j_format_version >= 2) { - if ((sb->s_feature_ro_compat & - ~cpu_to_be32(JFS_KNOWN_ROCOMPAT_FEATURES)) || - (sb->s_feature_incompat & - ~cpu_to_be32(JFS_KNOWN_INCOMPAT_FEATURES))) { - printk (KERN_WARNING - "JBD: Unrecognised features on journal\n"); - return -EINVAL; - } - } - - /* Let the recovery code check whether it needs to recover any - * data from the journal. */ - if (journal_recover(journal)) - goto recovery_error; - - /* OK, we've finished with the dynamic journal bits: - * reinitialise the dynamic contents of the superblock in memory - * and reset them on disk. */ - if (journal_reset(journal)) - goto recovery_error; - - journal->j_flags &= ~JFS_ABORT; - journal->j_flags |= JFS_LOADED; - return 0; - -recovery_error: - printk (KERN_WARNING "JBD: recovery failed\n"); - return -EIO; -} - -/** - * void journal_destroy() - Release a journal_t structure. - * @journal: Journal to act on. - * - * Release a journal_t structure once it is no longer in use by the - * journaled object. - * Return <0 if we couldn't clean up the journal. - */ -int journal_destroy(journal_t *journal) -{ - int err = 0; - - - /* Wait for the commit thread to wake up and die. */ - journal_kill_thread(journal); - - /* Force a final log commit */ - if (journal->j_running_transaction) - journal_commit_transaction(journal); - - /* Force any old transactions to disk */ - - /* We cannot race with anybody but must keep assertions happy */ - mutex_lock(&journal->j_checkpoint_mutex); - /* Totally anal locking here... */ - spin_lock(&journal->j_list_lock); - while (journal->j_checkpoint_transactions != NULL) { - spin_unlock(&journal->j_list_lock); - log_do_checkpoint(journal); - spin_lock(&journal->j_list_lock); - } - - J_ASSERT(journal->j_running_transaction == NULL); - J_ASSERT(journal->j_committing_transaction == NULL); - J_ASSERT(journal->j_checkpoint_transactions == NULL); - spin_unlock(&journal->j_list_lock); - - if (journal->j_sb_buffer) { - if (!is_journal_aborted(journal)) { - journal->j_tail_sequence = - ++journal->j_transaction_sequence; - mark_journal_empty(journal); - } else - err = -EIO; - brelse(journal->j_sb_buffer); - } - mutex_unlock(&journal->j_checkpoint_mutex); - - iput(journal->j_inode); - if (journal->j_revoke) - journal_destroy_revoke(journal); - kfree(journal->j_wbuf); - kfree(journal); - - return err; -} - - -/** - *int journal_check_used_features () - Check if features specified are used. - * @journal: Journal to check. - * @compat: bitmask of compatible features - * @ro: bitmask of features that force read-only mount - * @incompat: bitmask of incompatible features - * - * Check whether the journal uses all of a given set of - * features. Return true (non-zero) if it does. - **/ - -int journal_check_used_features (journal_t *journal, unsigned long compat, - unsigned long ro, unsigned long incompat) -{ - journal_superblock_t *sb; - - if (!compat && !ro && !incompat) - return 1; - if (journal->j_format_version == 1) - return 0; - - sb = journal->j_superblock; - - if (((be32_to_cpu(sb->s_feature_compat) & compat) == compat) && - ((be32_to_cpu(sb->s_feature_ro_compat) & ro) == ro) && - ((be32_to_cpu(sb->s_feature_incompat) & incompat) == incompat)) - return 1; - - return 0; -} - -/** - * int journal_check_available_features() - Check feature set in journalling layer - * @journal: Journal to check. - * @compat: bitmask of compatible features - * @ro: bitmask of features that force read-only mount - * @incompat: bitmask of incompatible features - * - * Check whether the journaling code supports the use of - * all of a given set of features on this journal. Return true - * (non-zero) if it can. */ - -int journal_check_available_features (journal_t *journal, unsigned long compat, - unsigned long ro, unsigned long incompat) -{ - if (!compat && !ro && !incompat) - return 1; - - /* We can support any known requested features iff the - * superblock is in version 2. Otherwise we fail to support any - * extended sb features. */ - - if (journal->j_format_version != 2) - return 0; - - if ((compat & JFS_KNOWN_COMPAT_FEATURES) == compat && - (ro & JFS_KNOWN_ROCOMPAT_FEATURES) == ro && - (incompat & JFS_KNOWN_INCOMPAT_FEATURES) == incompat) - return 1; - - return 0; -} - -/** - * int journal_set_features () - Mark a given journal feature in the superblock - * @journal: Journal to act on. - * @compat: bitmask of compatible features - * @ro: bitmask of features that force read-only mount - * @incompat: bitmask of incompatible features - * - * Mark a given journal feature as present on the - * superblock. Returns true if the requested features could be set. - * - */ - -int journal_set_features (journal_t *journal, unsigned long compat, - unsigned long ro, unsigned long incompat) -{ - journal_superblock_t *sb; - - if (journal_check_used_features(journal, compat, ro, incompat)) - return 1; - - if (!journal_check_available_features(journal, compat, ro, incompat)) - return 0; - - jbd_debug(1, "Setting new features 0x%lx/0x%lx/0x%lx\n", - compat, ro, incompat); - - sb = journal->j_superblock; - - sb->s_feature_compat |= cpu_to_be32(compat); - sb->s_feature_ro_compat |= cpu_to_be32(ro); - sb->s_feature_incompat |= cpu_to_be32(incompat); - - return 1; -} - - -/** - * int journal_update_format () - Update on-disk journal structure. - * @journal: Journal to act on. - * - * Given an initialised but unloaded journal struct, poke about in the - * on-disk structure to update it to the most recent supported version. - */ -int journal_update_format (journal_t *journal) -{ - journal_superblock_t *sb; - int err; - - err = journal_get_superblock(journal); - if (err) - return err; - - sb = journal->j_superblock; - - switch (be32_to_cpu(sb->s_header.h_blocktype)) { - case JFS_SUPERBLOCK_V2: - return 0; - case JFS_SUPERBLOCK_V1: - return journal_convert_superblock_v1(journal, sb); - default: - break; - } - return -EINVAL; -} - -static int journal_convert_superblock_v1(journal_t *journal, - journal_superblock_t *sb) -{ - int offset, blocksize; - struct buffer_head *bh; - - printk(KERN_WARNING - "JBD: Converting superblock from version 1 to 2.\n"); - - /* Pre-initialise new fields to zero */ - offset = ((char *) &(sb->s_feature_compat)) - ((char *) sb); - blocksize = be32_to_cpu(sb->s_blocksize); - memset(&sb->s_feature_compat, 0, blocksize-offset); - - sb->s_nr_users = cpu_to_be32(1); - sb->s_header.h_blocktype = cpu_to_be32(JFS_SUPERBLOCK_V2); - journal->j_format_version = 2; - - bh = journal->j_sb_buffer; - BUFFER_TRACE(bh, "marking dirty"); - mark_buffer_dirty(bh); - sync_dirty_buffer(bh); - return 0; -} - - -/** - * int journal_flush () - Flush journal - * @journal: Journal to act on. - * - * Flush all data for a given journal to disk and empty the journal. - * Filesystems can use this when remounting readonly to ensure that - * recovery does not need to happen on remount. - */ - -int journal_flush(journal_t *journal) -{ - int err = 0; - transaction_t *transaction = NULL; - - spin_lock(&journal->j_state_lock); - - /* Force everything buffered to the log... */ - if (journal->j_running_transaction) { - transaction = journal->j_running_transaction; - __log_start_commit(journal, transaction->t_tid); - } else if (journal->j_committing_transaction) - transaction = journal->j_committing_transaction; - - /* Wait for the log commit to complete... */ - if (transaction) { - tid_t tid = transaction->t_tid; - - spin_unlock(&journal->j_state_lock); - log_wait_commit(journal, tid); - } else { - spin_unlock(&journal->j_state_lock); - } - - /* ...and flush everything in the log out to disk. */ - spin_lock(&journal->j_list_lock); - while (!err && journal->j_checkpoint_transactions != NULL) { - spin_unlock(&journal->j_list_lock); - mutex_lock(&journal->j_checkpoint_mutex); - err = log_do_checkpoint(journal); - mutex_unlock(&journal->j_checkpoint_mutex); - spin_lock(&journal->j_list_lock); - } - spin_unlock(&journal->j_list_lock); - - if (is_journal_aborted(journal)) - return -EIO; - - mutex_lock(&journal->j_checkpoint_mutex); - cleanup_journal_tail(journal); - - /* Finally, mark the journal as really needing no recovery. - * This sets s_start==0 in the underlying superblock, which is - * the magic code for a fully-recovered superblock. Any future - * commits of data to the journal will restore the current - * s_start value. */ - mark_journal_empty(journal); - mutex_unlock(&journal->j_checkpoint_mutex); - spin_lock(&journal->j_state_lock); - J_ASSERT(!journal->j_running_transaction); - J_ASSERT(!journal->j_committing_transaction); - J_ASSERT(!journal->j_checkpoint_transactions); - J_ASSERT(journal->j_head == journal->j_tail); - J_ASSERT(journal->j_tail_sequence == journal->j_transaction_sequence); - spin_unlock(&journal->j_state_lock); - return 0; -} - -/** - * int journal_wipe() - Wipe journal contents - * @journal: Journal to act on. - * @write: flag (see below) - * - * Wipe out all of the contents of a journal, safely. This will produce - * a warning if the journal contains any valid recovery information. - * Must be called between journal_init_*() and journal_load(). - * - * If 'write' is non-zero, then we wipe out the journal on disk; otherwise - * we merely suppress recovery. - */ - -int journal_wipe(journal_t *journal, int write) -{ - int err = 0; - - J_ASSERT (!(journal->j_flags & JFS_LOADED)); - - err = load_superblock(journal); - if (err) - return err; - - if (!journal->j_tail) - goto no_recovery; - - printk (KERN_WARNING "JBD: %s recovery information on journal\n", - write ? "Clearing" : "Ignoring"); - - err = journal_skip_recovery(journal); - if (write) { - /* Lock to make assertions happy... */ - mutex_lock(&journal->j_checkpoint_mutex); - mark_journal_empty(journal); - mutex_unlock(&journal->j_checkpoint_mutex); - } - - no_recovery: - return err; -} - -/* - * journal_dev_name: format a character string to describe on what - * device this journal is present. - */ - -static const char *journal_dev_name(journal_t *journal, char *buffer) -{ - struct block_device *bdev; - - if (journal->j_inode) - bdev = journal->j_inode->i_sb->s_bdev; - else - bdev = journal->j_dev; - - return bdevname(bdev, buffer); -} - -/* - * Journal abort has very specific semantics, which we describe - * for journal abort. - * - * Two internal function, which provide abort to te jbd layer - * itself are here. - */ - -/* - * Quick version for internal journal use (doesn't lock the journal). - * Aborts hard --- we mark the abort as occurred, but do _nothing_ else, - * and don't attempt to make any other journal updates. - */ -static void __journal_abort_hard(journal_t *journal) -{ - transaction_t *transaction; - char b[BDEVNAME_SIZE]; - - if (journal->j_flags & JFS_ABORT) - return; - - printk(KERN_ERR "Aborting journal on device %s.\n", - journal_dev_name(journal, b)); - - spin_lock(&journal->j_state_lock); - journal->j_flags |= JFS_ABORT; - transaction = journal->j_running_transaction; - if (transaction) - __log_start_commit(journal, transaction->t_tid); - spin_unlock(&journal->j_state_lock); -} - -/* Soft abort: record the abort error status in the journal superblock, - * but don't do any other IO. */ -static void __journal_abort_soft (journal_t *journal, int errno) -{ - if (journal->j_flags & JFS_ABORT) - return; - - if (!journal->j_errno) - journal->j_errno = errno; - - __journal_abort_hard(journal); - - if (errno) - journal_update_sb_errno(journal); -} - -/** - * void journal_abort () - Shutdown the journal immediately. - * @journal: the journal to shutdown. - * @errno: an error number to record in the journal indicating - * the reason for the shutdown. - * - * Perform a complete, immediate shutdown of the ENTIRE - * journal (not of a single transaction). This operation cannot be - * undone without closing and reopening the journal. - * - * The journal_abort function is intended to support higher level error - * recovery mechanisms such as the ext2/ext3 remount-readonly error - * mode. - * - * Journal abort has very specific semantics. Any existing dirty, - * unjournaled buffers in the main filesystem will still be written to - * disk by bdflush, but the journaling mechanism will be suspended - * immediately and no further transaction commits will be honoured. - * - * Any dirty, journaled buffers will be written back to disk without - * hitting the journal. Atomicity cannot be guaranteed on an aborted - * filesystem, but we _do_ attempt to leave as much data as possible - * behind for fsck to use for cleanup. - * - * Any attempt to get a new transaction handle on a journal which is in - * ABORT state will just result in an -EROFS error return. A - * journal_stop on an existing handle will return -EIO if we have - * entered abort state during the update. - * - * Recursive transactions are not disturbed by journal abort until the - * final journal_stop, which will receive the -EIO error. - * - * Finally, the journal_abort call allows the caller to supply an errno - * which will be recorded (if possible) in the journal superblock. This - * allows a client to record failure conditions in the middle of a - * transaction without having to complete the transaction to record the - * failure to disk. ext3_error, for example, now uses this - * functionality. - * - * Errors which originate from within the journaling layer will NOT - * supply an errno; a null errno implies that absolutely no further - * writes are done to the journal (unless there are any already in - * progress). - * - */ - -void journal_abort(journal_t *journal, int errno) -{ - __journal_abort_soft(journal, errno); -} - -/** - * int journal_errno () - returns the journal's error state. - * @journal: journal to examine. - * - * This is the errno numbet set with journal_abort(), the last - * time the journal was mounted - if the journal was stopped - * without calling abort this will be 0. - * - * If the journal has been aborted on this mount time -EROFS will - * be returned. - */ -int journal_errno(journal_t *journal) -{ - int err; - - spin_lock(&journal->j_state_lock); - if (journal->j_flags & JFS_ABORT) - err = -EROFS; - else - err = journal->j_errno; - spin_unlock(&journal->j_state_lock); - return err; -} - -/** - * int journal_clear_err () - clears the journal's error state - * @journal: journal to act on. - * - * An error must be cleared or Acked to take a FS out of readonly - * mode. - */ -int journal_clear_err(journal_t *journal) -{ - int err = 0; - - spin_lock(&journal->j_state_lock); - if (journal->j_flags & JFS_ABORT) - err = -EROFS; - else - journal->j_errno = 0; - spin_unlock(&journal->j_state_lock); - return err; -} - -/** - * void journal_ack_err() - Ack journal err. - * @journal: journal to act on. - * - * An error must be cleared or Acked to take a FS out of readonly - * mode. - */ -void journal_ack_err(journal_t *journal) -{ - spin_lock(&journal->j_state_lock); - if (journal->j_errno) - journal->j_flags |= JFS_ACK_ERR; - spin_unlock(&journal->j_state_lock); -} - -int journal_blocks_per_page(struct inode *inode) -{ - return 1 << (PAGE_CACHE_SHIFT - inode->i_sb->s_blocksize_bits); -} - -/* - * Journal_head storage management - */ -static struct kmem_cache *journal_head_cache; -#ifdef CONFIG_JBD_DEBUG -static atomic_t nr_journal_heads = ATOMIC_INIT(0); -#endif - -static int journal_init_journal_head_cache(void) -{ - int retval; - - J_ASSERT(journal_head_cache == NULL); - journal_head_cache = kmem_cache_create("journal_head", - sizeof(struct journal_head), - 0, /* offset */ - SLAB_TEMPORARY, /* flags */ - NULL); /* ctor */ - retval = 0; - if (!journal_head_cache) { - retval = -ENOMEM; - printk(KERN_EMERG "JBD: no memory for journal_head cache\n"); - } - return retval; -} - -static void journal_destroy_journal_head_cache(void) -{ - if (journal_head_cache) { - kmem_cache_destroy(journal_head_cache); - journal_head_cache = NULL; - } -} - -/* - * journal_head splicing and dicing - */ -static struct journal_head *journal_alloc_journal_head(void) -{ - struct journal_head *ret; - -#ifdef CONFIG_JBD_DEBUG - atomic_inc(&nr_journal_heads); -#endif - ret = kmem_cache_zalloc(journal_head_cache, GFP_NOFS); - if (ret == NULL) { - jbd_debug(1, "out of memory for journal_head\n"); - printk_ratelimited(KERN_NOTICE "ENOMEM in %s, retrying.\n", - __func__); - - while (ret == NULL) { - yield(); - ret = kmem_cache_zalloc(journal_head_cache, GFP_NOFS); - } - } - return ret; -} - -static void journal_free_journal_head(struct journal_head *jh) -{ -#ifdef CONFIG_JBD_DEBUG - atomic_dec(&nr_journal_heads); - memset(jh, JBD_POISON_FREE, sizeof(*jh)); -#endif - kmem_cache_free(journal_head_cache, jh); -} - -/* - * A journal_head is attached to a buffer_head whenever JBD has an - * interest in the buffer. - * - * Whenever a buffer has an attached journal_head, its ->b_state:BH_JBD bit - * is set. This bit is tested in core kernel code where we need to take - * JBD-specific actions. Testing the zeroness of ->b_private is not reliable - * there. - * - * When a buffer has its BH_JBD bit set, its ->b_count is elevated by one. - * - * When a buffer has its BH_JBD bit set it is immune from being released by - * core kernel code, mainly via ->b_count. - * - * A journal_head is detached from its buffer_head when the journal_head's - * b_jcount reaches zero. Running transaction (b_transaction) and checkpoint - * transaction (b_cp_transaction) hold their references to b_jcount. - * - * Various places in the kernel want to attach a journal_head to a buffer_head - * _before_ attaching the journal_head to a transaction. To protect the - * journal_head in this situation, journal_add_journal_head elevates the - * journal_head's b_jcount refcount by one. The caller must call - * journal_put_journal_head() to undo this. - * - * So the typical usage would be: - * - * (Attach a journal_head if needed. Increments b_jcount) - * struct journal_head *jh = journal_add_journal_head(bh); - * ... - * (Get another reference for transaction) - * journal_grab_journal_head(bh); - * jh->b_transaction = xxx; - * (Put original reference) - * journal_put_journal_head(jh); - */ - -/* - * Give a buffer_head a journal_head. - * - * May sleep. - */ -struct journal_head *journal_add_journal_head(struct buffer_head *bh) -{ - struct journal_head *jh; - struct journal_head *new_jh = NULL; - -repeat: - if (!buffer_jbd(bh)) - new_jh = journal_alloc_journal_head(); - - jbd_lock_bh_journal_head(bh); - if (buffer_jbd(bh)) { - jh = bh2jh(bh); - } else { - J_ASSERT_BH(bh, - (atomic_read(&bh->b_count) > 0) || - (bh->b_page && bh->b_page->mapping)); - - if (!new_jh) { - jbd_unlock_bh_journal_head(bh); - goto repeat; - } - - jh = new_jh; - new_jh = NULL; /* We consumed it */ - set_buffer_jbd(bh); - bh->b_private = jh; - jh->b_bh = bh; - get_bh(bh); - BUFFER_TRACE(bh, "added journal_head"); - } - jh->b_jcount++; - jbd_unlock_bh_journal_head(bh); - if (new_jh) - journal_free_journal_head(new_jh); - return bh->b_private; -} - -/* - * Grab a ref against this buffer_head's journal_head. If it ended up not - * having a journal_head, return NULL - */ -struct journal_head *journal_grab_journal_head(struct buffer_head *bh) -{ - struct journal_head *jh = NULL; - - jbd_lock_bh_journal_head(bh); - if (buffer_jbd(bh)) { - jh = bh2jh(bh); - jh->b_jcount++; - } - jbd_unlock_bh_journal_head(bh); - return jh; -} - -static void __journal_remove_journal_head(struct buffer_head *bh) -{ - struct journal_head *jh = bh2jh(bh); - - J_ASSERT_JH(jh, jh->b_jcount >= 0); - J_ASSERT_JH(jh, jh->b_transaction == NULL); - J_ASSERT_JH(jh, jh->b_next_transaction == NULL); - J_ASSERT_JH(jh, jh->b_cp_transaction == NULL); - J_ASSERT_JH(jh, jh->b_jlist == BJ_None); - J_ASSERT_BH(bh, buffer_jbd(bh)); - J_ASSERT_BH(bh, jh2bh(jh) == bh); - BUFFER_TRACE(bh, "remove journal_head"); - if (jh->b_frozen_data) { - printk(KERN_WARNING "%s: freeing b_frozen_data\n", __func__); - jbd_free(jh->b_frozen_data, bh->b_size); - } - if (jh->b_committed_data) { - printk(KERN_WARNING "%s: freeing b_committed_data\n", __func__); - jbd_free(jh->b_committed_data, bh->b_size); - } - bh->b_private = NULL; - jh->b_bh = NULL; /* debug, really */ - clear_buffer_jbd(bh); - journal_free_journal_head(jh); -} - -/* - * Drop a reference on the passed journal_head. If it fell to zero then - * release the journal_head from the buffer_head. - */ -void journal_put_journal_head(struct journal_head *jh) -{ - struct buffer_head *bh = jh2bh(jh); - - jbd_lock_bh_journal_head(bh); - J_ASSERT_JH(jh, jh->b_jcount > 0); - --jh->b_jcount; - if (!jh->b_jcount) { - __journal_remove_journal_head(bh); - jbd_unlock_bh_journal_head(bh); - __brelse(bh); - } else - jbd_unlock_bh_journal_head(bh); -} - -/* - * debugfs tunables - */ -#ifdef CONFIG_JBD_DEBUG - -u8 journal_enable_debug __read_mostly; -EXPORT_SYMBOL(journal_enable_debug); - -static struct dentry *jbd_debugfs_dir; -static struct dentry *jbd_debug; - -static void __init jbd_create_debugfs_entry(void) -{ - jbd_debugfs_dir = debugfs_create_dir("jbd", NULL); - if (jbd_debugfs_dir) - jbd_debug = debugfs_create_u8("jbd-debug", S_IRUGO | S_IWUSR, - jbd_debugfs_dir, - &journal_enable_debug); -} - -static void __exit jbd_remove_debugfs_entry(void) -{ - debugfs_remove(jbd_debug); - debugfs_remove(jbd_debugfs_dir); -} - -#else - -static inline void jbd_create_debugfs_entry(void) -{ -} - -static inline void jbd_remove_debugfs_entry(void) -{ -} - -#endif - -struct kmem_cache *jbd_handle_cache; - -static int __init journal_init_handle_cache(void) -{ - jbd_handle_cache = kmem_cache_create("journal_handle", - sizeof(handle_t), - 0, /* offset */ - SLAB_TEMPORARY, /* flags */ - NULL); /* ctor */ - if (jbd_handle_cache == NULL) { - printk(KERN_EMERG "JBD: failed to create handle cache\n"); - return -ENOMEM; - } - return 0; -} - -static void journal_destroy_handle_cache(void) -{ - if (jbd_handle_cache) - kmem_cache_destroy(jbd_handle_cache); -} - -/* - * Module startup and shutdown - */ - -static int __init journal_init_caches(void) -{ - int ret; - - ret = journal_init_revoke_caches(); - if (ret == 0) - ret = journal_init_journal_head_cache(); - if (ret == 0) - ret = journal_init_handle_cache(); - return ret; -} - -static void journal_destroy_caches(void) -{ - journal_destroy_revoke_caches(); - journal_destroy_journal_head_cache(); - journal_destroy_handle_cache(); -} - -static int __init journal_init(void) -{ - int ret; - - BUILD_BUG_ON(sizeof(struct journal_superblock_s) != 1024); - - ret = journal_init_caches(); - if (ret != 0) - journal_destroy_caches(); - jbd_create_debugfs_entry(); - return ret; -} - -static void __exit journal_exit(void) -{ -#ifdef CONFIG_JBD_DEBUG - int n = atomic_read(&nr_journal_heads); - if (n) - printk(KERN_ERR "JBD: leaked %d journal_heads!\n", n); -#endif - jbd_remove_debugfs_entry(); - journal_destroy_caches(); -} - -MODULE_LICENSE("GPL"); -module_init(journal_init); -module_exit(journal_exit); - diff --git a/fs/jbd/recovery.c b/fs/jbd/recovery.c deleted file mode 100644 index a748fe21465abd..00000000000000 --- a/fs/jbd/recovery.c +++ /dev/null @@ -1,594 +0,0 @@ -/* - * linux/fs/jbd/recovery.c - * - * Written by Stephen C. Tweedie , 1999 - * - * Copyright 1999-2000 Red Hat Software --- All Rights Reserved - * - * This file is part of the Linux kernel and is made available under - * the terms of the GNU General Public License, version 2, or at your - * option, any later version, incorporated herein by reference. - * - * Journal recovery routines for the generic filesystem journaling code; - * part of the ext2fs journaling system. - */ - -#ifndef __KERNEL__ -#include "jfs_user.h" -#else -#include -#include -#include -#include -#include -#endif - -/* - * Maintain information about the progress of the recovery job, so that - * the different passes can carry information between them. - */ -struct recovery_info -{ - tid_t start_transaction; - tid_t end_transaction; - - int nr_replays; - int nr_revokes; - int nr_revoke_hits; -}; - -enum passtype {PASS_SCAN, PASS_REVOKE, PASS_REPLAY}; -static int do_one_pass(journal_t *journal, - struct recovery_info *info, enum passtype pass); -static int scan_revoke_records(journal_t *, struct buffer_head *, - tid_t, struct recovery_info *); - -#ifdef __KERNEL__ - -/* Release readahead buffers after use */ -static void journal_brelse_array(struct buffer_head *b[], int n) -{ - while (--n >= 0) - brelse (b[n]); -} - - -/* - * When reading from the journal, we are going through the block device - * layer directly and so there is no readahead being done for us. We - * need to implement any readahead ourselves if we want it to happen at - * all. Recovery is basically one long sequential read, so make sure we - * do the IO in reasonably large chunks. - * - * This is not so critical that we need to be enormously clever about - * the readahead size, though. 128K is a purely arbitrary, good-enough - * fixed value. - */ - -#define MAXBUF 8 -static int do_readahead(journal_t *journal, unsigned int start) -{ - int err; - unsigned int max, nbufs, next; - unsigned int blocknr; - struct buffer_head *bh; - - struct buffer_head * bufs[MAXBUF]; - - /* Do up to 128K of readahead */ - max = start + (128 * 1024 / journal->j_blocksize); - if (max > journal->j_maxlen) - max = journal->j_maxlen; - - /* Do the readahead itself. We'll submit MAXBUF buffer_heads at - * a time to the block device IO layer. */ - - nbufs = 0; - - for (next = start; next < max; next++) { - err = journal_bmap(journal, next, &blocknr); - - if (err) { - printk (KERN_ERR "JBD: bad block at offset %u\n", - next); - goto failed; - } - - bh = __getblk(journal->j_dev, blocknr, journal->j_blocksize); - if (!bh) { - err = -ENOMEM; - goto failed; - } - - if (!buffer_uptodate(bh) && !buffer_locked(bh)) { - bufs[nbufs++] = bh; - if (nbufs == MAXBUF) { - ll_rw_block(READ, nbufs, bufs); - journal_brelse_array(bufs, nbufs); - nbufs = 0; - } - } else - brelse(bh); - } - - if (nbufs) - ll_rw_block(READ, nbufs, bufs); - err = 0; - -failed: - if (nbufs) - journal_brelse_array(bufs, nbufs); - return err; -} - -#endif /* __KERNEL__ */ - - -/* - * Read a block from the journal - */ - -static int jread(struct buffer_head **bhp, journal_t *journal, - unsigned int offset) -{ - int err; - unsigned int blocknr; - struct buffer_head *bh; - - *bhp = NULL; - - if (offset >= journal->j_maxlen) { - printk(KERN_ERR "JBD: corrupted journal superblock\n"); - return -EIO; - } - - err = journal_bmap(journal, offset, &blocknr); - - if (err) { - printk (KERN_ERR "JBD: bad block at offset %u\n", - offset); - return err; - } - - bh = __getblk(journal->j_dev, blocknr, journal->j_blocksize); - if (!bh) - return -ENOMEM; - - if (!buffer_uptodate(bh)) { - /* If this is a brand new buffer, start readahead. - Otherwise, we assume we are already reading it. */ - if (!buffer_req(bh)) - do_readahead(journal, offset); - wait_on_buffer(bh); - } - - if (!buffer_uptodate(bh)) { - printk (KERN_ERR "JBD: Failed to read block at offset %u\n", - offset); - brelse(bh); - return -EIO; - } - - *bhp = bh; - return 0; -} - - -/* - * Count the number of in-use tags in a journal descriptor block. - */ - -static int count_tags(struct buffer_head *bh, int size) -{ - char * tagp; - journal_block_tag_t * tag; - int nr = 0; - - tagp = &bh->b_data[sizeof(journal_header_t)]; - - while ((tagp - bh->b_data + sizeof(journal_block_tag_t)) <= size) { - tag = (journal_block_tag_t *) tagp; - - nr++; - tagp += sizeof(journal_block_tag_t); - if (!(tag->t_flags & cpu_to_be32(JFS_FLAG_SAME_UUID))) - tagp += 16; - - if (tag->t_flags & cpu_to_be32(JFS_FLAG_LAST_TAG)) - break; - } - - return nr; -} - - -/* Make sure we wrap around the log correctly! */ -#define wrap(journal, var) \ -do { \ - if (var >= (journal)->j_last) \ - var -= ((journal)->j_last - (journal)->j_first); \ -} while (0) - -/** - * journal_recover - recovers a on-disk journal - * @journal: the journal to recover - * - * The primary function for recovering the log contents when mounting a - * journaled device. - * - * Recovery is done in three passes. In the first pass, we look for the - * end of the log. In the second, we assemble the list of revoke - * blocks. In the third and final pass, we replay any un-revoked blocks - * in the log. - */ -int journal_recover(journal_t *journal) -{ - int err, err2; - journal_superblock_t * sb; - - struct recovery_info info; - - memset(&info, 0, sizeof(info)); - sb = journal->j_superblock; - - /* - * The journal superblock's s_start field (the current log head) - * is always zero if, and only if, the journal was cleanly - * unmounted. - */ - - if (!sb->s_start) { - jbd_debug(1, "No recovery required, last transaction %d\n", - be32_to_cpu(sb->s_sequence)); - journal->j_transaction_sequence = be32_to_cpu(sb->s_sequence) + 1; - return 0; - } - - err = do_one_pass(journal, &info, PASS_SCAN); - if (!err) - err = do_one_pass(journal, &info, PASS_REVOKE); - if (!err) - err = do_one_pass(journal, &info, PASS_REPLAY); - - jbd_debug(1, "JBD: recovery, exit status %d, " - "recovered transactions %u to %u\n", - err, info.start_transaction, info.end_transaction); - jbd_debug(1, "JBD: Replayed %d and revoked %d/%d blocks\n", - info.nr_replays, info.nr_revoke_hits, info.nr_revokes); - - /* Restart the log at the next transaction ID, thus invalidating - * any existing commit records in the log. */ - journal->j_transaction_sequence = ++info.end_transaction; - - journal_clear_revoke(journal); - err2 = sync_blockdev(journal->j_fs_dev); - if (!err) - err = err2; - /* Flush disk caches to get replayed data on the permanent storage */ - if (journal->j_flags & JFS_BARRIER) { - err2 = blkdev_issue_flush(journal->j_fs_dev, GFP_KERNEL, NULL); - if (!err) - err = err2; - } - - return err; -} - -/** - * journal_skip_recovery - Start journal and wipe exiting records - * @journal: journal to startup - * - * Locate any valid recovery information from the journal and set up the - * journal structures in memory to ignore it (presumably because the - * caller has evidence that it is out of date). - * This function does'nt appear to be exorted.. - * - * We perform one pass over the journal to allow us to tell the user how - * much recovery information is being erased, and to let us initialise - * the journal transaction sequence numbers to the next unused ID. - */ -int journal_skip_recovery(journal_t *journal) -{ - int err; - struct recovery_info info; - - memset (&info, 0, sizeof(info)); - - err = do_one_pass(journal, &info, PASS_SCAN); - - if (err) { - printk(KERN_ERR "JBD: error %d scanning journal\n", err); - ++journal->j_transaction_sequence; - } else { -#ifdef CONFIG_JBD_DEBUG - int dropped = info.end_transaction - - be32_to_cpu(journal->j_superblock->s_sequence); - jbd_debug(1, - "JBD: ignoring %d transaction%s from the journal.\n", - dropped, (dropped == 1) ? "" : "s"); -#endif - journal->j_transaction_sequence = ++info.end_transaction; - } - - journal->j_tail = 0; - return err; -} - -static int do_one_pass(journal_t *journal, - struct recovery_info *info, enum passtype pass) -{ - unsigned int first_commit_ID, next_commit_ID; - unsigned int next_log_block; - int err, success = 0; - journal_superblock_t * sb; - journal_header_t * tmp; - struct buffer_head * bh; - unsigned int sequence; - int blocktype; - - /* - * First thing is to establish what we expect to find in the log - * (in terms of transaction IDs), and where (in terms of log - * block offsets): query the superblock. - */ - - sb = journal->j_superblock; - next_commit_ID = be32_to_cpu(sb->s_sequence); - next_log_block = be32_to_cpu(sb->s_start); - - first_commit_ID = next_commit_ID; - if (pass == PASS_SCAN) - info->start_transaction = first_commit_ID; - - jbd_debug(1, "Starting recovery pass %d\n", pass); - - /* - * Now we walk through the log, transaction by transaction, - * making sure that each transaction has a commit block in the - * expected place. Each complete transaction gets replayed back - * into the main filesystem. - */ - - while (1) { - int flags; - char * tagp; - journal_block_tag_t * tag; - struct buffer_head * obh; - struct buffer_head * nbh; - - cond_resched(); - - /* If we already know where to stop the log traversal, - * check right now that we haven't gone past the end of - * the log. */ - - if (pass != PASS_SCAN) - if (tid_geq(next_commit_ID, info->end_transaction)) - break; - - jbd_debug(2, "Scanning for sequence ID %u at %u/%u\n", - next_commit_ID, next_log_block, journal->j_last); - - /* Skip over each chunk of the transaction looking - * either the next descriptor block or the final commit - * record. */ - - jbd_debug(3, "JBD: checking block %u\n", next_log_block); - err = jread(&bh, journal, next_log_block); - if (err) - goto failed; - - next_log_block++; - wrap(journal, next_log_block); - - /* What kind of buffer is it? - * - * If it is a descriptor block, check that it has the - * expected sequence number. Otherwise, we're all done - * here. */ - - tmp = (journal_header_t *)bh->b_data; - - if (tmp->h_magic != cpu_to_be32(JFS_MAGIC_NUMBER)) { - brelse(bh); - break; - } - - blocktype = be32_to_cpu(tmp->h_blocktype); - sequence = be32_to_cpu(tmp->h_sequence); - jbd_debug(3, "Found magic %d, sequence %d\n", - blocktype, sequence); - - if (sequence != next_commit_ID) { - brelse(bh); - break; - } - - /* OK, we have a valid descriptor block which matches - * all of the sequence number checks. What are we going - * to do with it? That depends on the pass... */ - - switch(blocktype) { - case JFS_DESCRIPTOR_BLOCK: - /* If it is a valid descriptor block, replay it - * in pass REPLAY; otherwise, just skip over the - * blocks it describes. */ - if (pass != PASS_REPLAY) { - next_log_block += - count_tags(bh, journal->j_blocksize); - wrap(journal, next_log_block); - brelse(bh); - continue; - } - - /* A descriptor block: we can now write all of - * the data blocks. Yay, useful work is finally - * getting done here! */ - - tagp = &bh->b_data[sizeof(journal_header_t)]; - while ((tagp - bh->b_data +sizeof(journal_block_tag_t)) - <= journal->j_blocksize) { - unsigned int io_block; - - tag = (journal_block_tag_t *) tagp; - flags = be32_to_cpu(tag->t_flags); - - io_block = next_log_block++; - wrap(journal, next_log_block); - err = jread(&obh, journal, io_block); - if (err) { - /* Recover what we can, but - * report failure at the end. */ - success = err; - printk (KERN_ERR - "JBD: IO error %d recovering " - "block %u in log\n", - err, io_block); - } else { - unsigned int blocknr; - - J_ASSERT(obh != NULL); - blocknr = be32_to_cpu(tag->t_blocknr); - - /* If the block has been - * revoked, then we're all done - * here. */ - if (journal_test_revoke - (journal, blocknr, - next_commit_ID)) { - brelse(obh); - ++info->nr_revoke_hits; - goto skip_write; - } - - /* Find a buffer for the new - * data being restored */ - nbh = __getblk(journal->j_fs_dev, - blocknr, - journal->j_blocksize); - if (nbh == NULL) { - printk(KERN_ERR - "JBD: Out of memory " - "during recovery.\n"); - err = -ENOMEM; - brelse(bh); - brelse(obh); - goto failed; - } - - lock_buffer(nbh); - memcpy(nbh->b_data, obh->b_data, - journal->j_blocksize); - if (flags & JFS_FLAG_ESCAPE) { - *((__be32 *)nbh->b_data) = - cpu_to_be32(JFS_MAGIC_NUMBER); - } - - BUFFER_TRACE(nbh, "marking dirty"); - set_buffer_uptodate(nbh); - mark_buffer_dirty(nbh); - BUFFER_TRACE(nbh, "marking uptodate"); - ++info->nr_replays; - /* ll_rw_block(WRITE, 1, &nbh); */ - unlock_buffer(nbh); - brelse(obh); - brelse(nbh); - } - - skip_write: - tagp += sizeof(journal_block_tag_t); - if (!(flags & JFS_FLAG_SAME_UUID)) - tagp += 16; - - if (flags & JFS_FLAG_LAST_TAG) - break; - } - - brelse(bh); - continue; - - case JFS_COMMIT_BLOCK: - /* Found an expected commit block: not much to - * do other than move on to the next sequence - * number. */ - brelse(bh); - next_commit_ID++; - continue; - - case JFS_REVOKE_BLOCK: - /* If we aren't in the REVOKE pass, then we can - * just skip over this block. */ - if (pass != PASS_REVOKE) { - brelse(bh); - continue; - } - - err = scan_revoke_records(journal, bh, - next_commit_ID, info); - brelse(bh); - if (err) - goto failed; - continue; - - default: - jbd_debug(3, "Unrecognised magic %d, end of scan.\n", - blocktype); - brelse(bh); - goto done; - } - } - - done: - /* - * We broke out of the log scan loop: either we came to the - * known end of the log or we found an unexpected block in the - * log. If the latter happened, then we know that the "current" - * transaction marks the end of the valid log. - */ - - if (pass == PASS_SCAN) - info->end_transaction = next_commit_ID; - else { - /* It's really bad news if different passes end up at - * different places (but possible due to IO errors). */ - if (info->end_transaction != next_commit_ID) { - printk (KERN_ERR "JBD: recovery pass %d ended at " - "transaction %u, expected %u\n", - pass, next_commit_ID, info->end_transaction); - if (!success) - success = -EIO; - } - } - - return success; - - failed: - return err; -} - - -/* Scan a revoke record, marking all blocks mentioned as revoked. */ - -static int scan_revoke_records(journal_t *journal, struct buffer_head *bh, - tid_t sequence, struct recovery_info *info) -{ - journal_revoke_header_t *header; - int offset, max; - - header = (journal_revoke_header_t *) bh->b_data; - offset = sizeof(journal_revoke_header_t); - max = be32_to_cpu(header->r_count); - - while (offset < max) { - unsigned int blocknr; - int err; - - blocknr = be32_to_cpu(* ((__be32 *) (bh->b_data+offset))); - offset += 4; - err = journal_set_revoke(journal, blocknr, sequence); - if (err) - return err; - ++info->nr_revokes; - } - return 0; -} diff --git a/fs/jbd/revoke.c b/fs/jbd/revoke.c deleted file mode 100644 index dcead636c33b5e..00000000000000 --- a/fs/jbd/revoke.c +++ /dev/null @@ -1,733 +0,0 @@ -/* - * linux/fs/jbd/revoke.c - * - * Written by Stephen C. Tweedie , 2000 - * - * Copyright 2000 Red Hat corp --- All Rights Reserved - * - * This file is part of the Linux kernel and is made available under - * the terms of the GNU General Public License, version 2, or at your - * option, any later version, incorporated herein by reference. - * - * Journal revoke routines for the generic filesystem journaling code; - * part of the ext2fs journaling system. - * - * Revoke is the mechanism used to prevent old log records for deleted - * metadata from being replayed on top of newer data using the same - * blocks. The revoke mechanism is used in two separate places: - * - * + Commit: during commit we write the entire list of the current - * transaction's revoked blocks to the journal - * - * + Recovery: during recovery we record the transaction ID of all - * revoked blocks. If there are multiple revoke records in the log - * for a single block, only the last one counts, and if there is a log - * entry for a block beyond the last revoke, then that log entry still - * gets replayed. - * - * We can get interactions between revokes and new log data within a - * single transaction: - * - * Block is revoked and then journaled: - * The desired end result is the journaling of the new block, so we - * cancel the revoke before the transaction commits. - * - * Block is journaled and then revoked: - * The revoke must take precedence over the write of the block, so we - * need either to cancel the journal entry or to write the revoke - * later in the log than the log block. In this case, we choose the - * latter: journaling a block cancels any revoke record for that block - * in the current transaction, so any revoke for that block in the - * transaction must have happened after the block was journaled and so - * the revoke must take precedence. - * - * Block is revoked and then written as data: - * The data write is allowed to succeed, but the revoke is _not_ - * cancelled. We still need to prevent old log records from - * overwriting the new data. We don't even need to clear the revoke - * bit here. - * - * We cache revoke status of a buffer in the current transaction in b_states - * bits. As the name says, revokevalid flag indicates that the cached revoke - * status of a buffer is valid and we can rely on the cached status. - * - * Revoke information on buffers is a tri-state value: - * - * RevokeValid clear: no cached revoke status, need to look it up - * RevokeValid set, Revoked clear: - * buffer has not been revoked, and cancel_revoke - * need do nothing. - * RevokeValid set, Revoked set: - * buffer has been revoked. - * - * Locking rules: - * We keep two hash tables of revoke records. One hashtable belongs to the - * running transaction (is pointed to by journal->j_revoke), the other one - * belongs to the committing transaction. Accesses to the second hash table - * happen only from the kjournald and no other thread touches this table. Also - * journal_switch_revoke_table() which switches which hashtable belongs to the - * running and which to the committing transaction is called only from - * kjournald. Therefore we need no locks when accessing the hashtable belonging - * to the committing transaction. - * - * All users operating on the hash table belonging to the running transaction - * have a handle to the transaction. Therefore they are safe from kjournald - * switching hash tables under them. For operations on the lists of entries in - * the hash table j_revoke_lock is used. - * - * Finally, also replay code uses the hash tables but at this moment no one else - * can touch them (filesystem isn't mounted yet) and hence no locking is - * needed. - */ - -#ifndef __KERNEL__ -#include "jfs_user.h" -#else -#include -#include -#include -#include -#include -#include -#include -#include -#endif -#include -#include - -static struct kmem_cache *revoke_record_cache; -static struct kmem_cache *revoke_table_cache; - -/* Each revoke record represents one single revoked block. During - journal replay, this involves recording the transaction ID of the - last transaction to revoke this block. */ - -struct jbd_revoke_record_s -{ - struct list_head hash; - tid_t sequence; /* Used for recovery only */ - unsigned int blocknr; -}; - - -/* The revoke table is just a simple hash table of revoke records. */ -struct jbd_revoke_table_s -{ - /* It is conceivable that we might want a larger hash table - * for recovery. Must be a power of two. */ - int hash_size; - int hash_shift; - struct list_head *hash_table; -}; - - -#ifdef __KERNEL__ -static void write_one_revoke_record(journal_t *, transaction_t *, - struct journal_head **, int *, - struct jbd_revoke_record_s *, int); -static void flush_descriptor(journal_t *, struct journal_head *, int, int); -#endif - -/* Utility functions to maintain the revoke table */ - -static inline int hash(journal_t *journal, unsigned int block) -{ - struct jbd_revoke_table_s *table = journal->j_revoke; - - return hash_32(block, table->hash_shift); -} - -static int insert_revoke_hash(journal_t *journal, unsigned int blocknr, - tid_t seq) -{ - struct list_head *hash_list; - struct jbd_revoke_record_s *record; - -repeat: - record = kmem_cache_alloc(revoke_record_cache, GFP_NOFS); - if (!record) - goto oom; - - record->sequence = seq; - record->blocknr = blocknr; - hash_list = &journal->j_revoke->hash_table[hash(journal, blocknr)]; - spin_lock(&journal->j_revoke_lock); - list_add(&record->hash, hash_list); - spin_unlock(&journal->j_revoke_lock); - return 0; - -oom: - if (!journal_oom_retry) - return -ENOMEM; - jbd_debug(1, "ENOMEM in %s, retrying\n", __func__); - yield(); - goto repeat; -} - -/* Find a revoke record in the journal's hash table. */ - -static struct jbd_revoke_record_s *find_revoke_record(journal_t *journal, - unsigned int blocknr) -{ - struct list_head *hash_list; - struct jbd_revoke_record_s *record; - - hash_list = &journal->j_revoke->hash_table[hash(journal, blocknr)]; - - spin_lock(&journal->j_revoke_lock); - record = (struct jbd_revoke_record_s *) hash_list->next; - while (&(record->hash) != hash_list) { - if (record->blocknr == blocknr) { - spin_unlock(&journal->j_revoke_lock); - return record; - } - record = (struct jbd_revoke_record_s *) record->hash.next; - } - spin_unlock(&journal->j_revoke_lock); - return NULL; -} - -void journal_destroy_revoke_caches(void) -{ - if (revoke_record_cache) { - kmem_cache_destroy(revoke_record_cache); - revoke_record_cache = NULL; - } - if (revoke_table_cache) { - kmem_cache_destroy(revoke_table_cache); - revoke_table_cache = NULL; - } -} - -int __init journal_init_revoke_caches(void) -{ - J_ASSERT(!revoke_record_cache); - J_ASSERT(!revoke_table_cache); - - revoke_record_cache = kmem_cache_create("revoke_record", - sizeof(struct jbd_revoke_record_s), - 0, - SLAB_HWCACHE_ALIGN|SLAB_TEMPORARY, - NULL); - if (!revoke_record_cache) - goto record_cache_failure; - - revoke_table_cache = kmem_cache_create("revoke_table", - sizeof(struct jbd_revoke_table_s), - 0, SLAB_TEMPORARY, NULL); - if (!revoke_table_cache) - goto table_cache_failure; - - return 0; - -table_cache_failure: - journal_destroy_revoke_caches(); -record_cache_failure: - return -ENOMEM; -} - -static struct jbd_revoke_table_s *journal_init_revoke_table(int hash_size) -{ - int i; - struct jbd_revoke_table_s *table; - - table = kmem_cache_alloc(revoke_table_cache, GFP_KERNEL); - if (!table) - goto out; - - table->hash_size = hash_size; - table->hash_shift = ilog2(hash_size); - table->hash_table = - kmalloc(hash_size * sizeof(struct list_head), GFP_KERNEL); - if (!table->hash_table) { - kmem_cache_free(revoke_table_cache, table); - table = NULL; - goto out; - } - - for (i = 0; i < hash_size; i++) - INIT_LIST_HEAD(&table->hash_table[i]); - -out: - return table; -} - -static void journal_destroy_revoke_table(struct jbd_revoke_table_s *table) -{ - int i; - struct list_head *hash_list; - - for (i = 0; i < table->hash_size; i++) { - hash_list = &table->hash_table[i]; - J_ASSERT(list_empty(hash_list)); - } - - kfree(table->hash_table); - kmem_cache_free(revoke_table_cache, table); -} - -/* Initialise the revoke table for a given journal to a given size. */ -int journal_init_revoke(journal_t *journal, int hash_size) -{ - J_ASSERT(journal->j_revoke_table[0] == NULL); - J_ASSERT(is_power_of_2(hash_size)); - - journal->j_revoke_table[0] = journal_init_revoke_table(hash_size); - if (!journal->j_revoke_table[0]) - goto fail0; - - journal->j_revoke_table[1] = journal_init_revoke_table(hash_size); - if (!journal->j_revoke_table[1]) - goto fail1; - - journal->j_revoke = journal->j_revoke_table[1]; - - spin_lock_init(&journal->j_revoke_lock); - - return 0; - -fail1: - journal_destroy_revoke_table(journal->j_revoke_table[0]); -fail0: - return -ENOMEM; -} - -/* Destroy a journal's revoke table. The table must already be empty! */ -void journal_destroy_revoke(journal_t *journal) -{ - journal->j_revoke = NULL; - if (journal->j_revoke_table[0]) - journal_destroy_revoke_table(journal->j_revoke_table[0]); - if (journal->j_revoke_table[1]) - journal_destroy_revoke_table(journal->j_revoke_table[1]); -} - - -#ifdef __KERNEL__ - -/* - * journal_revoke: revoke a given buffer_head from the journal. This - * prevents the block from being replayed during recovery if we take a - * crash after this current transaction commits. Any subsequent - * metadata writes of the buffer in this transaction cancel the - * revoke. - * - * Note that this call may block --- it is up to the caller to make - * sure that there are no further calls to journal_write_metadata - * before the revoke is complete. In ext3, this implies calling the - * revoke before clearing the block bitmap when we are deleting - * metadata. - * - * Revoke performs a journal_forget on any buffer_head passed in as a - * parameter, but does _not_ forget the buffer_head if the bh was only - * found implicitly. - * - * bh_in may not be a journalled buffer - it may have come off - * the hash tables without an attached journal_head. - * - * If bh_in is non-zero, journal_revoke() will decrement its b_count - * by one. - */ - -int journal_revoke(handle_t *handle, unsigned int blocknr, - struct buffer_head *bh_in) -{ - struct buffer_head *bh = NULL; - journal_t *journal; - struct block_device *bdev; - int err; - - might_sleep(); - if (bh_in) - BUFFER_TRACE(bh_in, "enter"); - - journal = handle->h_transaction->t_journal; - if (!journal_set_features(journal, 0, 0, JFS_FEATURE_INCOMPAT_REVOKE)){ - J_ASSERT (!"Cannot set revoke feature!"); - return -EINVAL; - } - - bdev = journal->j_fs_dev; - bh = bh_in; - - if (!bh) { - bh = __find_get_block(bdev, blocknr, journal->j_blocksize); - if (bh) - BUFFER_TRACE(bh, "found on hash"); - } -#ifdef JBD_EXPENSIVE_CHECKING - else { - struct buffer_head *bh2; - - /* If there is a different buffer_head lying around in - * memory anywhere... */ - bh2 = __find_get_block(bdev, blocknr, journal->j_blocksize); - if (bh2) { - /* ... and it has RevokeValid status... */ - if (bh2 != bh && buffer_revokevalid(bh2)) - /* ...then it better be revoked too, - * since it's illegal to create a revoke - * record against a buffer_head which is - * not marked revoked --- that would - * risk missing a subsequent revoke - * cancel. */ - J_ASSERT_BH(bh2, buffer_revoked(bh2)); - put_bh(bh2); - } - } -#endif - - /* We really ought not ever to revoke twice in a row without - first having the revoke cancelled: it's illegal to free a - block twice without allocating it in between! */ - if (bh) { - if (!J_EXPECT_BH(bh, !buffer_revoked(bh), - "inconsistent data on disk")) { - if (!bh_in) - brelse(bh); - return -EIO; - } - set_buffer_revoked(bh); - set_buffer_revokevalid(bh); - if (bh_in) { - BUFFER_TRACE(bh_in, "call journal_forget"); - journal_forget(handle, bh_in); - } else { - BUFFER_TRACE(bh, "call brelse"); - __brelse(bh); - } - } - - jbd_debug(2, "insert revoke for block %u, bh_in=%p\n", blocknr, bh_in); - err = insert_revoke_hash(journal, blocknr, - handle->h_transaction->t_tid); - BUFFER_TRACE(bh_in, "exit"); - return err; -} - -/* - * Cancel an outstanding revoke. For use only internally by the - * journaling code (called from journal_get_write_access). - * - * We trust buffer_revoked() on the buffer if the buffer is already - * being journaled: if there is no revoke pending on the buffer, then we - * don't do anything here. - * - * This would break if it were possible for a buffer to be revoked and - * discarded, and then reallocated within the same transaction. In such - * a case we would have lost the revoked bit, but when we arrived here - * the second time we would still have a pending revoke to cancel. So, - * do not trust the Revoked bit on buffers unless RevokeValid is also - * set. - */ -int journal_cancel_revoke(handle_t *handle, struct journal_head *jh) -{ - struct jbd_revoke_record_s *record; - journal_t *journal = handle->h_transaction->t_journal; - int need_cancel; - int did_revoke = 0; /* akpm: debug */ - struct buffer_head *bh = jh2bh(jh); - - jbd_debug(4, "journal_head %p, cancelling revoke\n", jh); - - /* Is the existing Revoke bit valid? If so, we trust it, and - * only perform the full cancel if the revoke bit is set. If - * not, we can't trust the revoke bit, and we need to do the - * full search for a revoke record. */ - if (test_set_buffer_revokevalid(bh)) { - need_cancel = test_clear_buffer_revoked(bh); - } else { - need_cancel = 1; - clear_buffer_revoked(bh); - } - - if (need_cancel) { - record = find_revoke_record(journal, bh->b_blocknr); - if (record) { - jbd_debug(4, "cancelled existing revoke on " - "blocknr %llu\n", (unsigned long long)bh->b_blocknr); - spin_lock(&journal->j_revoke_lock); - list_del(&record->hash); - spin_unlock(&journal->j_revoke_lock); - kmem_cache_free(revoke_record_cache, record); - did_revoke = 1; - } - } - -#ifdef JBD_EXPENSIVE_CHECKING - /* There better not be one left behind by now! */ - record = find_revoke_record(journal, bh->b_blocknr); - J_ASSERT_JH(jh, record == NULL); -#endif - - /* Finally, have we just cleared revoke on an unhashed - * buffer_head? If so, we'd better make sure we clear the - * revoked status on any hashed alias too, otherwise the revoke - * state machine will get very upset later on. */ - if (need_cancel) { - struct buffer_head *bh2; - bh2 = __find_get_block(bh->b_bdev, bh->b_blocknr, bh->b_size); - if (bh2) { - if (bh2 != bh) - clear_buffer_revoked(bh2); - __brelse(bh2); - } - } - return did_revoke; -} - -/* - * journal_clear_revoked_flags clears revoked flag of buffers in - * revoke table to reflect there is no revoked buffer in the next - * transaction which is going to be started. - */ -void journal_clear_buffer_revoked_flags(journal_t *journal) -{ - struct jbd_revoke_table_s *revoke = journal->j_revoke; - int i = 0; - - for (i = 0; i < revoke->hash_size; i++) { - struct list_head *hash_list; - struct list_head *list_entry; - hash_list = &revoke->hash_table[i]; - - list_for_each(list_entry, hash_list) { - struct jbd_revoke_record_s *record; - struct buffer_head *bh; - record = (struct jbd_revoke_record_s *)list_entry; - bh = __find_get_block(journal->j_fs_dev, - record->blocknr, - journal->j_blocksize); - if (bh) { - clear_buffer_revoked(bh); - __brelse(bh); - } - } - } -} - -/* journal_switch_revoke table select j_revoke for next transaction - * we do not want to suspend any processing until all revokes are - * written -bzzz - */ -void journal_switch_revoke_table(journal_t *journal) -{ - int i; - - if (journal->j_revoke == journal->j_revoke_table[0]) - journal->j_revoke = journal->j_revoke_table[1]; - else - journal->j_revoke = journal->j_revoke_table[0]; - - for (i = 0; i < journal->j_revoke->hash_size; i++) - INIT_LIST_HEAD(&journal->j_revoke->hash_table[i]); -} - -/* - * Write revoke records to the journal for all entries in the current - * revoke hash, deleting the entries as we go. - */ -void journal_write_revoke_records(journal_t *journal, - transaction_t *transaction, int write_op) -{ - struct journal_head *descriptor; - struct jbd_revoke_record_s *record; - struct jbd_revoke_table_s *revoke; - struct list_head *hash_list; - int i, offset, count; - - descriptor = NULL; - offset = 0; - count = 0; - - /* select revoke table for committing transaction */ - revoke = journal->j_revoke == journal->j_revoke_table[0] ? - journal->j_revoke_table[1] : journal->j_revoke_table[0]; - - for (i = 0; i < revoke->hash_size; i++) { - hash_list = &revoke->hash_table[i]; - - while (!list_empty(hash_list)) { - record = (struct jbd_revoke_record_s *) - hash_list->next; - write_one_revoke_record(journal, transaction, - &descriptor, &offset, - record, write_op); - count++; - list_del(&record->hash); - kmem_cache_free(revoke_record_cache, record); - } - } - if (descriptor) - flush_descriptor(journal, descriptor, offset, write_op); - jbd_debug(1, "Wrote %d revoke records\n", count); -} - -/* - * Write out one revoke record. We need to create a new descriptor - * block if the old one is full or if we have not already created one. - */ - -static void write_one_revoke_record(journal_t *journal, - transaction_t *transaction, - struct journal_head **descriptorp, - int *offsetp, - struct jbd_revoke_record_s *record, - int write_op) -{ - struct journal_head *descriptor; - int offset; - journal_header_t *header; - - /* If we are already aborting, this all becomes a noop. We - still need to go round the loop in - journal_write_revoke_records in order to free all of the - revoke records: only the IO to the journal is omitted. */ - if (is_journal_aborted(journal)) - return; - - descriptor = *descriptorp; - offset = *offsetp; - - /* Make sure we have a descriptor with space left for the record */ - if (descriptor) { - if (offset == journal->j_blocksize) { - flush_descriptor(journal, descriptor, offset, write_op); - descriptor = NULL; - } - } - - if (!descriptor) { - descriptor = journal_get_descriptor_buffer(journal); - if (!descriptor) - return; - header = (journal_header_t *) &jh2bh(descriptor)->b_data[0]; - header->h_magic = cpu_to_be32(JFS_MAGIC_NUMBER); - header->h_blocktype = cpu_to_be32(JFS_REVOKE_BLOCK); - header->h_sequence = cpu_to_be32(transaction->t_tid); - - /* Record it so that we can wait for IO completion later */ - JBUFFER_TRACE(descriptor, "file as BJ_LogCtl"); - journal_file_buffer(descriptor, transaction, BJ_LogCtl); - - offset = sizeof(journal_revoke_header_t); - *descriptorp = descriptor; - } - - * ((__be32 *)(&jh2bh(descriptor)->b_data[offset])) = - cpu_to_be32(record->blocknr); - offset += 4; - *offsetp = offset; -} - -/* - * Flush a revoke descriptor out to the journal. If we are aborting, - * this is a noop; otherwise we are generating a buffer which needs to - * be waited for during commit, so it has to go onto the appropriate - * journal buffer list. - */ - -static void flush_descriptor(journal_t *journal, - struct journal_head *descriptor, - int offset, int write_op) -{ - journal_revoke_header_t *header; - struct buffer_head *bh = jh2bh(descriptor); - - if (is_journal_aborted(journal)) { - put_bh(bh); - return; - } - - header = (journal_revoke_header_t *) jh2bh(descriptor)->b_data; - header->r_count = cpu_to_be32(offset); - set_buffer_jwrite(bh); - BUFFER_TRACE(bh, "write"); - set_buffer_dirty(bh); - write_dirty_buffer(bh, write_op); -} -#endif - -/* - * Revoke support for recovery. - * - * Recovery needs to be able to: - * - * record all revoke records, including the tid of the latest instance - * of each revoke in the journal - * - * check whether a given block in a given transaction should be replayed - * (ie. has not been revoked by a revoke record in that or a subsequent - * transaction) - * - * empty the revoke table after recovery. - */ - -/* - * First, setting revoke records. We create a new revoke record for - * every block ever revoked in the log as we scan it for recovery, and - * we update the existing records if we find multiple revokes for a - * single block. - */ - -int journal_set_revoke(journal_t *journal, - unsigned int blocknr, - tid_t sequence) -{ - struct jbd_revoke_record_s *record; - - record = find_revoke_record(journal, blocknr); - if (record) { - /* If we have multiple occurrences, only record the - * latest sequence number in the hashed record */ - if (tid_gt(sequence, record->sequence)) - record->sequence = sequence; - return 0; - } - return insert_revoke_hash(journal, blocknr, sequence); -} - -/* - * Test revoke records. For a given block referenced in the log, has - * that block been revoked? A revoke record with a given transaction - * sequence number revokes all blocks in that transaction and earlier - * ones, but later transactions still need replayed. - */ - -int journal_test_revoke(journal_t *journal, - unsigned int blocknr, - tid_t sequence) -{ - struct jbd_revoke_record_s *record; - - record = find_revoke_record(journal, blocknr); - if (!record) - return 0; - if (tid_gt(sequence, record->sequence)) - return 0; - return 1; -} - -/* - * Finally, once recovery is over, we need to clear the revoke table so - * that it can be reused by the running filesystem. - */ - -void journal_clear_revoke(journal_t *journal) -{ - int i; - struct list_head *hash_list; - struct jbd_revoke_record_s *record; - struct jbd_revoke_table_s *revoke; - - revoke = journal->j_revoke; - - for (i = 0; i < revoke->hash_size; i++) { - hash_list = &revoke->hash_table[i]; - while (!list_empty(hash_list)) { - record = (struct jbd_revoke_record_s*) hash_list->next; - list_del(&record->hash); - kmem_cache_free(revoke_record_cache, record); - } - } -} diff --git a/fs/jbd/transaction.c b/fs/jbd/transaction.c deleted file mode 100644 index 1695ba8334a2a6..00000000000000 --- a/fs/jbd/transaction.c +++ /dev/null @@ -1,2237 +0,0 @@ -/* - * linux/fs/jbd/transaction.c - * - * Written by Stephen C. Tweedie , 1998 - * - * Copyright 1998 Red Hat corp --- All Rights Reserved - * - * This file is part of the Linux kernel and is made available under - * the terms of the GNU General Public License, version 2, or at your - * option, any later version, incorporated herein by reference. - * - * Generic filesystem transaction handling code; part of the ext2fs - * journaling system. - * - * This file manages transactions (compound commits managed by the - * journaling code) and handles (individual atomic operations by the - * filesystem). - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -static void __journal_temp_unlink_buffer(struct journal_head *jh); - -/* - * get_transaction: obtain a new transaction_t object. - * - * Simply allocate and initialise a new transaction. Create it in - * RUNNING state and add it to the current journal (which should not - * have an existing running transaction: we only make a new transaction - * once we have started to commit the old one). - * - * Preconditions: - * The journal MUST be locked. We don't perform atomic mallocs on the - * new transaction and we can't block without protecting against other - * processes trying to touch the journal while it is in transition. - * - * Called under j_state_lock - */ - -static transaction_t * -get_transaction(journal_t *journal, transaction_t *transaction) -{ - transaction->t_journal = journal; - transaction->t_state = T_RUNNING; - transaction->t_start_time = ktime_get(); - transaction->t_tid = journal->j_transaction_sequence++; - transaction->t_expires = jiffies + journal->j_commit_interval; - spin_lock_init(&transaction->t_handle_lock); - - /* Set up the commit timer for the new transaction. */ - journal->j_commit_timer.expires = - round_jiffies_up(transaction->t_expires); - add_timer(&journal->j_commit_timer); - - J_ASSERT(journal->j_running_transaction == NULL); - journal->j_running_transaction = transaction; - - return transaction; -} - -/* - * Handle management. - * - * A handle_t is an object which represents a single atomic update to a - * filesystem, and which tracks all of the modifications which form part - * of that one update. - */ - -/* - * start_this_handle: Given a handle, deal with any locking or stalling - * needed to make sure that there is enough journal space for the handle - * to begin. Attach the handle to a transaction and set up the - * transaction's buffer credits. - */ - -static int start_this_handle(journal_t *journal, handle_t *handle) -{ - transaction_t *transaction; - int needed; - int nblocks = handle->h_buffer_credits; - transaction_t *new_transaction = NULL; - int ret = 0; - - if (nblocks > journal->j_max_transaction_buffers) { - printk(KERN_ERR "JBD: %s wants too many credits (%d > %d)\n", - current->comm, nblocks, - journal->j_max_transaction_buffers); - ret = -ENOSPC; - goto out; - } - -alloc_transaction: - if (!journal->j_running_transaction) { - new_transaction = kzalloc(sizeof(*new_transaction), - GFP_NOFS|__GFP_NOFAIL); - if (!new_transaction) { - ret = -ENOMEM; - goto out; - } - } - - jbd_debug(3, "New handle %p going live.\n", handle); - -repeat: - - /* - * We need to hold j_state_lock until t_updates has been incremented, - * for proper journal barrier handling - */ - spin_lock(&journal->j_state_lock); -repeat_locked: - if (is_journal_aborted(journal) || - (journal->j_errno != 0 && !(journal->j_flags & JFS_ACK_ERR))) { - spin_unlock(&journal->j_state_lock); - ret = -EROFS; - goto out; - } - - /* Wait on the journal's transaction barrier if necessary */ - if (journal->j_barrier_count) { - spin_unlock(&journal->j_state_lock); - wait_event(journal->j_wait_transaction_locked, - journal->j_barrier_count == 0); - goto repeat; - } - - if (!journal->j_running_transaction) { - if (!new_transaction) { - spin_unlock(&journal->j_state_lock); - goto alloc_transaction; - } - get_transaction(journal, new_transaction); - new_transaction = NULL; - } - - transaction = journal->j_running_transaction; - - /* - * If the current transaction is locked down for commit, wait for the - * lock to be released. - */ - if (transaction->t_state == T_LOCKED) { - DEFINE_WAIT(wait); - - prepare_to_wait(&journal->j_wait_transaction_locked, - &wait, TASK_UNINTERRUPTIBLE); - spin_unlock(&journal->j_state_lock); - schedule(); - finish_wait(&journal->j_wait_transaction_locked, &wait); - goto repeat; - } - - /* - * If there is not enough space left in the log to write all potential - * buffers requested by this operation, we need to stall pending a log - * checkpoint to free some more log space. - */ - spin_lock(&transaction->t_handle_lock); - needed = transaction->t_outstanding_credits + nblocks; - - if (needed > journal->j_max_transaction_buffers) { - /* - * If the current transaction is already too large, then start - * to commit it: we can then go back and attach this handle to - * a new transaction. - */ - DEFINE_WAIT(wait); - - jbd_debug(2, "Handle %p starting new commit...\n", handle); - spin_unlock(&transaction->t_handle_lock); - prepare_to_wait(&journal->j_wait_transaction_locked, &wait, - TASK_UNINTERRUPTIBLE); - __log_start_commit(journal, transaction->t_tid); - spin_unlock(&journal->j_state_lock); - schedule(); - finish_wait(&journal->j_wait_transaction_locked, &wait); - goto repeat; - } - - /* - * The commit code assumes that it can get enough log space - * without forcing a checkpoint. This is *critical* for - * correctness: a checkpoint of a buffer which is also - * associated with a committing transaction creates a deadlock, - * so commit simply cannot force through checkpoints. - * - * We must therefore ensure the necessary space in the journal - * *before* starting to dirty potentially checkpointed buffers - * in the new transaction. - * - * The worst part is, any transaction currently committing can - * reduce the free space arbitrarily. Be careful to account for - * those buffers when checkpointing. - */ - - /* - * @@@ AKPM: This seems rather over-defensive. We're giving commit - * a _lot_ of headroom: 1/4 of the journal plus the size of - * the committing transaction. Really, we only need to give it - * committing_transaction->t_outstanding_credits plus "enough" for - * the log control blocks. - * Also, this test is inconsistent with the matching one in - * journal_extend(). - */ - if (__log_space_left(journal) < jbd_space_needed(journal)) { - jbd_debug(2, "Handle %p waiting for checkpoint...\n", handle); - spin_unlock(&transaction->t_handle_lock); - __log_wait_for_space(journal); - goto repeat_locked; - } - - /* OK, account for the buffers that this operation expects to - * use and add the handle to the running transaction. */ - - handle->h_transaction = transaction; - transaction->t_outstanding_credits += nblocks; - transaction->t_updates++; - transaction->t_handle_count++; - jbd_debug(4, "Handle %p given %d credits (total %d, free %d)\n", - handle, nblocks, transaction->t_outstanding_credits, - __log_space_left(journal)); - spin_unlock(&transaction->t_handle_lock); - spin_unlock(&journal->j_state_lock); - - lock_map_acquire(&handle->h_lockdep_map); -out: - if (unlikely(new_transaction)) /* It's usually NULL */ - kfree(new_transaction); - return ret; -} - -static struct lock_class_key jbd_handle_key; - -/* Allocate a new handle. This should probably be in a slab... */ -static handle_t *new_handle(int nblocks) -{ - handle_t *handle = jbd_alloc_handle(GFP_NOFS); - if (!handle) - return NULL; - handle->h_buffer_credits = nblocks; - handle->h_ref = 1; - - lockdep_init_map(&handle->h_lockdep_map, "jbd_handle", &jbd_handle_key, 0); - - return handle; -} - -/** - * handle_t *journal_start() - Obtain a new handle. - * @journal: Journal to start transaction on. - * @nblocks: number of block buffer we might modify - * - * We make sure that the transaction can guarantee at least nblocks of - * modified buffers in the log. We block until the log can guarantee - * that much space. - * - * This function is visible to journal users (like ext3fs), so is not - * called with the journal already locked. - * - * Return a pointer to a newly allocated handle, or an ERR_PTR() value - * on failure. - */ -handle_t *journal_start(journal_t *journal, int nblocks) -{ - handle_t *handle = journal_current_handle(); - int err; - - if (!journal) - return ERR_PTR(-EROFS); - - if (handle) { - J_ASSERT(handle->h_transaction->t_journal == journal); - handle->h_ref++; - return handle; - } - - handle = new_handle(nblocks); - if (!handle) - return ERR_PTR(-ENOMEM); - - current->journal_info = handle; - - err = start_this_handle(journal, handle); - if (err < 0) { - jbd_free_handle(handle); - current->journal_info = NULL; - handle = ERR_PTR(err); - } - return handle; -} - -/** - * int journal_extend() - extend buffer credits. - * @handle: handle to 'extend' - * @nblocks: nr blocks to try to extend by. - * - * Some transactions, such as large extends and truncates, can be done - * atomically all at once or in several stages. The operation requests - * a credit for a number of buffer modications in advance, but can - * extend its credit if it needs more. - * - * journal_extend tries to give the running handle more buffer credits. - * It does not guarantee that allocation - this is a best-effort only. - * The calling process MUST be able to deal cleanly with a failure to - * extend here. - * - * Return 0 on success, non-zero on failure. - * - * return code < 0 implies an error - * return code > 0 implies normal transaction-full status. - */ -int journal_extend(handle_t *handle, int nblocks) -{ - transaction_t *transaction = handle->h_transaction; - journal_t *journal = transaction->t_journal; - int result; - int wanted; - - result = -EIO; - if (is_handle_aborted(handle)) - goto out; - - result = 1; - - spin_lock(&journal->j_state_lock); - - /* Don't extend a locked-down transaction! */ - if (handle->h_transaction->t_state != T_RUNNING) { - jbd_debug(3, "denied handle %p %d blocks: " - "transaction not running\n", handle, nblocks); - goto error_out; - } - - spin_lock(&transaction->t_handle_lock); - wanted = transaction->t_outstanding_credits + nblocks; - - if (wanted > journal->j_max_transaction_buffers) { - jbd_debug(3, "denied handle %p %d blocks: " - "transaction too large\n", handle, nblocks); - goto unlock; - } - - if (wanted > __log_space_left(journal)) { - jbd_debug(3, "denied handle %p %d blocks: " - "insufficient log space\n", handle, nblocks); - goto unlock; - } - - handle->h_buffer_credits += nblocks; - transaction->t_outstanding_credits += nblocks; - result = 0; - - jbd_debug(3, "extended handle %p by %d\n", handle, nblocks); -unlock: - spin_unlock(&transaction->t_handle_lock); -error_out: - spin_unlock(&journal->j_state_lock); -out: - return result; -} - - -/** - * int journal_restart() - restart a handle. - * @handle: handle to restart - * @nblocks: nr credits requested - * - * Restart a handle for a multi-transaction filesystem - * operation. - * - * If the journal_extend() call above fails to grant new buffer credits - * to a running handle, a call to journal_restart will commit the - * handle's transaction so far and reattach the handle to a new - * transaction capabable of guaranteeing the requested number of - * credits. - */ - -int journal_restart(handle_t *handle, int nblocks) -{ - transaction_t *transaction = handle->h_transaction; - journal_t *journal = transaction->t_journal; - int ret; - - /* If we've had an abort of any type, don't even think about - * actually doing the restart! */ - if (is_handle_aborted(handle)) - return 0; - - /* - * First unlink the handle from its current transaction, and start the - * commit on that. - */ - J_ASSERT(transaction->t_updates > 0); - J_ASSERT(journal_current_handle() == handle); - - spin_lock(&journal->j_state_lock); - spin_lock(&transaction->t_handle_lock); - transaction->t_outstanding_credits -= handle->h_buffer_credits; - transaction->t_updates--; - - if (!transaction->t_updates) - wake_up(&journal->j_wait_updates); - spin_unlock(&transaction->t_handle_lock); - - jbd_debug(2, "restarting handle %p\n", handle); - __log_start_commit(journal, transaction->t_tid); - spin_unlock(&journal->j_state_lock); - - lock_map_release(&handle->h_lockdep_map); - handle->h_buffer_credits = nblocks; - ret = start_this_handle(journal, handle); - return ret; -} - - -/** - * void journal_lock_updates () - establish a transaction barrier. - * @journal: Journal to establish a barrier on. - * - * This locks out any further updates from being started, and blocks until all - * existing updates have completed, returning only once the journal is in a - * quiescent state with no updates running. - * - * We do not use simple mutex for synchronization as there are syscalls which - * want to return with filesystem locked and that trips up lockdep. Also - * hibernate needs to lock filesystem but locked mutex then blocks hibernation. - * Since locking filesystem is rare operation, we use simple counter and - * waitqueue for locking. - */ -void journal_lock_updates(journal_t *journal) -{ - DEFINE_WAIT(wait); - -wait: - /* Wait for previous locked operation to finish */ - wait_event(journal->j_wait_transaction_locked, - journal->j_barrier_count == 0); - - spin_lock(&journal->j_state_lock); - /* - * Check reliably under the lock whether we are the ones winning the race - * and locking the journal - */ - if (journal->j_barrier_count > 0) { - spin_unlock(&journal->j_state_lock); - goto wait; - } - ++journal->j_barrier_count; - - /* Wait until there are no running updates */ - while (1) { - transaction_t *transaction = journal->j_running_transaction; - - if (!transaction) - break; - - spin_lock(&transaction->t_handle_lock); - if (!transaction->t_updates) { - spin_unlock(&transaction->t_handle_lock); - break; - } - prepare_to_wait(&journal->j_wait_updates, &wait, - TASK_UNINTERRUPTIBLE); - spin_unlock(&transaction->t_handle_lock); - spin_unlock(&journal->j_state_lock); - schedule(); - finish_wait(&journal->j_wait_updates, &wait); - spin_lock(&journal->j_state_lock); - } - spin_unlock(&journal->j_state_lock); -} - -/** - * void journal_unlock_updates (journal_t* journal) - release barrier - * @journal: Journal to release the barrier on. - * - * Release a transaction barrier obtained with journal_lock_updates(). - */ -void journal_unlock_updates (journal_t *journal) -{ - J_ASSERT(journal->j_barrier_count != 0); - - spin_lock(&journal->j_state_lock); - --journal->j_barrier_count; - spin_unlock(&journal->j_state_lock); - wake_up(&journal->j_wait_transaction_locked); -} - -static void warn_dirty_buffer(struct buffer_head *bh) -{ - char b[BDEVNAME_SIZE]; - - printk(KERN_WARNING - "JBD: Spotted dirty metadata buffer (dev = %s, blocknr = %llu). " - "There's a risk of filesystem corruption in case of system " - "crash.\n", - bdevname(bh->b_bdev, b), (unsigned long long)bh->b_blocknr); -} - -/* - * If the buffer is already part of the current transaction, then there - * is nothing we need to do. If it is already part of a prior - * transaction which we are still committing to disk, then we need to - * make sure that we do not overwrite the old copy: we do copy-out to - * preserve the copy going to disk. We also account the buffer against - * the handle's metadata buffer credits (unless the buffer is already - * part of the transaction, that is). - * - */ -static int -do_get_write_access(handle_t *handle, struct journal_head *jh, - int force_copy) -{ - struct buffer_head *bh; - transaction_t *transaction; - journal_t *journal; - int error; - char *frozen_buffer = NULL; - int need_copy = 0; - - if (is_handle_aborted(handle)) - return -EROFS; - - transaction = handle->h_transaction; - journal = transaction->t_journal; - - jbd_debug(5, "journal_head %p, force_copy %d\n", jh, force_copy); - - JBUFFER_TRACE(jh, "entry"); -repeat: - bh = jh2bh(jh); - - /* @@@ Need to check for errors here at some point. */ - - lock_buffer(bh); - jbd_lock_bh_state(bh); - - /* We now hold the buffer lock so it is safe to query the buffer - * state. Is the buffer dirty? - * - * If so, there are two possibilities. The buffer may be - * non-journaled, and undergoing a quite legitimate writeback. - * Otherwise, it is journaled, and we don't expect dirty buffers - * in that state (the buffers should be marked JBD_Dirty - * instead.) So either the IO is being done under our own - * control and this is a bug, or it's a third party IO such as - * dump(8) (which may leave the buffer scheduled for read --- - * ie. locked but not dirty) or tune2fs (which may actually have - * the buffer dirtied, ugh.) */ - - if (buffer_dirty(bh)) { - /* - * First question: is this buffer already part of the current - * transaction or the existing committing transaction? - */ - if (jh->b_transaction) { - J_ASSERT_JH(jh, - jh->b_transaction == transaction || - jh->b_transaction == - journal->j_committing_transaction); - if (jh->b_next_transaction) - J_ASSERT_JH(jh, jh->b_next_transaction == - transaction); - warn_dirty_buffer(bh); - } - /* - * In any case we need to clean the dirty flag and we must - * do it under the buffer lock to be sure we don't race - * with running write-out. - */ - JBUFFER_TRACE(jh, "Journalling dirty buffer"); - clear_buffer_dirty(bh); - set_buffer_jbddirty(bh); - } - - unlock_buffer(bh); - - error = -EROFS; - if (is_handle_aborted(handle)) { - jbd_unlock_bh_state(bh); - goto out; - } - error = 0; - - /* - * The buffer is already part of this transaction if b_transaction or - * b_next_transaction points to it - */ - if (jh->b_transaction == transaction || - jh->b_next_transaction == transaction) - goto done; - - /* - * this is the first time this transaction is touching this buffer, - * reset the modified flag - */ - jh->b_modified = 0; - - /* - * If there is already a copy-out version of this buffer, then we don't - * need to make another one - */ - if (jh->b_frozen_data) { - JBUFFER_TRACE(jh, "has frozen data"); - J_ASSERT_JH(jh, jh->b_next_transaction == NULL); - jh->b_next_transaction = transaction; - goto done; - } - - /* Is there data here we need to preserve? */ - - if (jh->b_transaction && jh->b_transaction != transaction) { - JBUFFER_TRACE(jh, "owned by older transaction"); - J_ASSERT_JH(jh, jh->b_next_transaction == NULL); - J_ASSERT_JH(jh, jh->b_transaction == - journal->j_committing_transaction); - - /* There is one case we have to be very careful about. - * If the committing transaction is currently writing - * this buffer out to disk and has NOT made a copy-out, - * then we cannot modify the buffer contents at all - * right now. The essence of copy-out is that it is the - * extra copy, not the primary copy, which gets - * journaled. If the primary copy is already going to - * disk then we cannot do copy-out here. */ - - if (jh->b_jlist == BJ_Shadow) { - DEFINE_WAIT_BIT(wait, &bh->b_state, BH_Unshadow); - wait_queue_head_t *wqh; - - wqh = bit_waitqueue(&bh->b_state, BH_Unshadow); - - JBUFFER_TRACE(jh, "on shadow: sleep"); - jbd_unlock_bh_state(bh); - /* commit wakes up all shadow buffers after IO */ - for ( ; ; ) { - prepare_to_wait(wqh, &wait.wait, - TASK_UNINTERRUPTIBLE); - if (jh->b_jlist != BJ_Shadow) - break; - schedule(); - } - finish_wait(wqh, &wait.wait); - goto repeat; - } - - /* Only do the copy if the currently-owning transaction - * still needs it. If it is on the Forget list, the - * committing transaction is past that stage. The - * buffer had better remain locked during the kmalloc, - * but that should be true --- we hold the journal lock - * still and the buffer is already on the BUF_JOURNAL - * list so won't be flushed. - * - * Subtle point, though: if this is a get_undo_access, - * then we will be relying on the frozen_data to contain - * the new value of the committed_data record after the - * transaction, so we HAVE to force the frozen_data copy - * in that case. */ - - if (jh->b_jlist != BJ_Forget || force_copy) { - JBUFFER_TRACE(jh, "generate frozen data"); - if (!frozen_buffer) { - JBUFFER_TRACE(jh, "allocate memory for buffer"); - jbd_unlock_bh_state(bh); - frozen_buffer = - jbd_alloc(jh2bh(jh)->b_size, - GFP_NOFS); - if (!frozen_buffer) { - printk(KERN_ERR - "%s: OOM for frozen_buffer\n", - __func__); - JBUFFER_TRACE(jh, "oom!"); - error = -ENOMEM; - jbd_lock_bh_state(bh); - goto done; - } - goto repeat; - } - jh->b_frozen_data = frozen_buffer; - frozen_buffer = NULL; - need_copy = 1; - } - jh->b_next_transaction = transaction; - } - - - /* - * Finally, if the buffer is not journaled right now, we need to make - * sure it doesn't get written to disk before the caller actually - * commits the new data - */ - if (!jh->b_transaction) { - JBUFFER_TRACE(jh, "no transaction"); - J_ASSERT_JH(jh, !jh->b_next_transaction); - JBUFFER_TRACE(jh, "file as BJ_Reserved"); - spin_lock(&journal->j_list_lock); - __journal_file_buffer(jh, transaction, BJ_Reserved); - spin_unlock(&journal->j_list_lock); - } - -done: - if (need_copy) { - struct page *page; - int offset; - char *source; - - J_EXPECT_JH(jh, buffer_uptodate(jh2bh(jh)), - "Possible IO failure.\n"); - page = jh2bh(jh)->b_page; - offset = offset_in_page(jh2bh(jh)->b_data); - source = kmap_atomic(page); - memcpy(jh->b_frozen_data, source+offset, jh2bh(jh)->b_size); - kunmap_atomic(source); - } - jbd_unlock_bh_state(bh); - - /* - * If we are about to journal a buffer, then any revoke pending on it is - * no longer valid - */ - journal_cancel_revoke(handle, jh); - -out: - if (unlikely(frozen_buffer)) /* It's usually NULL */ - jbd_free(frozen_buffer, bh->b_size); - - JBUFFER_TRACE(jh, "exit"); - return error; -} - -/** - * int journal_get_write_access() - notify intent to modify a buffer for metadata (not data) update. - * @handle: transaction to add buffer modifications to - * @bh: bh to be used for metadata writes - * - * Returns an error code or 0 on success. - * - * In full data journalling mode the buffer may be of type BJ_AsyncData, - * because we're write()ing a buffer which is also part of a shared mapping. - */ - -int journal_get_write_access(handle_t *handle, struct buffer_head *bh) -{ - struct journal_head *jh = journal_add_journal_head(bh); - int rc; - - /* We do not want to get caught playing with fields which the - * log thread also manipulates. Make sure that the buffer - * completes any outstanding IO before proceeding. */ - rc = do_get_write_access(handle, jh, 0); - journal_put_journal_head(jh); - return rc; -} - - -/* - * When the user wants to journal a newly created buffer_head - * (ie. getblk() returned a new buffer and we are going to populate it - * manually rather than reading off disk), then we need to keep the - * buffer_head locked until it has been completely filled with new - * data. In this case, we should be able to make the assertion that - * the bh is not already part of an existing transaction. - * - * The buffer should already be locked by the caller by this point. - * There is no lock ranking violation: it was a newly created, - * unlocked buffer beforehand. */ - -/** - * int journal_get_create_access () - notify intent to use newly created bh - * @handle: transaction to new buffer to - * @bh: new buffer. - * - * Call this if you create a new bh. - */ -int journal_get_create_access(handle_t *handle, struct buffer_head *bh) -{ - transaction_t *transaction = handle->h_transaction; - journal_t *journal = transaction->t_journal; - struct journal_head *jh = journal_add_journal_head(bh); - int err; - - jbd_debug(5, "journal_head %p\n", jh); - err = -EROFS; - if (is_handle_aborted(handle)) - goto out; - err = 0; - - JBUFFER_TRACE(jh, "entry"); - /* - * The buffer may already belong to this transaction due to pre-zeroing - * in the filesystem's new_block code. It may also be on the previous, - * committing transaction's lists, but it HAS to be in Forget state in - * that case: the transaction must have deleted the buffer for it to be - * reused here. - */ - jbd_lock_bh_state(bh); - spin_lock(&journal->j_list_lock); - J_ASSERT_JH(jh, (jh->b_transaction == transaction || - jh->b_transaction == NULL || - (jh->b_transaction == journal->j_committing_transaction && - jh->b_jlist == BJ_Forget))); - - J_ASSERT_JH(jh, jh->b_next_transaction == NULL); - J_ASSERT_JH(jh, buffer_locked(jh2bh(jh))); - - if (jh->b_transaction == NULL) { - /* - * Previous journal_forget() could have left the buffer - * with jbddirty bit set because it was being committed. When - * the commit finished, we've filed the buffer for - * checkpointing and marked it dirty. Now we are reallocating - * the buffer so the transaction freeing it must have - * committed and so it's safe to clear the dirty bit. - */ - clear_buffer_dirty(jh2bh(jh)); - - /* first access by this transaction */ - jh->b_modified = 0; - - JBUFFER_TRACE(jh, "file as BJ_Reserved"); - __journal_file_buffer(jh, transaction, BJ_Reserved); - } else if (jh->b_transaction == journal->j_committing_transaction) { - /* first access by this transaction */ - jh->b_modified = 0; - - JBUFFER_TRACE(jh, "set next transaction"); - jh->b_next_transaction = transaction; - } - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - - /* - * akpm: I added this. ext3_alloc_branch can pick up new indirect - * blocks which contain freed but then revoked metadata. We need - * to cancel the revoke in case we end up freeing it yet again - * and the reallocating as data - this would cause a second revoke, - * which hits an assertion error. - */ - JBUFFER_TRACE(jh, "cancelling revoke"); - journal_cancel_revoke(handle, jh); -out: - journal_put_journal_head(jh); - return err; -} - -/** - * int journal_get_undo_access() - Notify intent to modify metadata with non-rewindable consequences - * @handle: transaction - * @bh: buffer to undo - * - * Sometimes there is a need to distinguish between metadata which has - * been committed to disk and that which has not. The ext3fs code uses - * this for freeing and allocating space, we have to make sure that we - * do not reuse freed space until the deallocation has been committed, - * since if we overwrote that space we would make the delete - * un-rewindable in case of a crash. - * - * To deal with that, journal_get_undo_access requests write access to a - * buffer for parts of non-rewindable operations such as delete - * operations on the bitmaps. The journaling code must keep a copy of - * the buffer's contents prior to the undo_access call until such time - * as we know that the buffer has definitely been committed to disk. - * - * We never need to know which transaction the committed data is part - * of, buffers touched here are guaranteed to be dirtied later and so - * will be committed to a new transaction in due course, at which point - * we can discard the old committed data pointer. - * - * Returns error number or 0 on success. - */ -int journal_get_undo_access(handle_t *handle, struct buffer_head *bh) -{ - int err; - struct journal_head *jh = journal_add_journal_head(bh); - char *committed_data = NULL; - - JBUFFER_TRACE(jh, "entry"); - - /* - * Do this first --- it can drop the journal lock, so we want to - * make sure that obtaining the committed_data is done - * atomically wrt. completion of any outstanding commits. - */ - err = do_get_write_access(handle, jh, 1); - if (err) - goto out; - -repeat: - if (!jh->b_committed_data) { - committed_data = jbd_alloc(jh2bh(jh)->b_size, GFP_NOFS); - if (!committed_data) { - printk(KERN_ERR "%s: No memory for committed data\n", - __func__); - err = -ENOMEM; - goto out; - } - } - - jbd_lock_bh_state(bh); - if (!jh->b_committed_data) { - /* Copy out the current buffer contents into the - * preserved, committed copy. */ - JBUFFER_TRACE(jh, "generate b_committed data"); - if (!committed_data) { - jbd_unlock_bh_state(bh); - goto repeat; - } - - jh->b_committed_data = committed_data; - committed_data = NULL; - memcpy(jh->b_committed_data, bh->b_data, bh->b_size); - } - jbd_unlock_bh_state(bh); -out: - journal_put_journal_head(jh); - if (unlikely(committed_data)) - jbd_free(committed_data, bh->b_size); - return err; -} - -/** - * int journal_dirty_data() - mark a buffer as containing dirty data to be flushed - * @handle: transaction - * @bh: bufferhead to mark - * - * Description: - * Mark a buffer as containing dirty data which needs to be flushed before - * we can commit the current transaction. - * - * The buffer is placed on the transaction's data list and is marked as - * belonging to the transaction. - * - * Returns error number or 0 on success. - * - * journal_dirty_data() can be called via page_launder->ext3_writepage - * by kswapd. - */ -int journal_dirty_data(handle_t *handle, struct buffer_head *bh) -{ - journal_t *journal = handle->h_transaction->t_journal; - int need_brelse = 0; - struct journal_head *jh; - int ret = 0; - - if (is_handle_aborted(handle)) - return ret; - - jh = journal_add_journal_head(bh); - JBUFFER_TRACE(jh, "entry"); - - /* - * The buffer could *already* be dirty. Writeout can start - * at any time. - */ - jbd_debug(4, "jh: %p, tid:%d\n", jh, handle->h_transaction->t_tid); - - /* - * What if the buffer is already part of a running transaction? - * - * There are two cases: - * 1) It is part of the current running transaction. Refile it, - * just in case we have allocated it as metadata, deallocated - * it, then reallocated it as data. - * 2) It is part of the previous, still-committing transaction. - * If all we want to do is to guarantee that the buffer will be - * written to disk before this new transaction commits, then - * being sure that the *previous* transaction has this same - * property is sufficient for us! Just leave it on its old - * transaction. - * - * In case (2), the buffer must not already exist as metadata - * --- that would violate write ordering (a transaction is free - * to write its data at any point, even before the previous - * committing transaction has committed). The caller must - * never, ever allow this to happen: there's nothing we can do - * about it in this layer. - */ - jbd_lock_bh_state(bh); - spin_lock(&journal->j_list_lock); - - /* Now that we have bh_state locked, are we really still mapped? */ - if (!buffer_mapped(bh)) { - JBUFFER_TRACE(jh, "unmapped buffer, bailing out"); - goto no_journal; - } - - if (jh->b_transaction) { - JBUFFER_TRACE(jh, "has transaction"); - if (jh->b_transaction != handle->h_transaction) { - JBUFFER_TRACE(jh, "belongs to older transaction"); - J_ASSERT_JH(jh, jh->b_transaction == - journal->j_committing_transaction); - - /* @@@ IS THIS TRUE ? */ - /* - * Not any more. Scenario: someone does a write() - * in data=journal mode. The buffer's transaction has - * moved into commit. Then someone does another - * write() to the file. We do the frozen data copyout - * and set b_next_transaction to point to j_running_t. - * And while we're in that state, someone does a - * writepage() in an attempt to pageout the same area - * of the file via a shared mapping. At present that - * calls journal_dirty_data(), and we get right here. - * It may be too late to journal the data. Simply - * falling through to the next test will suffice: the - * data will be dirty and wil be checkpointed. The - * ordering comments in the next comment block still - * apply. - */ - //J_ASSERT_JH(jh, jh->b_next_transaction == NULL); - - /* - * If we're journalling data, and this buffer was - * subject to a write(), it could be metadata, forget - * or shadow against the committing transaction. Now, - * someone has dirtied the same darn page via a mapping - * and it is being writepage()'d. - * We *could* just steal the page from commit, with some - * fancy locking there. Instead, we just skip it - - * don't tie the page's buffers to the new transaction - * at all. - * Implication: if we crash before the writepage() data - * is written into the filesystem, recovery will replay - * the write() data. - */ - if (jh->b_jlist != BJ_None && - jh->b_jlist != BJ_SyncData && - jh->b_jlist != BJ_Locked) { - JBUFFER_TRACE(jh, "Not stealing"); - goto no_journal; - } - - /* - * This buffer may be undergoing writeout in commit. We - * can't return from here and let the caller dirty it - * again because that can cause the write-out loop in - * commit to never terminate. - */ - if (buffer_dirty(bh)) { - get_bh(bh); - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - need_brelse = 1; - sync_dirty_buffer(bh); - jbd_lock_bh_state(bh); - spin_lock(&journal->j_list_lock); - /* Since we dropped the lock... */ - if (!buffer_mapped(bh)) { - JBUFFER_TRACE(jh, "buffer got unmapped"); - goto no_journal; - } - /* The buffer may become locked again at any - time if it is redirtied */ - } - - /* - * We cannot remove the buffer with io error from the - * committing transaction, because otherwise it would - * miss the error and the commit would not abort. - */ - if (unlikely(!buffer_uptodate(bh))) { - ret = -EIO; - goto no_journal; - } - /* We might have slept so buffer could be refiled now */ - if (jh->b_transaction != NULL && - jh->b_transaction != handle->h_transaction) { - JBUFFER_TRACE(jh, "unfile from commit"); - __journal_temp_unlink_buffer(jh); - /* It still points to the committing - * transaction; move it to this one so - * that the refile assert checks are - * happy. */ - jh->b_transaction = handle->h_transaction; - } - /* The buffer will be refiled below */ - - } - /* - * Special case --- the buffer might actually have been - * allocated and then immediately deallocated in the previous, - * committing transaction, so might still be left on that - * transaction's metadata lists. - */ - if (jh->b_jlist != BJ_SyncData && jh->b_jlist != BJ_Locked) { - JBUFFER_TRACE(jh, "not on correct data list: unfile"); - J_ASSERT_JH(jh, jh->b_jlist != BJ_Shadow); - JBUFFER_TRACE(jh, "file as data"); - __journal_file_buffer(jh, handle->h_transaction, - BJ_SyncData); - } - } else { - JBUFFER_TRACE(jh, "not on a transaction"); - __journal_file_buffer(jh, handle->h_transaction, BJ_SyncData); - } -no_journal: - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - if (need_brelse) { - BUFFER_TRACE(bh, "brelse"); - __brelse(bh); - } - JBUFFER_TRACE(jh, "exit"); - journal_put_journal_head(jh); - return ret; -} - -/** - * int journal_dirty_metadata() - mark a buffer as containing dirty metadata - * @handle: transaction to add buffer to. - * @bh: buffer to mark - * - * Mark dirty metadata which needs to be journaled as part of the current - * transaction. - * - * The buffer is placed on the transaction's metadata list and is marked - * as belonging to the transaction. - * - * Returns error number or 0 on success. - * - * Special care needs to be taken if the buffer already belongs to the - * current committing transaction (in which case we should have frozen - * data present for that commit). In that case, we don't relink the - * buffer: that only gets done when the old transaction finally - * completes its commit. - */ -int journal_dirty_metadata(handle_t *handle, struct buffer_head *bh) -{ - transaction_t *transaction = handle->h_transaction; - journal_t *journal = transaction->t_journal; - struct journal_head *jh = bh2jh(bh); - - jbd_debug(5, "journal_head %p\n", jh); - JBUFFER_TRACE(jh, "entry"); - if (is_handle_aborted(handle)) - goto out; - - jbd_lock_bh_state(bh); - - if (jh->b_modified == 0) { - /* - * This buffer's got modified and becoming part - * of the transaction. This needs to be done - * once a transaction -bzzz - */ - jh->b_modified = 1; - J_ASSERT_JH(jh, handle->h_buffer_credits > 0); - handle->h_buffer_credits--; - } - - /* - * fastpath, to avoid expensive locking. If this buffer is already - * on the running transaction's metadata list there is nothing to do. - * Nobody can take it off again because there is a handle open. - * I _think_ we're OK here with SMP barriers - a mistaken decision will - * result in this test being false, so we go in and take the locks. - */ - if (jh->b_transaction == transaction && jh->b_jlist == BJ_Metadata) { - JBUFFER_TRACE(jh, "fastpath"); - J_ASSERT_JH(jh, jh->b_transaction == - journal->j_running_transaction); - goto out_unlock_bh; - } - - set_buffer_jbddirty(bh); - - /* - * Metadata already on the current transaction list doesn't - * need to be filed. Metadata on another transaction's list must - * be committing, and will be refiled once the commit completes: - * leave it alone for now. - */ - if (jh->b_transaction != transaction) { - JBUFFER_TRACE(jh, "already on other transaction"); - J_ASSERT_JH(jh, jh->b_transaction == - journal->j_committing_transaction); - J_ASSERT_JH(jh, jh->b_next_transaction == transaction); - /* And this case is illegal: we can't reuse another - * transaction's data buffer, ever. */ - goto out_unlock_bh; - } - - /* That test should have eliminated the following case: */ - J_ASSERT_JH(jh, jh->b_frozen_data == NULL); - - JBUFFER_TRACE(jh, "file as BJ_Metadata"); - spin_lock(&journal->j_list_lock); - __journal_file_buffer(jh, handle->h_transaction, BJ_Metadata); - spin_unlock(&journal->j_list_lock); -out_unlock_bh: - jbd_unlock_bh_state(bh); -out: - JBUFFER_TRACE(jh, "exit"); - return 0; -} - -/* - * journal_release_buffer: undo a get_write_access without any buffer - * updates, if the update decided in the end that it didn't need access. - * - */ -void -journal_release_buffer(handle_t *handle, struct buffer_head *bh) -{ - BUFFER_TRACE(bh, "entry"); -} - -/** - * void journal_forget() - bforget() for potentially-journaled buffers. - * @handle: transaction handle - * @bh: bh to 'forget' - * - * We can only do the bforget if there are no commits pending against the - * buffer. If the buffer is dirty in the current running transaction we - * can safely unlink it. - * - * bh may not be a journalled buffer at all - it may be a non-JBD - * buffer which came off the hashtable. Check for this. - * - * Decrements bh->b_count by one. - * - * Allow this call even if the handle has aborted --- it may be part of - * the caller's cleanup after an abort. - */ -int journal_forget (handle_t *handle, struct buffer_head *bh) -{ - transaction_t *transaction = handle->h_transaction; - journal_t *journal = transaction->t_journal; - struct journal_head *jh; - int drop_reserve = 0; - int err = 0; - int was_modified = 0; - - BUFFER_TRACE(bh, "entry"); - - jbd_lock_bh_state(bh); - spin_lock(&journal->j_list_lock); - - if (!buffer_jbd(bh)) - goto not_jbd; - jh = bh2jh(bh); - - /* Critical error: attempting to delete a bitmap buffer, maybe? - * Don't do any jbd operations, and return an error. */ - if (!J_EXPECT_JH(jh, !jh->b_committed_data, - "inconsistent data on disk")) { - err = -EIO; - goto not_jbd; - } - - /* keep track of whether or not this transaction modified us */ - was_modified = jh->b_modified; - - /* - * The buffer's going from the transaction, we must drop - * all references -bzzz - */ - jh->b_modified = 0; - - if (jh->b_transaction == handle->h_transaction) { - J_ASSERT_JH(jh, !jh->b_frozen_data); - - /* If we are forgetting a buffer which is already part - * of this transaction, then we can just drop it from - * the transaction immediately. */ - clear_buffer_dirty(bh); - clear_buffer_jbddirty(bh); - - JBUFFER_TRACE(jh, "belongs to current transaction: unfile"); - - /* - * we only want to drop a reference if this transaction - * modified the buffer - */ - if (was_modified) - drop_reserve = 1; - - /* - * We are no longer going to journal this buffer. - * However, the commit of this transaction is still - * important to the buffer: the delete that we are now - * processing might obsolete an old log entry, so by - * committing, we can satisfy the buffer's checkpoint. - * - * So, if we have a checkpoint on the buffer, we should - * now refile the buffer on our BJ_Forget list so that - * we know to remove the checkpoint after we commit. - */ - - if (jh->b_cp_transaction) { - __journal_temp_unlink_buffer(jh); - __journal_file_buffer(jh, transaction, BJ_Forget); - } else { - __journal_unfile_buffer(jh); - if (!buffer_jbd(bh)) { - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - __bforget(bh); - goto drop; - } - } - } else if (jh->b_transaction) { - J_ASSERT_JH(jh, (jh->b_transaction == - journal->j_committing_transaction)); - /* However, if the buffer is still owned by a prior - * (committing) transaction, we can't drop it yet... */ - JBUFFER_TRACE(jh, "belongs to older transaction"); - /* ... but we CAN drop it from the new transaction if we - * have also modified it since the original commit. */ - - if (jh->b_next_transaction) { - J_ASSERT(jh->b_next_transaction == transaction); - jh->b_next_transaction = NULL; - - /* - * only drop a reference if this transaction modified - * the buffer - */ - if (was_modified) - drop_reserve = 1; - } - } - -not_jbd: - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - __brelse(bh); -drop: - if (drop_reserve) { - /* no need to reserve log space for this block -bzzz */ - handle->h_buffer_credits++; - } - return err; -} - -/** - * int journal_stop() - complete a transaction - * @handle: tranaction to complete. - * - * All done for a particular handle. - * - * There is not much action needed here. We just return any remaining - * buffer credits to the transaction and remove the handle. The only - * complication is that we need to start a commit operation if the - * filesystem is marked for synchronous update. - * - * journal_stop itself will not usually return an error, but it may - * do so in unusual circumstances. In particular, expect it to - * return -EIO if a journal_abort has been executed since the - * transaction began. - */ -int journal_stop(handle_t *handle) -{ - transaction_t *transaction = handle->h_transaction; - journal_t *journal = transaction->t_journal; - int err; - pid_t pid; - - J_ASSERT(journal_current_handle() == handle); - - if (is_handle_aborted(handle)) - err = -EIO; - else { - J_ASSERT(transaction->t_updates > 0); - err = 0; - } - - if (--handle->h_ref > 0) { - jbd_debug(4, "h_ref %d -> %d\n", handle->h_ref + 1, - handle->h_ref); - return err; - } - - jbd_debug(4, "Handle %p going down\n", handle); - - /* - * Implement synchronous transaction batching. If the handle - * was synchronous, don't force a commit immediately. Let's - * yield and let another thread piggyback onto this transaction. - * Keep doing that while new threads continue to arrive. - * It doesn't cost much - we're about to run a commit and sleep - * on IO anyway. Speeds up many-threaded, many-dir operations - * by 30x or more... - * - * We try and optimize the sleep time against what the underlying disk - * can do, instead of having a static sleep time. This is useful for - * the case where our storage is so fast that it is more optimal to go - * ahead and force a flush and wait for the transaction to be committed - * than it is to wait for an arbitrary amount of time for new writers to - * join the transaction. We achieve this by measuring how long it takes - * to commit a transaction, and compare it with how long this - * transaction has been running, and if run time < commit time then we - * sleep for the delta and commit. This greatly helps super fast disks - * that would see slowdowns as more threads started doing fsyncs. - * - * But don't do this if this process was the most recent one to - * perform a synchronous write. We do this to detect the case where a - * single process is doing a stream of sync writes. No point in waiting - * for joiners in that case. - */ - pid = current->pid; - if (handle->h_sync && journal->j_last_sync_writer != pid) { - u64 commit_time, trans_time; - - journal->j_last_sync_writer = pid; - - spin_lock(&journal->j_state_lock); - commit_time = journal->j_average_commit_time; - spin_unlock(&journal->j_state_lock); - - trans_time = ktime_to_ns(ktime_sub(ktime_get(), - transaction->t_start_time)); - - commit_time = min_t(u64, commit_time, - 1000*jiffies_to_usecs(1)); - - if (trans_time < commit_time) { - ktime_t expires = ktime_add_ns(ktime_get(), - commit_time); - set_current_state(TASK_UNINTERRUPTIBLE); - schedule_hrtimeout(&expires, HRTIMER_MODE_ABS); - } - } - - current->journal_info = NULL; - spin_lock(&journal->j_state_lock); - spin_lock(&transaction->t_handle_lock); - transaction->t_outstanding_credits -= handle->h_buffer_credits; - transaction->t_updates--; - if (!transaction->t_updates) { - wake_up(&journal->j_wait_updates); - if (journal->j_barrier_count) - wake_up(&journal->j_wait_transaction_locked); - } - - /* - * If the handle is marked SYNC, we need to set another commit - * going! We also want to force a commit if the current - * transaction is occupying too much of the log, or if the - * transaction is too old now. - */ - if (handle->h_sync || - transaction->t_outstanding_credits > - journal->j_max_transaction_buffers || - time_after_eq(jiffies, transaction->t_expires)) { - /* Do this even for aborted journals: an abort still - * completes the commit thread, it just doesn't write - * anything to disk. */ - tid_t tid = transaction->t_tid; - - spin_unlock(&transaction->t_handle_lock); - jbd_debug(2, "transaction too old, requesting commit for " - "handle %p\n", handle); - /* This is non-blocking */ - __log_start_commit(journal, transaction->t_tid); - spin_unlock(&journal->j_state_lock); - - /* - * Special case: JFS_SYNC synchronous updates require us - * to wait for the commit to complete. - */ - if (handle->h_sync && !(current->flags & PF_MEMALLOC)) - err = log_wait_commit(journal, tid); - } else { - spin_unlock(&transaction->t_handle_lock); - spin_unlock(&journal->j_state_lock); - } - - lock_map_release(&handle->h_lockdep_map); - - jbd_free_handle(handle); - return err; -} - -/** - * int journal_force_commit() - force any uncommitted transactions - * @journal: journal to force - * - * For synchronous operations: force any uncommitted transactions - * to disk. May seem kludgy, but it reuses all the handle batching - * code in a very simple manner. - */ -int journal_force_commit(journal_t *journal) -{ - handle_t *handle; - int ret; - - handle = journal_start(journal, 1); - if (IS_ERR(handle)) { - ret = PTR_ERR(handle); - } else { - handle->h_sync = 1; - ret = journal_stop(handle); - } - return ret; -} - -/* - * - * List management code snippets: various functions for manipulating the - * transaction buffer lists. - * - */ - -/* - * Append a buffer to a transaction list, given the transaction's list head - * pointer. - * - * j_list_lock is held. - * - * jbd_lock_bh_state(jh2bh(jh)) is held. - */ - -static inline void -__blist_add_buffer(struct journal_head **list, struct journal_head *jh) -{ - if (!*list) { - jh->b_tnext = jh->b_tprev = jh; - *list = jh; - } else { - /* Insert at the tail of the list to preserve order */ - struct journal_head *first = *list, *last = first->b_tprev; - jh->b_tprev = last; - jh->b_tnext = first; - last->b_tnext = first->b_tprev = jh; - } -} - -/* - * Remove a buffer from a transaction list, given the transaction's list - * head pointer. - * - * Called with j_list_lock held, and the journal may not be locked. - * - * jbd_lock_bh_state(jh2bh(jh)) is held. - */ - -static inline void -__blist_del_buffer(struct journal_head **list, struct journal_head *jh) -{ - if (*list == jh) { - *list = jh->b_tnext; - if (*list == jh) - *list = NULL; - } - jh->b_tprev->b_tnext = jh->b_tnext; - jh->b_tnext->b_tprev = jh->b_tprev; -} - -/* - * Remove a buffer from the appropriate transaction list. - * - * Note that this function can *change* the value of - * bh->b_transaction->t_sync_datalist, t_buffers, t_forget, - * t_iobuf_list, t_shadow_list, t_log_list or t_reserved_list. If the caller - * is holding onto a copy of one of thee pointers, it could go bad. - * Generally the caller needs to re-read the pointer from the transaction_t. - * - * Called under j_list_lock. The journal may not be locked. - */ -static void __journal_temp_unlink_buffer(struct journal_head *jh) -{ - struct journal_head **list = NULL; - transaction_t *transaction; - struct buffer_head *bh = jh2bh(jh); - - J_ASSERT_JH(jh, jbd_is_locked_bh_state(bh)); - transaction = jh->b_transaction; - if (transaction) - assert_spin_locked(&transaction->t_journal->j_list_lock); - - J_ASSERT_JH(jh, jh->b_jlist < BJ_Types); - if (jh->b_jlist != BJ_None) - J_ASSERT_JH(jh, transaction != NULL); - - switch (jh->b_jlist) { - case BJ_None: - return; - case BJ_SyncData: - list = &transaction->t_sync_datalist; - break; - case BJ_Metadata: - transaction->t_nr_buffers--; - J_ASSERT_JH(jh, transaction->t_nr_buffers >= 0); - list = &transaction->t_buffers; - break; - case BJ_Forget: - list = &transaction->t_forget; - break; - case BJ_IO: - list = &transaction->t_iobuf_list; - break; - case BJ_Shadow: - list = &transaction->t_shadow_list; - break; - case BJ_LogCtl: - list = &transaction->t_log_list; - break; - case BJ_Reserved: - list = &transaction->t_reserved_list; - break; - case BJ_Locked: - list = &transaction->t_locked_list; - break; - } - - __blist_del_buffer(list, jh); - jh->b_jlist = BJ_None; - if (test_clear_buffer_jbddirty(bh)) - mark_buffer_dirty(bh); /* Expose it to the VM */ -} - -/* - * Remove buffer from all transactions. - * - * Called with bh_state lock and j_list_lock - * - * jh and bh may be already freed when this function returns. - */ -void __journal_unfile_buffer(struct journal_head *jh) -{ - __journal_temp_unlink_buffer(jh); - jh->b_transaction = NULL; - journal_put_journal_head(jh); -} - -void journal_unfile_buffer(journal_t *journal, struct journal_head *jh) -{ - struct buffer_head *bh = jh2bh(jh); - - /* Get reference so that buffer cannot be freed before we unlock it */ - get_bh(bh); - jbd_lock_bh_state(bh); - spin_lock(&journal->j_list_lock); - __journal_unfile_buffer(jh); - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - __brelse(bh); -} - -/* - * Called from journal_try_to_free_buffers(). - * - * Called under jbd_lock_bh_state(bh) - */ -static void -__journal_try_to_free_buffer(journal_t *journal, struct buffer_head *bh) -{ - struct journal_head *jh; - - jh = bh2jh(bh); - - if (buffer_locked(bh) || buffer_dirty(bh)) - goto out; - - if (jh->b_next_transaction != NULL) - goto out; - - spin_lock(&journal->j_list_lock); - if (jh->b_transaction != NULL && jh->b_cp_transaction == NULL) { - if (jh->b_jlist == BJ_SyncData || jh->b_jlist == BJ_Locked) { - /* A written-back ordered data buffer */ - JBUFFER_TRACE(jh, "release data"); - __journal_unfile_buffer(jh); - } - } else if (jh->b_cp_transaction != NULL && jh->b_transaction == NULL) { - /* written-back checkpointed metadata buffer */ - if (jh->b_jlist == BJ_None) { - JBUFFER_TRACE(jh, "remove from checkpoint list"); - __journal_remove_checkpoint(jh); - } - } - spin_unlock(&journal->j_list_lock); -out: - return; -} - -/** - * int journal_try_to_free_buffers() - try to free page buffers. - * @journal: journal for operation - * @page: to try and free - * @gfp_mask: we use the mask to detect how hard should we try to release - * buffers. If __GFP_WAIT and __GFP_FS is set, we wait for commit code to - * release the buffers. - * - * - * For all the buffers on this page, - * if they are fully written out ordered data, move them onto BUF_CLEAN - * so try_to_free_buffers() can reap them. - * - * This function returns non-zero if we wish try_to_free_buffers() - * to be called. We do this if the page is releasable by try_to_free_buffers(). - * We also do it if the page has locked or dirty buffers and the caller wants - * us to perform sync or async writeout. - * - * This complicates JBD locking somewhat. We aren't protected by the - * BKL here. We wish to remove the buffer from its committing or - * running transaction's ->t_datalist via __journal_unfile_buffer. - * - * This may *change* the value of transaction_t->t_datalist, so anyone - * who looks at t_datalist needs to lock against this function. - * - * Even worse, someone may be doing a journal_dirty_data on this - * buffer. So we need to lock against that. journal_dirty_data() - * will come out of the lock with the buffer dirty, which makes it - * ineligible for release here. - * - * Who else is affected by this? hmm... Really the only contender - * is do_get_write_access() - it could be looking at the buffer while - * journal_try_to_free_buffer() is changing its state. But that - * cannot happen because we never reallocate freed data as metadata - * while the data is part of a transaction. Yes? - * - * Return 0 on failure, 1 on success - */ -int journal_try_to_free_buffers(journal_t *journal, - struct page *page, gfp_t gfp_mask) -{ - struct buffer_head *head; - struct buffer_head *bh; - int ret = 0; - - J_ASSERT(PageLocked(page)); - - head = page_buffers(page); - bh = head; - do { - struct journal_head *jh; - - /* - * We take our own ref against the journal_head here to avoid - * having to add tons of locking around each instance of - * journal_put_journal_head(). - */ - jh = journal_grab_journal_head(bh); - if (!jh) - continue; - - jbd_lock_bh_state(bh); - __journal_try_to_free_buffer(journal, bh); - journal_put_journal_head(jh); - jbd_unlock_bh_state(bh); - if (buffer_jbd(bh)) - goto busy; - } while ((bh = bh->b_this_page) != head); - - ret = try_to_free_buffers(page); - -busy: - return ret; -} - -/* - * This buffer is no longer needed. If it is on an older transaction's - * checkpoint list we need to record it on this transaction's forget list - * to pin this buffer (and hence its checkpointing transaction) down until - * this transaction commits. If the buffer isn't on a checkpoint list, we - * release it. - * Returns non-zero if JBD no longer has an interest in the buffer. - * - * Called under j_list_lock. - * - * Called under jbd_lock_bh_state(bh). - */ -static int __dispose_buffer(struct journal_head *jh, transaction_t *transaction) -{ - int may_free = 1; - struct buffer_head *bh = jh2bh(jh); - - if (jh->b_cp_transaction) { - JBUFFER_TRACE(jh, "on running+cp transaction"); - __journal_temp_unlink_buffer(jh); - /* - * We don't want to write the buffer anymore, clear the - * bit so that we don't confuse checks in - * __journal_file_buffer - */ - clear_buffer_dirty(bh); - __journal_file_buffer(jh, transaction, BJ_Forget); - may_free = 0; - } else { - JBUFFER_TRACE(jh, "on running transaction"); - __journal_unfile_buffer(jh); - } - return may_free; -} - -/* - * journal_invalidatepage - * - * This code is tricky. It has a number of cases to deal with. - * - * There are two invariants which this code relies on: - * - * i_size must be updated on disk before we start calling invalidatepage on the - * data. - * - * This is done in ext3 by defining an ext3_setattr method which - * updates i_size before truncate gets going. By maintaining this - * invariant, we can be sure that it is safe to throw away any buffers - * attached to the current transaction: once the transaction commits, - * we know that the data will not be needed. - * - * Note however that we can *not* throw away data belonging to the - * previous, committing transaction! - * - * Any disk blocks which *are* part of the previous, committing - * transaction (and which therefore cannot be discarded immediately) are - * not going to be reused in the new running transaction - * - * The bitmap committed_data images guarantee this: any block which is - * allocated in one transaction and removed in the next will be marked - * as in-use in the committed_data bitmap, so cannot be reused until - * the next transaction to delete the block commits. This means that - * leaving committing buffers dirty is quite safe: the disk blocks - * cannot be reallocated to a different file and so buffer aliasing is - * not possible. - * - * - * The above applies mainly to ordered data mode. In writeback mode we - * don't make guarantees about the order in which data hits disk --- in - * particular we don't guarantee that new dirty data is flushed before - * transaction commit --- so it is always safe just to discard data - * immediately in that mode. --sct - */ - -/* - * The journal_unmap_buffer helper function returns zero if the buffer - * concerned remains pinned as an anonymous buffer belonging to an older - * transaction. - * - * We're outside-transaction here. Either or both of j_running_transaction - * and j_committing_transaction may be NULL. - */ -static int journal_unmap_buffer(journal_t *journal, struct buffer_head *bh, - int partial_page) -{ - transaction_t *transaction; - struct journal_head *jh; - int may_free = 1; - - BUFFER_TRACE(bh, "entry"); - -retry: - /* - * It is safe to proceed here without the j_list_lock because the - * buffers cannot be stolen by try_to_free_buffers as long as we are - * holding the page lock. --sct - */ - - if (!buffer_jbd(bh)) - goto zap_buffer_unlocked; - - spin_lock(&journal->j_state_lock); - jbd_lock_bh_state(bh); - spin_lock(&journal->j_list_lock); - - jh = journal_grab_journal_head(bh); - if (!jh) - goto zap_buffer_no_jh; - - /* - * We cannot remove the buffer from checkpoint lists until the - * transaction adding inode to orphan list (let's call it T) - * is committed. Otherwise if the transaction changing the - * buffer would be cleaned from the journal before T is - * committed, a crash will cause that the correct contents of - * the buffer will be lost. On the other hand we have to - * clear the buffer dirty bit at latest at the moment when the - * transaction marking the buffer as freed in the filesystem - * structures is committed because from that moment on the - * block can be reallocated and used by a different page. - * Since the block hasn't been freed yet but the inode has - * already been added to orphan list, it is safe for us to add - * the buffer to BJ_Forget list of the newest transaction. - * - * Also we have to clear buffer_mapped flag of a truncated buffer - * because the buffer_head may be attached to the page straddling - * i_size (can happen only when blocksize < pagesize) and thus the - * buffer_head can be reused when the file is extended again. So we end - * up keeping around invalidated buffers attached to transactions' - * BJ_Forget list just to stop checkpointing code from cleaning up - * the transaction this buffer was modified in. - */ - transaction = jh->b_transaction; - if (transaction == NULL) { - /* First case: not on any transaction. If it - * has no checkpoint link, then we can zap it: - * it's a writeback-mode buffer so we don't care - * if it hits disk safely. */ - if (!jh->b_cp_transaction) { - JBUFFER_TRACE(jh, "not on any transaction: zap"); - goto zap_buffer; - } - - if (!buffer_dirty(bh)) { - /* bdflush has written it. We can drop it now */ - goto zap_buffer; - } - - /* OK, it must be in the journal but still not - * written fully to disk: it's metadata or - * journaled data... */ - - if (journal->j_running_transaction) { - /* ... and once the current transaction has - * committed, the buffer won't be needed any - * longer. */ - JBUFFER_TRACE(jh, "checkpointed: add to BJ_Forget"); - may_free = __dispose_buffer(jh, - journal->j_running_transaction); - goto zap_buffer; - } else { - /* There is no currently-running transaction. So the - * orphan record which we wrote for this file must have - * passed into commit. We must attach this buffer to - * the committing transaction, if it exists. */ - if (journal->j_committing_transaction) { - JBUFFER_TRACE(jh, "give to committing trans"); - may_free = __dispose_buffer(jh, - journal->j_committing_transaction); - goto zap_buffer; - } else { - /* The orphan record's transaction has - * committed. We can cleanse this buffer */ - clear_buffer_jbddirty(bh); - goto zap_buffer; - } - } - } else if (transaction == journal->j_committing_transaction) { - JBUFFER_TRACE(jh, "on committing transaction"); - if (jh->b_jlist == BJ_Locked) { - /* - * The buffer is on the committing transaction's locked - * list. We have the buffer locked, so I/O has - * completed. So we can nail the buffer now. - */ - may_free = __dispose_buffer(jh, transaction); - goto zap_buffer; - } - /* - * The buffer is committing, we simply cannot touch - * it. If the page is straddling i_size we have to wait - * for commit and try again. - */ - if (partial_page) { - tid_t tid = journal->j_committing_transaction->t_tid; - - journal_put_journal_head(jh); - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - spin_unlock(&journal->j_state_lock); - unlock_buffer(bh); - log_wait_commit(journal, tid); - lock_buffer(bh); - goto retry; - } - /* - * OK, buffer won't be reachable after truncate. We just set - * j_next_transaction to the running transaction (if there is - * one) and mark buffer as freed so that commit code knows it - * should clear dirty bits when it is done with the buffer. - */ - set_buffer_freed(bh); - if (journal->j_running_transaction && buffer_jbddirty(bh)) - jh->b_next_transaction = journal->j_running_transaction; - journal_put_journal_head(jh); - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - spin_unlock(&journal->j_state_lock); - return 0; - } else { - /* Good, the buffer belongs to the running transaction. - * We are writing our own transaction's data, not any - * previous one's, so it is safe to throw it away - * (remember that we expect the filesystem to have set - * i_size already for this truncate so recovery will not - * expose the disk blocks we are discarding here.) */ - J_ASSERT_JH(jh, transaction == journal->j_running_transaction); - JBUFFER_TRACE(jh, "on running transaction"); - may_free = __dispose_buffer(jh, transaction); - } - -zap_buffer: - /* - * This is tricky. Although the buffer is truncated, it may be reused - * if blocksize < pagesize and it is attached to the page straddling - * EOF. Since the buffer might have been added to BJ_Forget list of the - * running transaction, journal_get_write_access() won't clear - * b_modified and credit accounting gets confused. So clear b_modified - * here. */ - jh->b_modified = 0; - journal_put_journal_head(jh); -zap_buffer_no_jh: - spin_unlock(&journal->j_list_lock); - jbd_unlock_bh_state(bh); - spin_unlock(&journal->j_state_lock); -zap_buffer_unlocked: - clear_buffer_dirty(bh); - J_ASSERT_BH(bh, !buffer_jbddirty(bh)); - clear_buffer_mapped(bh); - clear_buffer_req(bh); - clear_buffer_new(bh); - bh->b_bdev = NULL; - return may_free; -} - -/** - * void journal_invalidatepage() - invalidate a journal page - * @journal: journal to use for flush - * @page: page to flush - * @offset: offset of the range to invalidate - * @length: length of the range to invalidate - * - * Reap page buffers containing data in specified range in page. - */ -void journal_invalidatepage(journal_t *journal, - struct page *page, - unsigned int offset, - unsigned int length) -{ - struct buffer_head *head, *bh, *next; - unsigned int stop = offset + length; - unsigned int curr_off = 0; - int partial_page = (offset || length < PAGE_CACHE_SIZE); - int may_free = 1; - - if (!PageLocked(page)) - BUG(); - if (!page_has_buffers(page)) - return; - - BUG_ON(stop > PAGE_CACHE_SIZE || stop < length); - - /* We will potentially be playing with lists other than just the - * data lists (especially for journaled data mode), so be - * cautious in our locking. */ - - head = bh = page_buffers(page); - do { - unsigned int next_off = curr_off + bh->b_size; - next = bh->b_this_page; - - if (next_off > stop) - return; - - if (offset <= curr_off) { - /* This block is wholly outside the truncation point */ - lock_buffer(bh); - may_free &= journal_unmap_buffer(journal, bh, - partial_page); - unlock_buffer(bh); - } - curr_off = next_off; - bh = next; - - } while (bh != head); - - if (!partial_page) { - if (may_free && try_to_free_buffers(page)) - J_ASSERT(!page_has_buffers(page)); - } -} - -/* - * File a buffer on the given transaction list. - */ -void __journal_file_buffer(struct journal_head *jh, - transaction_t *transaction, int jlist) -{ - struct journal_head **list = NULL; - int was_dirty = 0; - struct buffer_head *bh = jh2bh(jh); - - J_ASSERT_JH(jh, jbd_is_locked_bh_state(bh)); - assert_spin_locked(&transaction->t_journal->j_list_lock); - - J_ASSERT_JH(jh, jh->b_jlist < BJ_Types); - J_ASSERT_JH(jh, jh->b_transaction == transaction || - jh->b_transaction == NULL); - - if (jh->b_transaction && jh->b_jlist == jlist) - return; - - if (jlist == BJ_Metadata || jlist == BJ_Reserved || - jlist == BJ_Shadow || jlist == BJ_Forget) { - /* - * For metadata buffers, we track dirty bit in buffer_jbddirty - * instead of buffer_dirty. We should not see a dirty bit set - * here because we clear it in do_get_write_access but e.g. - * tune2fs can modify the sb and set the dirty bit at any time - * so we try to gracefully handle that. - */ - if (buffer_dirty(bh)) - warn_dirty_buffer(bh); - if (test_clear_buffer_dirty(bh) || - test_clear_buffer_jbddirty(bh)) - was_dirty = 1; - } - - if (jh->b_transaction) - __journal_temp_unlink_buffer(jh); - else - journal_grab_journal_head(bh); - jh->b_transaction = transaction; - - switch (jlist) { - case BJ_None: - J_ASSERT_JH(jh, !jh->b_committed_data); - J_ASSERT_JH(jh, !jh->b_frozen_data); - return; - case BJ_SyncData: - list = &transaction->t_sync_datalist; - break; - case BJ_Metadata: - transaction->t_nr_buffers++; - list = &transaction->t_buffers; - break; - case BJ_Forget: - list = &transaction->t_forget; - break; - case BJ_IO: - list = &transaction->t_iobuf_list; - break; - case BJ_Shadow: - list = &transaction->t_shadow_list; - break; - case BJ_LogCtl: - list = &transaction->t_log_list; - break; - case BJ_Reserved: - list = &transaction->t_reserved_list; - break; - case BJ_Locked: - list = &transaction->t_locked_list; - break; - } - - __blist_add_buffer(list, jh); - jh->b_jlist = jlist; - - if (was_dirty) - set_buffer_jbddirty(bh); -} - -void journal_file_buffer(struct journal_head *jh, - transaction_t *transaction, int jlist) -{ - jbd_lock_bh_state(jh2bh(jh)); - spin_lock(&transaction->t_journal->j_list_lock); - __journal_file_buffer(jh, transaction, jlist); - spin_unlock(&transaction->t_journal->j_list_lock); - jbd_unlock_bh_state(jh2bh(jh)); -} - -/* - * Remove a buffer from its current buffer list in preparation for - * dropping it from its current transaction entirely. If the buffer has - * already started to be used by a subsequent transaction, refile the - * buffer on that transaction's metadata list. - * - * Called under j_list_lock - * Called under jbd_lock_bh_state(jh2bh(jh)) - * - * jh and bh may be already free when this function returns - */ -void __journal_refile_buffer(struct journal_head *jh) -{ - int was_dirty, jlist; - struct buffer_head *bh = jh2bh(jh); - - J_ASSERT_JH(jh, jbd_is_locked_bh_state(bh)); - if (jh->b_transaction) - assert_spin_locked(&jh->b_transaction->t_journal->j_list_lock); - - /* If the buffer is now unused, just drop it. */ - if (jh->b_next_transaction == NULL) { - __journal_unfile_buffer(jh); - return; - } - - /* - * It has been modified by a later transaction: add it to the new - * transaction's metadata list. - */ - - was_dirty = test_clear_buffer_jbddirty(bh); - __journal_temp_unlink_buffer(jh); - /* - * We set b_transaction here because b_next_transaction will inherit - * our jh reference and thus __journal_file_buffer() must not take a - * new one. - */ - jh->b_transaction = jh->b_next_transaction; - jh->b_next_transaction = NULL; - if (buffer_freed(bh)) - jlist = BJ_Forget; - else if (jh->b_modified) - jlist = BJ_Metadata; - else - jlist = BJ_Reserved; - __journal_file_buffer(jh, jh->b_transaction, jlist); - J_ASSERT_JH(jh, jh->b_transaction->t_state == T_RUNNING); - - if (was_dirty) - set_buffer_jbddirty(bh); -} - -/* - * __journal_refile_buffer() with necessary locking added. We take our bh - * reference so that we can safely unlock bh. - * - * The jh and bh may be freed by this call. - */ -void journal_refile_buffer(journal_t *journal, struct journal_head *jh) -{ - struct buffer_head *bh = jh2bh(jh); - - /* Get reference so that buffer cannot be freed before we unlock it */ - get_bh(bh); - jbd_lock_bh_state(bh); - spin_lock(&journal->j_list_lock); - __journal_refile_buffer(jh); - jbd_unlock_bh_state(bh); - spin_unlock(&journal->j_list_lock); - __brelse(bh); -} diff --git a/fs/jbd2/checkpoint.c b/fs/jbd2/checkpoint.c index 4227dc4f7437d5..8c44654ce2748a 100644 --- a/fs/jbd2/checkpoint.c +++ b/fs/jbd2/checkpoint.c @@ -417,12 +417,12 @@ int jbd2_cleanup_journal_tail(journal_t *journal) * journal_clean_one_cp_list * * Find all the written-back checkpoint buffers in the given list and - * release them. + * release them. If 'destroy' is set, clean all buffers unconditionally. * * Called with j_list_lock held. * Returns 1 if we freed the transaction, 0 otherwise. */ -static int journal_clean_one_cp_list(struct journal_head *jh) +static int journal_clean_one_cp_list(struct journal_head *jh, bool destroy) { struct journal_head *last_jh; struct journal_head *next_jh = jh; @@ -436,7 +436,10 @@ static int journal_clean_one_cp_list(struct journal_head *jh) do { jh = next_jh; next_jh = jh->b_cpnext; - ret = __try_to_free_cp_buf(jh); + if (!destroy) + ret = __try_to_free_cp_buf(jh); + else + ret = __jbd2_journal_remove_checkpoint(jh) + 1; if (!ret) return freed; if (ret == 2) @@ -459,10 +462,11 @@ static int journal_clean_one_cp_list(struct journal_head *jh) * journal_clean_checkpoint_list * * Find all the written-back checkpoint buffers in the journal and release them. + * If 'destroy' is set, release all buffers unconditionally. * * Called with j_list_lock held. */ -void __jbd2_journal_clean_checkpoint_list(journal_t *journal) +void __jbd2_journal_clean_checkpoint_list(journal_t *journal, bool destroy) { transaction_t *transaction, *last_transaction, *next_transaction; int ret; @@ -476,7 +480,8 @@ void __jbd2_journal_clean_checkpoint_list(journal_t *journal) do { transaction = next_transaction; next_transaction = transaction->t_cpnext; - ret = journal_clean_one_cp_list(transaction->t_checkpoint_list); + ret = journal_clean_one_cp_list(transaction->t_checkpoint_list, + destroy); /* * This function only frees up some memory if possible so we * dont have an obligation to finish processing. Bail out if @@ -492,7 +497,7 @@ void __jbd2_journal_clean_checkpoint_list(journal_t *journal) * we can possibly see not yet submitted buffers on io_list */ ret = journal_clean_one_cp_list(transaction-> - t_checkpoint_io_list); + t_checkpoint_io_list, destroy); if (need_resched()) return; /* @@ -505,6 +510,28 @@ void __jbd2_journal_clean_checkpoint_list(journal_t *journal) } while (transaction != last_transaction); } +/* + * Remove buffers from all checkpoint lists as journal is aborted and we just + * need to free memory + */ +void jbd2_journal_destroy_checkpoint(journal_t *journal) +{ + /* + * We loop because __jbd2_journal_clean_checkpoint_list() may abort + * early due to a need of rescheduling. + */ + while (1) { + spin_lock(&journal->j_list_lock); + if (!journal->j_checkpoint_transactions) { + spin_unlock(&journal->j_list_lock); + break; + } + __jbd2_journal_clean_checkpoint_list(journal, true); + spin_unlock(&journal->j_list_lock); + cond_resched(); + } +} + /* * journal_remove_checkpoint: called after a buffer has been committed * to disk (either by being write-back flushed to disk, or being diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c index b73e0215baa7ce..362e5f614450e5 100644 --- a/fs/jbd2/commit.c +++ b/fs/jbd2/commit.c @@ -510,7 +510,7 @@ void jbd2_journal_commit_transaction(journal_t *journal) * frees some memory */ spin_lock(&journal->j_list_lock); - __jbd2_journal_clean_checkpoint_list(journal); + __jbd2_journal_clean_checkpoint_list(journal, false); spin_unlock(&journal->j_list_lock); jbd_debug(3, "JBD2: commit phase 1\n"); diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c index 4ff3fad4e9e3f5..8270fe9e3641eb 100644 --- a/fs/jbd2/journal.c +++ b/fs/jbd2/journal.c @@ -1456,7 +1456,7 @@ void jbd2_journal_update_sb_errno(journal_t *journal) sb->s_errno = cpu_to_be32(journal->j_errno); read_unlock(&journal->j_state_lock); - jbd2_write_superblock(journal, WRITE_SYNC); + jbd2_write_superblock(journal, WRITE_FUA); } EXPORT_SYMBOL(jbd2_journal_update_sb_errno); @@ -1693,8 +1693,17 @@ int jbd2_journal_destroy(journal_t *journal) while (journal->j_checkpoint_transactions != NULL) { spin_unlock(&journal->j_list_lock); mutex_lock(&journal->j_checkpoint_mutex); - jbd2_log_do_checkpoint(journal); + err = jbd2_log_do_checkpoint(journal); mutex_unlock(&journal->j_checkpoint_mutex); + /* + * If checkpointing failed, just free the buffers to avoid + * looping forever + */ + if (err) { + jbd2_journal_destroy_checkpoint(journal); + spin_lock(&journal->j_list_lock); + break; + } spin_lock(&journal->j_list_lock); } diff --git a/fs/jbd2/transaction.c b/fs/jbd2/transaction.c index f3d06174b05100..6b8338ec246454 100644 --- a/fs/jbd2/transaction.c +++ b/fs/jbd2/transaction.c @@ -204,6 +204,20 @@ static int add_transaction_credits(journal_t *journal, int blocks, * attach this handle to a new transaction. */ atomic_sub(total, &t->t_outstanding_credits); + + /* + * Is the number of reserved credits in the current transaction too + * big to fit this handle? Wait until reserved credits are freed. + */ + if (atomic_read(&journal->j_reserved_credits) + total > + journal->j_max_transaction_buffers) { + read_unlock(&journal->j_state_lock); + wait_event(journal->j_wait_reserved, + atomic_read(&journal->j_reserved_credits) + total <= + journal->j_max_transaction_buffers); + return 1; + } + wait_transaction_locked(journal); return 1; } @@ -262,20 +276,24 @@ static int start_this_handle(journal_t *journal, handle_t *handle, int rsv_blocks = 0; unsigned long ts = jiffies; + if (handle->h_rsv_handle) + rsv_blocks = handle->h_rsv_handle->h_buffer_credits; + /* - * 1/2 of transaction can be reserved so we can practically handle - * only 1/2 of maximum transaction size per operation + * Limit the number of reserved credits to 1/2 of maximum transaction + * size and limit the number of total credits to not exceed maximum + * transaction size per operation. */ - if (WARN_ON(blocks > journal->j_max_transaction_buffers / 2)) { - printk(KERN_ERR "JBD2: %s wants too many credits (%d > %d)\n", - current->comm, blocks, - journal->j_max_transaction_buffers / 2); + if ((rsv_blocks > journal->j_max_transaction_buffers / 2) || + (rsv_blocks + blocks > journal->j_max_transaction_buffers)) { + printk(KERN_ERR "JBD2: %s wants too many credits " + "credits:%d rsv_credits:%d max:%d\n", + current->comm, blocks, rsv_blocks, + journal->j_max_transaction_buffers); + WARN_ON(1); return -ENOSPC; } - if (handle->h_rsv_handle) - rsv_blocks = handle->h_rsv_handle->h_buffer_credits; - alloc_transaction: if (!journal->j_running_transaction) { /* @@ -1280,8 +1298,6 @@ void jbd2_buffer_abort_trigger(struct journal_head *jh, triggers->t_abort(triggers, jh2bh(jh)); } - - /** * int jbd2_journal_dirty_metadata() - mark a buffer as containing dirty metadata * @handle: transaction to add buffer to. @@ -1314,12 +1330,41 @@ int jbd2_journal_dirty_metadata(handle_t *handle, struct buffer_head *bh) if (is_handle_aborted(handle)) return -EROFS; - journal = transaction->t_journal; - jh = jbd2_journal_grab_journal_head(bh); - if (!jh) { + if (!buffer_jbd(bh)) { ret = -EUCLEAN; goto out; } + /* + * We don't grab jh reference here since the buffer must be part + * of the running transaction. + */ + jh = bh2jh(bh); + /* + * This and the following assertions are unreliable since we may see jh + * in inconsistent state unless we grab bh_state lock. But this is + * crucial to catch bugs so let's do a reliable check until the + * lockless handling is fully proven. + */ + if (jh->b_transaction != transaction && + jh->b_next_transaction != transaction) { + jbd_lock_bh_state(bh); + J_ASSERT_JH(jh, jh->b_transaction == transaction || + jh->b_next_transaction == transaction); + jbd_unlock_bh_state(bh); + } + if (jh->b_modified == 1) { + /* If it's in our transaction it must be in BJ_Metadata list. */ + if (jh->b_transaction == transaction && + jh->b_jlist != BJ_Metadata) { + jbd_lock_bh_state(bh); + J_ASSERT_JH(jh, jh->b_transaction != transaction || + jh->b_jlist == BJ_Metadata); + jbd_unlock_bh_state(bh); + } + goto out; + } + + journal = transaction->t_journal; jbd_debug(5, "journal_head %p\n", jh); JBUFFER_TRACE(jh, "entry"); @@ -1410,7 +1455,6 @@ int jbd2_journal_dirty_metadata(handle_t *handle, struct buffer_head *bh) spin_unlock(&journal->j_list_lock); out_unlock_bh: jbd_unlock_bh_state(bh); - jbd2_journal_put_journal_head(jh); out: JBUFFER_TRACE(jh, "exit"); return ret; diff --git a/fs/jfs/file.c b/fs/jfs/file.c index b9dc23cd04f206..0e026a7bdcd4d9 100644 --- a/fs/jfs/file.c +++ b/fs/jfs/file.c @@ -107,8 +107,11 @@ int jfs_setattr(struct dentry *dentry, struct iattr *iattr) if (rc) return rc; - if (is_quota_modification(inode, iattr)) - dquot_initialize(inode); + if (is_quota_modification(inode, iattr)) { + rc = dquot_initialize(inode); + if (rc) + return rc; + } if ((iattr->ia_valid & ATTR_UID && !uid_eq(iattr->ia_uid, inode->i_uid)) || (iattr->ia_valid & ATTR_GID && !gid_eq(iattr->ia_gid, inode->i_gid))) { rc = dquot_transfer(inode, iattr); diff --git a/fs/jfs/jfs_inode.c b/fs/jfs/jfs_inode.c index 6b0f816201a229..cf7936fe2e6823 100644 --- a/fs/jfs/jfs_inode.c +++ b/fs/jfs/jfs_inode.c @@ -109,7 +109,9 @@ struct inode *ialloc(struct inode *parent, umode_t mode) /* * Allocate inode to quota. */ - dquot_initialize(inode); + rc = dquot_initialize(inode); + if (rc) + goto fail_drop; rc = dquot_alloc_inode(inode); if (rc) goto fail_drop; diff --git a/fs/jfs/namei.c b/fs/jfs/namei.c index a5ac97b9a933af..35976bdccafcf3 100644 --- a/fs/jfs/namei.c +++ b/fs/jfs/namei.c @@ -86,7 +86,9 @@ static int jfs_create(struct inode *dip, struct dentry *dentry, umode_t mode, jfs_info("jfs_create: dip:0x%p name:%pd", dip, dentry); - dquot_initialize(dip); + rc = dquot_initialize(dip); + if (rc) + goto out1; /* * search parent directory for entry/freespace @@ -218,7 +220,9 @@ static int jfs_mkdir(struct inode *dip, struct dentry *dentry, umode_t mode) jfs_info("jfs_mkdir: dip:0x%p name:%pd", dip, dentry); - dquot_initialize(dip); + rc = dquot_initialize(dip); + if (rc) + goto out1; /* * search parent directory for entry/freespace @@ -355,8 +359,12 @@ static int jfs_rmdir(struct inode *dip, struct dentry *dentry) jfs_info("jfs_rmdir: dip:0x%p name:%pd", dip, dentry); /* Init inode for quota operations. */ - dquot_initialize(dip); - dquot_initialize(ip); + rc = dquot_initialize(dip); + if (rc) + goto out; + rc = dquot_initialize(ip); + if (rc) + goto out; /* directory must be empty to be removed */ if (!dtEmpty(ip)) { @@ -483,8 +491,12 @@ static int jfs_unlink(struct inode *dip, struct dentry *dentry) jfs_info("jfs_unlink: dip:0x%p name:%pd", dip, dentry); /* Init inode for quota operations. */ - dquot_initialize(dip); - dquot_initialize(ip); + rc = dquot_initialize(dip); + if (rc) + goto out; + rc = dquot_initialize(ip); + if (rc) + goto out; if ((rc = get_UCSname(&dname, dentry))) goto out; @@ -799,7 +811,9 @@ static int jfs_link(struct dentry *old_dentry, jfs_info("jfs_link: %pd %pd", old_dentry, dentry); - dquot_initialize(dir); + rc = dquot_initialize(dir); + if (rc) + goto out; tid = txBegin(ip->i_sb, 0); @@ -810,7 +824,7 @@ static int jfs_link(struct dentry *old_dentry, * scan parent directory for entry/freespace */ if ((rc = get_UCSname(&dname, dentry))) - goto out; + goto out_tx; if ((rc = dtSearch(dir, &dname, &ino, &btstack, JFS_CREATE))) goto free_dname; @@ -842,12 +856,13 @@ static int jfs_link(struct dentry *old_dentry, free_dname: free_UCSname(&dname); - out: + out_tx: txEnd(tid); mutex_unlock(&JFS_IP(ip)->commit_mutex); mutex_unlock(&JFS_IP(dir)->commit_mutex); + out: jfs_info("jfs_link: rc:%d", rc); return rc; } @@ -891,7 +906,9 @@ static int jfs_symlink(struct inode *dip, struct dentry *dentry, jfs_info("jfs_symlink: dip:0x%p name:%s", dip, name); - dquot_initialize(dip); + rc = dquot_initialize(dip); + if (rc) + goto out1; ssize = strlen(name) + 1; @@ -1082,8 +1099,12 @@ static int jfs_rename(struct inode *old_dir, struct dentry *old_dentry, jfs_info("jfs_rename: %pd %pd", old_dentry, new_dentry); - dquot_initialize(old_dir); - dquot_initialize(new_dir); + rc = dquot_initialize(old_dir); + if (rc) + goto out1; + rc = dquot_initialize(new_dir); + if (rc) + goto out1; old_ip = d_inode(old_dentry); new_ip = d_inode(new_dentry); @@ -1130,7 +1151,9 @@ static int jfs_rename(struct inode *old_dir, struct dentry *old_dentry, } else if (new_ip) { IWRITE_LOCK(new_ip, RDWRLOCK_NORMAL); /* Init inode for quota operations. */ - dquot_initialize(new_ip); + rc = dquot_initialize(new_ip); + if (rc) + goto out_unlock; } /* @@ -1318,6 +1341,7 @@ static int jfs_rename(struct inode *old_dir, struct dentry *old_dentry, clear_cflag(COMMIT_Stale, old_dir); } + out_unlock: if (new_ip && !S_ISDIR(new_ip->i_mode)) IWRITE_UNLOCK(new_ip); out3: @@ -1353,7 +1377,9 @@ static int jfs_mknod(struct inode *dir, struct dentry *dentry, jfs_info("jfs_mknod: %pd", dentry); - dquot_initialize(dir); + rc = dquot_initialize(dir); + if (rc) + goto out; if ((rc = get_UCSname(&dname, dentry))) goto out; diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c index 719f7f4c7a37bd..7210583b472f52 100644 --- a/fs/ocfs2/file.c +++ b/fs/ocfs2/file.c @@ -105,8 +105,11 @@ static int ocfs2_file_open(struct inode *inode, struct file *file) file->f_path.dentry->d_name.len, file->f_path.dentry->d_name.name, mode); - if (file->f_mode & FMODE_WRITE) - dquot_initialize(inode); + if (file->f_mode & FMODE_WRITE) { + status = dquot_initialize(inode); + if (status) + goto leave; + } spin_lock(&oi->ip_lock); @@ -1155,8 +1158,11 @@ int ocfs2_setattr(struct dentry *dentry, struct iattr *attr) if (status) return status; - if (is_quota_modification(inode, attr)) - dquot_initialize(inode); + if (is_quota_modification(inode, attr)) { + status = dquot_initialize(inode); + if (status) + return status; + } size_change = S_ISREG(inode->i_mode) && attr->ia_valid & ATTR_SIZE; if (size_change) { status = ocfs2_rw_lock(inode, 1); @@ -1209,8 +1215,8 @@ int ocfs2_setattr(struct dentry *dentry, struct iattr *attr) && OCFS2_HAS_RO_COMPAT_FEATURE(sb, OCFS2_FEATURE_RO_COMPAT_USRQUOTA)) { transfer_to[USRQUOTA] = dqget(sb, make_kqid_uid(attr->ia_uid)); - if (!transfer_to[USRQUOTA]) { - status = -ESRCH; + if (IS_ERR(transfer_to[USRQUOTA])) { + status = PTR_ERR(transfer_to[USRQUOTA]); goto bail_unlock; } } @@ -1218,8 +1224,8 @@ int ocfs2_setattr(struct dentry *dentry, struct iattr *attr) && OCFS2_HAS_RO_COMPAT_FEATURE(sb, OCFS2_FEATURE_RO_COMPAT_GRPQUOTA)) { transfer_to[GRPQUOTA] = dqget(sb, make_kqid_gid(attr->ia_gid)); - if (!transfer_to[GRPQUOTA]) { - status = -ESRCH; + if (IS_ERR(transfer_to[GRPQUOTA])) { + status = PTR_ERR(transfer_to[GRPQUOTA]); goto bail_unlock; } } diff --git a/fs/ocfs2/namei.c b/fs/ocfs2/namei.c index 6e6abb93fda590..948681e37cfdb3 100644 --- a/fs/ocfs2/namei.c +++ b/fs/ocfs2/namei.c @@ -200,11 +200,12 @@ static struct dentry *ocfs2_lookup(struct inode *dir, struct dentry *dentry, static struct inode *ocfs2_get_init_inode(struct inode *dir, umode_t mode) { struct inode *inode; + int status; inode = new_inode(dir->i_sb); if (!inode) { mlog(ML_ERROR, "new_inode failed!\n"); - return NULL; + return ERR_PTR(-ENOMEM); } /* populate as many fields early on as possible - many of @@ -213,7 +214,10 @@ static struct inode *ocfs2_get_init_inode(struct inode *dir, umode_t mode) if (S_ISDIR(mode)) set_nlink(inode, 2); inode_init_owner(inode, dir, mode); - dquot_initialize(inode); + status = dquot_initialize(inode); + if (status) + return ERR_PTR(status); + return inode; } @@ -264,7 +268,11 @@ static int ocfs2_mknod(struct inode *dir, (unsigned long long)OCFS2_I(dir)->ip_blkno, (unsigned long)dev, mode); - dquot_initialize(dir); + status = dquot_initialize(dir); + if (status) { + mlog_errno(status); + return status; + } /* get our super block */ osb = OCFS2_SB(dir->i_sb); @@ -311,8 +319,9 @@ static int ocfs2_mknod(struct inode *dir, } inode = ocfs2_get_init_inode(dir, mode); - if (!inode) { - status = -ENOMEM; + if (IS_ERR(inode)) { + status = PTR_ERR(inode); + inode = NULL; mlog_errno(status); goto leave; } @@ -708,7 +717,11 @@ static int ocfs2_link(struct dentry *old_dentry, if (S_ISDIR(inode->i_mode)) return -EPERM; - dquot_initialize(dir); + err = dquot_initialize(dir); + if (err) { + mlog_errno(err); + return err; + } err = ocfs2_double_lock(osb, &old_dir_bh, old_dir, &parent_fe_bh, dir, 0); @@ -896,7 +909,11 @@ static int ocfs2_unlink(struct inode *dir, (unsigned long long)OCFS2_I(dir)->ip_blkno, (unsigned long long)OCFS2_I(inode)->ip_blkno); - dquot_initialize(dir); + status = dquot_initialize(dir); + if (status) { + mlog_errno(status); + return status; + } BUG_ON(d_inode(dentry->d_parent) != dir); @@ -1230,8 +1247,16 @@ static int ocfs2_rename(struct inode *old_dir, old_dentry->d_name.len, old_dentry->d_name.name, new_dentry->d_name.len, new_dentry->d_name.name); - dquot_initialize(old_dir); - dquot_initialize(new_dir); + status = dquot_initialize(old_dir); + if (status) { + mlog_errno(status); + goto bail; + } + status = dquot_initialize(new_dir); + if (status) { + mlog_errno(status); + goto bail; + } osb = OCFS2_SB(old_dir->i_sb); @@ -1786,7 +1811,11 @@ static int ocfs2_symlink(struct inode *dir, trace_ocfs2_symlink_begin(dir, dentry, symname, dentry->d_name.len, dentry->d_name.name); - dquot_initialize(dir); + status = dquot_initialize(dir); + if (status) { + mlog_errno(status); + goto bail; + } sb = dir->i_sb; osb = OCFS2_SB(sb); @@ -1831,8 +1860,9 @@ static int ocfs2_symlink(struct inode *dir, } inode = ocfs2_get_init_inode(dir, S_IFLNK | S_IRWXUGO); - if (!inode) { - status = -ENOMEM; + if (IS_ERR(inode)) { + status = PTR_ERR(inode); + inode = NULL; mlog_errno(status); goto bail; } @@ -2485,8 +2515,9 @@ int ocfs2_create_inode_in_orphan(struct inode *dir, } inode = ocfs2_get_init_inode(dir, mode); - if (!inode) { - status = -ENOMEM; + if (IS_ERR(inode)) { + status = PTR_ERR(inode); + inode = NULL; mlog_errno(status); goto leave; } diff --git a/fs/ocfs2/quota_local.c b/fs/ocfs2/quota_local.c index 3d0b63d34225e8..bb07004df72a36 100644 --- a/fs/ocfs2/quota_local.c +++ b/fs/ocfs2/quota_local.c @@ -499,8 +499,8 @@ static int ocfs2_recover_local_quota_file(struct inode *lqinode, dquot = dqget(sb, make_kqid(&init_user_ns, type, le64_to_cpu(dqblk->dqb_id))); - if (!dquot) { - status = -EIO; + if (IS_ERR(dquot)) { + status = PTR_ERR(dquot); mlog(ML_ERROR, "Failed to get quota structure " "for id %u, type %d. Cannot finish quota " "file recovery.\n", diff --git a/fs/ocfs2/refcounttree.c b/fs/ocfs2/refcounttree.c index b69dd14c0b9ba1..7dc818b87cd82e 100644 --- a/fs/ocfs2/refcounttree.c +++ b/fs/ocfs2/refcounttree.c @@ -4419,8 +4419,9 @@ static int ocfs2_vfs_reflink(struct dentry *old_dentry, struct inode *dir, } mutex_lock(&inode->i_mutex); - dquot_initialize(dir); - error = ocfs2_reflink(old_dentry, dir, new_dentry, preserve); + error = dquot_initialize(dir); + if (!error) + error = ocfs2_reflink(old_dentry, dir, new_dentry, preserve); mutex_unlock(&inode->i_mutex); if (!error) fsnotify_create(dir, new_dentry); diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c index 20d1f74561cf13..fed66e2c9fe843 100644 --- a/fs/quota/dquot.c +++ b/fs/quota/dquot.c @@ -247,7 +247,7 @@ struct dqstats dqstats; EXPORT_SYMBOL(dqstats); static qsize_t inode_get_rsv_space(struct inode *inode); -static void __dquot_initialize(struct inode *inode, int type); +static int __dquot_initialize(struct inode *inode, int type); static inline unsigned int hashfn(const struct super_block *sb, struct kqid qid) @@ -832,16 +832,17 @@ static struct dquot *get_empty_dquot(struct super_block *sb, int type) struct dquot *dqget(struct super_block *sb, struct kqid qid) { unsigned int hashent = hashfn(sb, qid); - struct dquot *dquot = NULL, *empty = NULL; + struct dquot *dquot, *empty = NULL; if (!sb_has_quota_active(sb, qid.type)) - return NULL; + return ERR_PTR(-ESRCH); we_slept: spin_lock(&dq_list_lock); spin_lock(&dq_state_lock); if (!sb_has_quota_active(sb, qid.type)) { spin_unlock(&dq_state_lock); spin_unlock(&dq_list_lock); + dquot = ERR_PTR(-ESRCH); goto out; } spin_unlock(&dq_state_lock); @@ -876,11 +877,15 @@ struct dquot *dqget(struct super_block *sb, struct kqid qid) * already finished or it will be canceled due to dq_count > 1 test */ wait_on_dquot(dquot); /* Read the dquot / allocate space in quota file */ - if (!test_bit(DQ_ACTIVE_B, &dquot->dq_flags) && - sb->dq_op->acquire_dquot(dquot) < 0) { - dqput(dquot); - dquot = NULL; - goto out; + if (!test_bit(DQ_ACTIVE_B, &dquot->dq_flags)) { + int err; + + err = sb->dq_op->acquire_dquot(dquot); + if (err < 0) { + dqput(dquot); + dquot = ERR_PTR(err); + goto out; + } } #ifdef CONFIG_QUOTA_DEBUG BUG_ON(!dquot->dq_sb); /* Has somebody invalidated entry under us? */ @@ -1390,15 +1395,16 @@ static int dquot_active(const struct inode *inode) * It is better to call this function outside of any transaction as it * might need a lot of space in journal for dquot structure allocation. */ -static void __dquot_initialize(struct inode *inode, int type) +static int __dquot_initialize(struct inode *inode, int type) { int cnt, init_needed = 0; struct dquot **dquots, *got[MAXQUOTAS]; struct super_block *sb = inode->i_sb; qsize_t rsv; + int ret = 0; if (!dquot_active(inode)) - return; + return 0; dquots = i_dquot(inode); @@ -1407,6 +1413,7 @@ static void __dquot_initialize(struct inode *inode, int type) struct kqid qid; kprojid_t projid; int rc; + struct dquot *dquot; got[cnt] = NULL; if (type != -1 && cnt != type) @@ -1438,16 +1445,25 @@ static void __dquot_initialize(struct inode *inode, int type) qid = make_kqid_projid(projid); break; } - got[cnt] = dqget(sb, qid); + dquot = dqget(sb, qid); + if (IS_ERR(dquot)) { + /* We raced with somebody turning quotas off... */ + if (PTR_ERR(dquot) != -ESRCH) { + ret = PTR_ERR(dquot); + goto out_put; + } + dquot = NULL; + } + got[cnt] = dquot; } /* All required i_dquot has been initialized */ if (!init_needed) - return; + return 0; spin_lock(&dq_data_lock); if (IS_NOQUOTA(inode)) - goto out_err; + goto out_lock; for (cnt = 0; cnt < MAXQUOTAS; cnt++) { if (type != -1 && cnt != type) continue; @@ -1469,15 +1485,18 @@ static void __dquot_initialize(struct inode *inode, int type) dquot_resv_space(dquots[cnt], rsv); } } -out_err: +out_lock: spin_unlock(&dq_data_lock); +out_put: /* Drop unused references */ dqput_all(got); + + return ret; } -void dquot_initialize(struct inode *inode) +int dquot_initialize(struct inode *inode) { - __dquot_initialize(inode, -1); + return __dquot_initialize(inode, -1); } EXPORT_SYMBOL(dquot_initialize); @@ -1961,18 +1980,37 @@ EXPORT_SYMBOL(__dquot_transfer); int dquot_transfer(struct inode *inode, struct iattr *iattr) { struct dquot *transfer_to[MAXQUOTAS] = {}; + struct dquot *dquot; struct super_block *sb = inode->i_sb; int ret; if (!dquot_active(inode)) return 0; - if (iattr->ia_valid & ATTR_UID && !uid_eq(iattr->ia_uid, inode->i_uid)) - transfer_to[USRQUOTA] = dqget(sb, make_kqid_uid(iattr->ia_uid)); - if (iattr->ia_valid & ATTR_GID && !gid_eq(iattr->ia_gid, inode->i_gid)) - transfer_to[GRPQUOTA] = dqget(sb, make_kqid_gid(iattr->ia_gid)); - + if (iattr->ia_valid & ATTR_UID && !uid_eq(iattr->ia_uid, inode->i_uid)){ + dquot = dqget(sb, make_kqid_uid(iattr->ia_uid)); + if (IS_ERR(dquot)) { + if (PTR_ERR(dquot) != -ESRCH) { + ret = PTR_ERR(dquot); + goto out_put; + } + dquot = NULL; + } + transfer_to[USRQUOTA] = dquot; + } + if (iattr->ia_valid & ATTR_GID && !gid_eq(iattr->ia_gid, inode->i_gid)){ + dquot = dqget(sb, make_kqid_gid(iattr->ia_gid)); + if (IS_ERR(dquot)) { + if (PTR_ERR(dquot) != -ESRCH) { + ret = PTR_ERR(dquot); + goto out_put; + } + dquot = NULL; + } + transfer_to[GRPQUOTA] = dquot; + } ret = __dquot_transfer(inode, transfer_to); +out_put: dqput_all(transfer_to); return ret; } @@ -2518,8 +2556,8 @@ int dquot_get_dqblk(struct super_block *sb, struct kqid qid, struct dquot *dquot; dquot = dqget(sb, qid); - if (!dquot) - return -ESRCH; + if (IS_ERR(dquot)) + return PTR_ERR(dquot); do_get_dqblk(dquot, di); dqput(dquot); @@ -2631,8 +2669,8 @@ int dquot_set_dqblk(struct super_block *sb, struct kqid qid, int rc; dquot = dqget(sb, qid); - if (!dquot) { - rc = -ESRCH; + if (IS_ERR(dquot)) { + rc = PTR_ERR(dquot); goto out; } rc = do_set_dqblk(dquot, di); diff --git a/fs/quota/quota.c b/fs/quota/quota.c index 86ded7375c216d..3746367098fda3 100644 --- a/fs/quota/quota.c +++ b/fs/quota/quota.c @@ -141,9 +141,9 @@ static int quota_getinfo(struct super_block *sb, int type, void __user *addr) if (tstate->flags & QCI_ROOT_SQUASH) uinfo.dqi_flags |= DQF_ROOT_SQUASH; uinfo.dqi_valid = IIF_ALL; - if (!ret && copy_to_user(addr, &uinfo, sizeof(uinfo))) + if (copy_to_user(addr, &uinfo, sizeof(uinfo))) return -EFAULT; - return ret; + return 0; } static int quota_setinfo(struct super_block *sb, int type, void __user *addr) diff --git a/fs/reiserfs/inode.c b/fs/reiserfs/inode.c index f6f2fbad9777d3..3d8e7e671d5ba7 100644 --- a/fs/reiserfs/inode.c +++ b/fs/reiserfs/inode.c @@ -3319,8 +3319,11 @@ int reiserfs_setattr(struct dentry *dentry, struct iattr *attr) /* must be turned off for recursive notify_change calls */ ia_valid = attr->ia_valid &= ~(ATTR_KILL_SUID|ATTR_KILL_SGID); - if (is_quota_modification(inode, attr)) - dquot_initialize(inode); + if (is_quota_modification(inode, attr)) { + error = dquot_initialize(inode); + if (error) + return error; + } reiserfs_write_lock(inode->i_sb); if (attr->ia_valid & ATTR_SIZE) { /* diff --git a/fs/reiserfs/namei.c b/fs/reiserfs/namei.c index b55a074653d7a4..5f1c9c29eb8c91 100644 --- a/fs/reiserfs/namei.c +++ b/fs/reiserfs/namei.c @@ -613,8 +613,7 @@ static int new_inode_init(struct inode *inode, struct inode *dir, umode_t mode) * we have to set uid and gid here */ inode_init_owner(inode, dir, mode); - dquot_initialize(inode); - return 0; + return dquot_initialize(inode); } static int reiserfs_create(struct inode *dir, struct dentry *dentry, umode_t mode, @@ -633,12 +632,18 @@ static int reiserfs_create(struct inode *dir, struct dentry *dentry, umode_t mod struct reiserfs_transaction_handle th; struct reiserfs_security_handle security; - dquot_initialize(dir); + retval = dquot_initialize(dir); + if (retval) + return retval; if (!(inode = new_inode(dir->i_sb))) { return -ENOMEM; } - new_inode_init(inode, dir, mode); + retval = new_inode_init(inode, dir, mode); + if (retval) { + drop_new_inode(inode); + return retval; + } jbegin_count += reiserfs_cache_default_acl(dir); retval = reiserfs_security_init(dir, inode, &dentry->d_name, &security); @@ -710,12 +715,18 @@ static int reiserfs_mknod(struct inode *dir, struct dentry *dentry, umode_t mode if (!new_valid_dev(rdev)) return -EINVAL; - dquot_initialize(dir); + retval = dquot_initialize(dir); + if (retval) + return retval; if (!(inode = new_inode(dir->i_sb))) { return -ENOMEM; } - new_inode_init(inode, dir, mode); + retval = new_inode_init(inode, dir, mode); + if (retval) { + drop_new_inode(inode); + return retval; + } jbegin_count += reiserfs_cache_default_acl(dir); retval = reiserfs_security_init(dir, inode, &dentry->d_name, &security); @@ -787,7 +798,9 @@ static int reiserfs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode 2 * (REISERFS_QUOTA_INIT_BLOCKS(dir->i_sb) + REISERFS_QUOTA_TRANS_BLOCKS(dir->i_sb)); - dquot_initialize(dir); + retval = dquot_initialize(dir); + if (retval) + return retval; #ifdef DISPLACE_NEW_PACKING_LOCALITIES /* @@ -800,7 +813,11 @@ static int reiserfs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode if (!(inode = new_inode(dir->i_sb))) { return -ENOMEM; } - new_inode_init(inode, dir, mode); + retval = new_inode_init(inode, dir, mode); + if (retval) { + drop_new_inode(inode); + return retval; + } jbegin_count += reiserfs_cache_default_acl(dir); retval = reiserfs_security_init(dir, inode, &dentry->d_name, &security); @@ -899,7 +916,9 @@ static int reiserfs_rmdir(struct inode *dir, struct dentry *dentry) JOURNAL_PER_BALANCE_CNT * 2 + 2 + 4 * REISERFS_QUOTA_TRANS_BLOCKS(dir->i_sb); - dquot_initialize(dir); + retval = dquot_initialize(dir); + if (retval) + return retval; reiserfs_write_lock(dir->i_sb); retval = journal_begin(&th, dir->i_sb, jbegin_count); @@ -985,7 +1004,9 @@ static int reiserfs_unlink(struct inode *dir, struct dentry *dentry) int jbegin_count; unsigned long savelink; - dquot_initialize(dir); + retval = dquot_initialize(dir); + if (retval) + return retval; inode = d_inode(dentry); @@ -1095,12 +1116,18 @@ static int reiserfs_symlink(struct inode *parent_dir, 2 * (REISERFS_QUOTA_INIT_BLOCKS(parent_dir->i_sb) + REISERFS_QUOTA_TRANS_BLOCKS(parent_dir->i_sb)); - dquot_initialize(parent_dir); + retval = dquot_initialize(parent_dir); + if (retval) + return retval; if (!(inode = new_inode(parent_dir->i_sb))) { return -ENOMEM; } - new_inode_init(inode, parent_dir, mode); + retval = new_inode_init(inode, parent_dir, mode); + if (retval) { + drop_new_inode(inode); + return retval; + } retval = reiserfs_security_init(parent_dir, inode, &dentry->d_name, &security); @@ -1184,7 +1211,9 @@ static int reiserfs_link(struct dentry *old_dentry, struct inode *dir, JOURNAL_PER_BALANCE_CNT * 3 + 2 * REISERFS_QUOTA_TRANS_BLOCKS(dir->i_sb); - dquot_initialize(dir); + retval = dquot_initialize(dir); + if (retval) + return retval; reiserfs_write_lock(dir->i_sb); if (inode->i_nlink >= REISERFS_LINK_MAX) { @@ -1308,8 +1337,12 @@ static int reiserfs_rename(struct inode *old_dir, struct dentry *old_dentry, JOURNAL_PER_BALANCE_CNT * 3 + 5 + 4 * REISERFS_QUOTA_TRANS_BLOCKS(old_dir->i_sb); - dquot_initialize(old_dir); - dquot_initialize(new_dir); + retval = dquot_initialize(old_dir); + if (retval) + return retval; + retval = dquot_initialize(new_dir); + if (retval) + return retval; old_inode = d_inode(old_dentry); new_dentry_inode = d_inode(new_dentry); diff --git a/fs/udf/super.c b/fs/udf/super.c index b96f190bc5679b..81155b9b445b33 100644 --- a/fs/udf/super.c +++ b/fs/udf/super.c @@ -2070,6 +2070,7 @@ static int udf_fill_super(struct super_block *sb, void *options, int silent) struct udf_options uopt; struct kernel_lb_addr rootdir, fileset; struct udf_sb_info *sbi; + bool lvid_open = false; uopt.flags = (1 << UDF_FLAG_USE_AD_IN_ICB) | (1 << UDF_FLAG_STRICT); uopt.uid = INVALID_UID; @@ -2216,8 +2217,10 @@ static int udf_fill_super(struct super_block *sb, void *options, int silent) le16_to_cpu(ts.year), ts.month, ts.day, ts.hour, ts.minute, le16_to_cpu(ts.typeAndTimezone)); } - if (!(sb->s_flags & MS_RDONLY)) + if (!(sb->s_flags & MS_RDONLY)) { udf_open_lvid(sb); + lvid_open = true; + } /* Assign the root inode */ /* assign inodes by physical block number */ @@ -2248,7 +2251,7 @@ static int udf_fill_super(struct super_block *sb, void *options, int silent) if (UDF_QUERY_FLAG(sb, UDF_FLAG_NLS_MAP)) unload_nls(sbi->s_nls_map); #endif - if (!(sb->s_flags & MS_RDONLY)) + if (lvid_open) udf_close_lvid(sb); brelse(sbi->s_lvid_bh); udf_sb_free_partitions(sb); diff --git a/include/asm-generic/atomic-long.h b/include/asm-generic/atomic-long.h index b7babf0206b870..a94cbebbc33df5 100644 --- a/include/asm-generic/atomic-long.h +++ b/include/asm-generic/atomic-long.h @@ -23,236 +23,159 @@ typedef atomic64_t atomic_long_t; #define ATOMIC_LONG_INIT(i) ATOMIC64_INIT(i) +#define ATOMIC_LONG_PFX(x) atomic64 ## x -static inline long atomic_long_read(atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - return (long)atomic64_read(v); -} - -static inline void atomic_long_set(atomic_long_t *l, long i) -{ - atomic64_t *v = (atomic64_t *)l; - - atomic64_set(v, i); -} - -static inline void atomic_long_inc(atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - atomic64_inc(v); -} - -static inline void atomic_long_dec(atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - atomic64_dec(v); -} - -static inline void atomic_long_add(long i, atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - atomic64_add(i, v); -} - -static inline void atomic_long_sub(long i, atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - atomic64_sub(i, v); -} - -static inline int atomic_long_sub_and_test(long i, atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - return atomic64_sub_and_test(i, v); -} - -static inline int atomic_long_dec_and_test(atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - return atomic64_dec_and_test(v); -} - -static inline int atomic_long_inc_and_test(atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - return atomic64_inc_and_test(v); -} - -static inline int atomic_long_add_negative(long i, atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - return atomic64_add_negative(i, v); -} - -static inline long atomic_long_add_return(long i, atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - return (long)atomic64_add_return(i, v); -} - -static inline long atomic_long_sub_return(long i, atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - return (long)atomic64_sub_return(i, v); -} - -static inline long atomic_long_inc_return(atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - return (long)atomic64_inc_return(v); -} - -static inline long atomic_long_dec_return(atomic_long_t *l) -{ - atomic64_t *v = (atomic64_t *)l; - - return (long)atomic64_dec_return(v); -} - -static inline long atomic_long_add_unless(atomic_long_t *l, long a, long u) -{ - atomic64_t *v = (atomic64_t *)l; - - return (long)atomic64_add_unless(v, a, u); -} - -#define atomic_long_inc_not_zero(l) atomic64_inc_not_zero((atomic64_t *)(l)) - -#define atomic_long_cmpxchg(l, old, new) \ - (atomic64_cmpxchg((atomic64_t *)(l), (old), (new))) -#define atomic_long_xchg(v, new) \ - (atomic64_xchg((atomic64_t *)(v), (new))) - -#else /* BITS_PER_LONG == 64 */ +#else typedef atomic_t atomic_long_t; #define ATOMIC_LONG_INIT(i) ATOMIC_INIT(i) -static inline long atomic_long_read(atomic_long_t *l) -{ - atomic_t *v = (atomic_t *)l; - - return (long)atomic_read(v); -} - -static inline void atomic_long_set(atomic_long_t *l, long i) -{ - atomic_t *v = (atomic_t *)l; - - atomic_set(v, i); -} +#define ATOMIC_LONG_PFX(x) atomic ## x + +#endif + +#define ATOMIC_LONG_READ_OP(mo) \ +static inline long atomic_long_read##mo(atomic_long_t *l) \ +{ \ + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; \ + \ + return (long)ATOMIC_LONG_PFX(_read##mo)(v); \ +} +ATOMIC_LONG_READ_OP() +ATOMIC_LONG_READ_OP(_acquire) + +#undef ATOMIC_LONG_READ_OP + +#define ATOMIC_LONG_SET_OP(mo) \ +static inline void atomic_long_set##mo(atomic_long_t *l, long i) \ +{ \ + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; \ + \ + ATOMIC_LONG_PFX(_set##mo)(v, i); \ +} +ATOMIC_LONG_SET_OP() +ATOMIC_LONG_SET_OP(_release) + +#undef ATOMIC_LONG_SET_OP + +#define ATOMIC_LONG_ADD_SUB_OP(op, mo) \ +static inline long \ +atomic_long_##op##_return##mo(long i, atomic_long_t *l) \ +{ \ + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; \ + \ + return (long)ATOMIC_LONG_PFX(_##op##_return##mo)(i, v); \ +} +ATOMIC_LONG_ADD_SUB_OP(add,) +ATOMIC_LONG_ADD_SUB_OP(add, _relaxed) +ATOMIC_LONG_ADD_SUB_OP(add, _acquire) +ATOMIC_LONG_ADD_SUB_OP(add, _release) +ATOMIC_LONG_ADD_SUB_OP(sub,) +ATOMIC_LONG_ADD_SUB_OP(sub, _relaxed) +ATOMIC_LONG_ADD_SUB_OP(sub, _acquire) +ATOMIC_LONG_ADD_SUB_OP(sub, _release) + +#undef ATOMIC_LONG_ADD_SUB_OP + +#define atomic_long_cmpxchg_relaxed(l, old, new) \ + (ATOMIC_LONG_PFX(_cmpxchg_relaxed)((ATOMIC_LONG_PFX(_t) *)(l), \ + (old), (new))) +#define atomic_long_cmpxchg_acquire(l, old, new) \ + (ATOMIC_LONG_PFX(_cmpxchg_acquire)((ATOMIC_LONG_PFX(_t) *)(l), \ + (old), (new))) +#define atomic_long_cmpxchg_release(l, old, new) \ + (ATOMIC_LONG_PFX(_cmpxchg_release)((ATOMIC_LONG_PFX(_t) *)(l), \ + (old), (new))) +#define atomic_long_cmpxchg(l, old, new) \ + (ATOMIC_LONG_PFX(_cmpxchg)((ATOMIC_LONG_PFX(_t) *)(l), (old), (new))) + +#define atomic_long_xchg_relaxed(v, new) \ + (ATOMIC_LONG_PFX(_xchg_relaxed)((ATOMIC_LONG_PFX(_t) *)(v), (new))) +#define atomic_long_xchg_acquire(v, new) \ + (ATOMIC_LONG_PFX(_xchg_acquire)((ATOMIC_LONG_PFX(_t) *)(v), (new))) +#define atomic_long_xchg_release(v, new) \ + (ATOMIC_LONG_PFX(_xchg_release)((ATOMIC_LONG_PFX(_t) *)(v), (new))) +#define atomic_long_xchg(v, new) \ + (ATOMIC_LONG_PFX(_xchg)((ATOMIC_LONG_PFX(_t) *)(v), (new))) static inline void atomic_long_inc(atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - atomic_inc(v); + ATOMIC_LONG_PFX(_inc)(v); } static inline void atomic_long_dec(atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - atomic_dec(v); + ATOMIC_LONG_PFX(_dec)(v); } static inline void atomic_long_add(long i, atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - atomic_add(i, v); + ATOMIC_LONG_PFX(_add)(i, v); } static inline void atomic_long_sub(long i, atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - atomic_sub(i, v); + ATOMIC_LONG_PFX(_sub)(i, v); } static inline int atomic_long_sub_and_test(long i, atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - return atomic_sub_and_test(i, v); + return ATOMIC_LONG_PFX(_sub_and_test)(i, v); } static inline int atomic_long_dec_and_test(atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - return atomic_dec_and_test(v); + return ATOMIC_LONG_PFX(_dec_and_test)(v); } static inline int atomic_long_inc_and_test(atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - return atomic_inc_and_test(v); + return ATOMIC_LONG_PFX(_inc_and_test)(v); } static inline int atomic_long_add_negative(long i, atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - return atomic_add_negative(i, v); -} - -static inline long atomic_long_add_return(long i, atomic_long_t *l) -{ - atomic_t *v = (atomic_t *)l; - - return (long)atomic_add_return(i, v); -} - -static inline long atomic_long_sub_return(long i, atomic_long_t *l) -{ - atomic_t *v = (atomic_t *)l; - - return (long)atomic_sub_return(i, v); + return ATOMIC_LONG_PFX(_add_negative)(i, v); } static inline long atomic_long_inc_return(atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - return (long)atomic_inc_return(v); + return (long)ATOMIC_LONG_PFX(_inc_return)(v); } static inline long atomic_long_dec_return(atomic_long_t *l) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - return (long)atomic_dec_return(v); + return (long)ATOMIC_LONG_PFX(_dec_return)(v); } static inline long atomic_long_add_unless(atomic_long_t *l, long a, long u) { - atomic_t *v = (atomic_t *)l; + ATOMIC_LONG_PFX(_t) *v = (ATOMIC_LONG_PFX(_t) *)l; - return (long)atomic_add_unless(v, a, u); + return (long)ATOMIC_LONG_PFX(_add_unless)(v, a, u); } -#define atomic_long_inc_not_zero(l) atomic_inc_not_zero((atomic_t *)(l)) - -#define atomic_long_cmpxchg(l, old, new) \ - (atomic_cmpxchg((atomic_t *)(l), (old), (new))) -#define atomic_long_xchg(v, new) \ - (atomic_xchg((atomic_t *)(v), (new))) - -#endif /* BITS_PER_LONG == 64 */ +#define atomic_long_inc_not_zero(l) \ + ATOMIC_LONG_PFX(_inc_not_zero)((ATOMIC_LONG_PFX(_t) *)(l)) #endif /* _ASM_GENERIC_ATOMIC_LONG_H */ diff --git a/include/asm-generic/atomic.h b/include/asm-generic/atomic.h index 1973ad2b13f4dd..d4d7e337fdcb5d 100644 --- a/include/asm-generic/atomic.h +++ b/include/asm-generic/atomic.h @@ -98,15 +98,16 @@ ATOMIC_OP_RETURN(add, +) ATOMIC_OP_RETURN(sub, -) #endif -#ifndef atomic_clear_mask +#ifndef atomic_and ATOMIC_OP(and, &) -#define atomic_clear_mask(i, v) atomic_and(~(i), (v)) #endif -#ifndef atomic_set_mask -#define CONFIG_ARCH_HAS_ATOMIC_OR +#ifndef atomic_or ATOMIC_OP(or, |) -#define atomic_set_mask(i, v) atomic_or((i), (v)) +#endif + +#ifndef atomic_xor +ATOMIC_OP(xor, ^) #endif #undef ATOMIC_OP_RETURN diff --git a/include/asm-generic/atomic64.h b/include/asm-generic/atomic64.h index 30ad9c86cebb62..d48e78ccad3dd8 100644 --- a/include/asm-generic/atomic64.h +++ b/include/asm-generic/atomic64.h @@ -32,6 +32,10 @@ extern long long atomic64_##op##_return(long long a, atomic64_t *v); ATOMIC64_OPS(add) ATOMIC64_OPS(sub) +ATOMIC64_OP(and) +ATOMIC64_OP(or) +ATOMIC64_OP(xor) + #undef ATOMIC64_OPS #undef ATOMIC64_OP_RETURN #undef ATOMIC64_OP diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h index 55e3abc2d02727..b42afada1280e0 100644 --- a/include/asm-generic/barrier.h +++ b/include/asm-generic/barrier.h @@ -108,12 +108,12 @@ do { \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ - ACCESS_ONCE(*p) = (v); \ + WRITE_ONCE(*p, v); \ } while (0) #define smp_load_acquire(p) \ ({ \ - typeof(*p) ___p1 = ACCESS_ONCE(*p); \ + typeof(*p) ___p1 = READ_ONCE(*p); \ compiletime_assert_atomic_type(*p); \ smp_mb(); \ ___p1; \ diff --git a/include/asm-generic/qrwlock.h b/include/asm-generic/qrwlock.h index 6383d54bf983a8..54a8e65e18b622 100644 --- a/include/asm-generic/qrwlock.h +++ b/include/asm-generic/qrwlock.h @@ -36,39 +36,39 @@ /* * External function declarations */ -extern void queue_read_lock_slowpath(struct qrwlock *lock); -extern void queue_write_lock_slowpath(struct qrwlock *lock); +extern void queued_read_lock_slowpath(struct qrwlock *lock, u32 cnts); +extern void queued_write_lock_slowpath(struct qrwlock *lock); /** - * queue_read_can_lock- would read_trylock() succeed? + * queued_read_can_lock- would read_trylock() succeed? * @lock: Pointer to queue rwlock structure */ -static inline int queue_read_can_lock(struct qrwlock *lock) +static inline int queued_read_can_lock(struct qrwlock *lock) { return !(atomic_read(&lock->cnts) & _QW_WMASK); } /** - * queue_write_can_lock- would write_trylock() succeed? + * queued_write_can_lock- would write_trylock() succeed? * @lock: Pointer to queue rwlock structure */ -static inline int queue_write_can_lock(struct qrwlock *lock) +static inline int queued_write_can_lock(struct qrwlock *lock) { return !atomic_read(&lock->cnts); } /** - * queue_read_trylock - try to acquire read lock of a queue rwlock + * queued_read_trylock - try to acquire read lock of a queue rwlock * @lock : Pointer to queue rwlock structure * Return: 1 if lock acquired, 0 if failed */ -static inline int queue_read_trylock(struct qrwlock *lock) +static inline int queued_read_trylock(struct qrwlock *lock) { u32 cnts; cnts = atomic_read(&lock->cnts); if (likely(!(cnts & _QW_WMASK))) { - cnts = (u32)atomic_add_return(_QR_BIAS, &lock->cnts); + cnts = (u32)atomic_add_return_acquire(_QR_BIAS, &lock->cnts); if (likely(!(cnts & _QW_WMASK))) return 1; atomic_sub(_QR_BIAS, &lock->cnts); @@ -77,11 +77,11 @@ static inline int queue_read_trylock(struct qrwlock *lock) } /** - * queue_write_trylock - try to acquire write lock of a queue rwlock + * queued_write_trylock - try to acquire write lock of a queue rwlock * @lock : Pointer to queue rwlock structure * Return: 1 if lock acquired, 0 if failed */ -static inline int queue_write_trylock(struct qrwlock *lock) +static inline int queued_write_trylock(struct qrwlock *lock) { u32 cnts; @@ -89,78 +89,70 @@ static inline int queue_write_trylock(struct qrwlock *lock) if (unlikely(cnts)) return 0; - return likely(atomic_cmpxchg(&lock->cnts, - cnts, cnts | _QW_LOCKED) == cnts); + return likely(atomic_cmpxchg_acquire(&lock->cnts, + cnts, cnts | _QW_LOCKED) == cnts); } /** - * queue_read_lock - acquire read lock of a queue rwlock + * queued_read_lock - acquire read lock of a queue rwlock * @lock: Pointer to queue rwlock structure */ -static inline void queue_read_lock(struct qrwlock *lock) +static inline void queued_read_lock(struct qrwlock *lock) { u32 cnts; - cnts = atomic_add_return(_QR_BIAS, &lock->cnts); + cnts = atomic_add_return_acquire(_QR_BIAS, &lock->cnts); if (likely(!(cnts & _QW_WMASK))) return; /* The slowpath will decrement the reader count, if necessary. */ - queue_read_lock_slowpath(lock); + queued_read_lock_slowpath(lock, cnts); } /** - * queue_write_lock - acquire write lock of a queue rwlock + * queued_write_lock - acquire write lock of a queue rwlock * @lock : Pointer to queue rwlock structure */ -static inline void queue_write_lock(struct qrwlock *lock) +static inline void queued_write_lock(struct qrwlock *lock) { /* Optimize for the unfair lock case where the fair flag is 0. */ - if (atomic_cmpxchg(&lock->cnts, 0, _QW_LOCKED) == 0) + if (atomic_cmpxchg_acquire(&lock->cnts, 0, _QW_LOCKED) == 0) return; - queue_write_lock_slowpath(lock); + queued_write_lock_slowpath(lock); } /** - * queue_read_unlock - release read lock of a queue rwlock + * queued_read_unlock - release read lock of a queue rwlock * @lock : Pointer to queue rwlock structure */ -static inline void queue_read_unlock(struct qrwlock *lock) +static inline void queued_read_unlock(struct qrwlock *lock) { /* * Atomically decrement the reader count */ - smp_mb__before_atomic(); - atomic_sub(_QR_BIAS, &lock->cnts); + (void)atomic_sub_return_release(_QR_BIAS, &lock->cnts); } -#ifndef queue_write_unlock /** - * queue_write_unlock - release write lock of a queue rwlock + * queued_write_unlock - release write lock of a queue rwlock * @lock : Pointer to queue rwlock structure */ -static inline void queue_write_unlock(struct qrwlock *lock) +static inline void queued_write_unlock(struct qrwlock *lock) { - /* - * If the writer field is atomic, it can be cleared directly. - * Otherwise, an atomic subtraction will be used to clear it. - */ - smp_mb__before_atomic(); - atomic_sub(_QW_LOCKED, &lock->cnts); + smp_store_release((u8 *)&lock->cnts, 0); } -#endif /* * Remapping rwlock architecture specific functions to the corresponding * queue rwlock functions. */ -#define arch_read_can_lock(l) queue_read_can_lock(l) -#define arch_write_can_lock(l) queue_write_can_lock(l) -#define arch_read_lock(l) queue_read_lock(l) -#define arch_write_lock(l) queue_write_lock(l) -#define arch_read_trylock(l) queue_read_trylock(l) -#define arch_write_trylock(l) queue_write_trylock(l) -#define arch_read_unlock(l) queue_read_unlock(l) -#define arch_write_unlock(l) queue_write_unlock(l) +#define arch_read_can_lock(l) queued_read_can_lock(l) +#define arch_write_can_lock(l) queued_write_can_lock(l) +#define arch_read_lock(l) queued_read_lock(l) +#define arch_write_lock(l) queued_write_lock(l) +#define arch_read_trylock(l) queued_read_trylock(l) +#define arch_write_trylock(l) queued_write_trylock(l) +#define arch_read_unlock(l) queued_read_unlock(l) +#define arch_write_unlock(l) queued_write_unlock(l) #endif /* __ASM_GENERIC_QRWLOCK_H */ diff --git a/include/dt-bindings/pinctrl/qcom,pmic-mpp.h b/include/dt-bindings/pinctrl/qcom,pmic-mpp.h index c10205491f8d65..a15c1704d0ec61 100644 --- a/include/dt-bindings/pinctrl/qcom,pmic-mpp.h +++ b/include/dt-bindings/pinctrl/qcom,pmic-mpp.h @@ -7,6 +7,47 @@ #define _DT_BINDINGS_PINCTRL_QCOM_PMIC_MPP_H /* power-source */ + +/* Digital Input/Output: level [PM8058] */ +#define PM8058_MPP_VPH 0 +#define PM8058_MPP_S3 1 +#define PM8058_MPP_L2 2 +#define PM8058_MPP_L3 3 + +/* Digital Input/Output: level [PM8901] */ +#define PM8901_MPP_MSMIO 0 +#define PM8901_MPP_DIG 1 +#define PM8901_MPP_L5 2 +#define PM8901_MPP_S4 3 +#define PM8901_MPP_VPH 4 + +/* Digital Input/Output: level [PM8921] */ +#define PM8921_MPP_S4 1 +#define PM8921_MPP_L15 3 +#define PM8921_MPP_L17 4 +#define PM8921_MPP_VPH 7 + +/* Digital Input/Output: level [PM8821] */ +#define PM8821_MPP_1P8 0 +#define PM8821_MPP_VPH 7 + +/* Digital Input/Output: level [PM8018] */ +#define PM8018_MPP_L4 0 +#define PM8018_MPP_L14 1 +#define PM8018_MPP_S3 2 +#define PM8018_MPP_L6 3 +#define PM8018_MPP_L2 4 +#define PM8018_MPP_L5 5 +#define PM8018_MPP_VPH 7 + +/* Digital Input/Output: level [PM8038] */ +#define PM8038_MPP_L20 0 +#define PM8038_MPP_L11 1 +#define PM8038_MPP_L5 2 +#define PM8038_MPP_L15 3 +#define PM8038_MPP_L17 4 +#define PM8038_MPP_VPH 7 + #define PM8841_MPP_VPH 0 #define PM8841_MPP_S3 2 @@ -37,6 +78,16 @@ #define PMIC_MPP_AMUX_ROUTE_ABUS3 6 #define PMIC_MPP_AMUX_ROUTE_ABUS4 7 +/* Analog Output: level */ +#define PMIC_MPP_AOUT_LVL_1V25 0 +#define PMIC_MPP_AOUT_LVL_1V25_2 1 +#define PMIC_MPP_AOUT_LVL_0V625 2 +#define PMIC_MPP_AOUT_LVL_0V3125 3 +#define PMIC_MPP_AOUT_LVL_MPP 4 +#define PMIC_MPP_AOUT_LVL_ABUS1 5 +#define PMIC_MPP_AOUT_LVL_ABUS2 6 +#define PMIC_MPP_AOUT_LVL_ABUS3 7 + /* To be used with "function" */ #define PMIC_MPP_FUNC_NORMAL "normal" #define PMIC_MPP_FUNC_PAIRED "paired" diff --git a/include/linux/atomic.h b/include/linux/atomic.h index 5b08a8540ecfc3..00a5763e850e94 100644 --- a/include/linux/atomic.h +++ b/include/linux/atomic.h @@ -2,6 +2,329 @@ #ifndef _LINUX_ATOMIC_H #define _LINUX_ATOMIC_H #include +#include + +/* + * Relaxed variants of xchg, cmpxchg and some atomic operations. + * + * We support four variants: + * + * - Fully ordered: The default implementation, no suffix required. + * - Acquire: Provides ACQUIRE semantics, _acquire suffix. + * - Release: Provides RELEASE semantics, _release suffix. + * - Relaxed: No ordering guarantees, _relaxed suffix. + * + * For compound atomics performing both a load and a store, ACQUIRE + * semantics apply only to the load and RELEASE semantics only to the + * store portion of the operation. Note that a failed cmpxchg_acquire + * does -not- imply any memory ordering constraints. + * + * See Documentation/memory-barriers.txt for ACQUIRE/RELEASE definitions. + */ + +#ifndef atomic_read_acquire +#define atomic_read_acquire(v) smp_load_acquire(&(v)->counter) +#endif + +#ifndef atomic_set_release +#define atomic_set_release(v, i) smp_store_release(&(v)->counter, (i)) +#endif + +/* + * The idea here is to build acquire/release variants by adding explicit + * barriers on top of the relaxed variant. In the case where the relaxed + * variant is already fully ordered, no additional barriers are needed. + */ +#define __atomic_op_acquire(op, args...) \ +({ \ + typeof(op##_relaxed(args)) __ret = op##_relaxed(args); \ + smp_mb__after_atomic(); \ + __ret; \ +}) + +#define __atomic_op_release(op, args...) \ +({ \ + smp_mb__before_atomic(); \ + op##_relaxed(args); \ +}) + +#define __atomic_op_fence(op, args...) \ +({ \ + typeof(op##_relaxed(args)) __ret; \ + smp_mb__before_atomic(); \ + __ret = op##_relaxed(args); \ + smp_mb__after_atomic(); \ + __ret; \ +}) + +/* atomic_add_return_relaxed */ +#ifndef atomic_add_return_relaxed +#define atomic_add_return_relaxed atomic_add_return +#define atomic_add_return_acquire atomic_add_return +#define atomic_add_return_release atomic_add_return + +#else /* atomic_add_return_relaxed */ + +#ifndef atomic_add_return_acquire +#define atomic_add_return_acquire(...) \ + __atomic_op_acquire(atomic_add_return, __VA_ARGS__) +#endif + +#ifndef atomic_add_return_release +#define atomic_add_return_release(...) \ + __atomic_op_release(atomic_add_return, __VA_ARGS__) +#endif + +#ifndef atomic_add_return +#define atomic_add_return(...) \ + __atomic_op_fence(atomic_add_return, __VA_ARGS__) +#endif +#endif /* atomic_add_return_relaxed */ + +/* atomic_sub_return_relaxed */ +#ifndef atomic_sub_return_relaxed +#define atomic_sub_return_relaxed atomic_sub_return +#define atomic_sub_return_acquire atomic_sub_return +#define atomic_sub_return_release atomic_sub_return + +#else /* atomic_sub_return_relaxed */ + +#ifndef atomic_sub_return_acquire +#define atomic_sub_return_acquire(...) \ + __atomic_op_acquire(atomic_sub_return, __VA_ARGS__) +#endif + +#ifndef atomic_sub_return_release +#define atomic_sub_return_release(...) \ + __atomic_op_release(atomic_sub_return, __VA_ARGS__) +#endif + +#ifndef atomic_sub_return +#define atomic_sub_return(...) \ + __atomic_op_fence(atomic_sub_return, __VA_ARGS__) +#endif +#endif /* atomic_sub_return_relaxed */ + +/* atomic_xchg_relaxed */ +#ifndef atomic_xchg_relaxed +#define atomic_xchg_relaxed atomic_xchg +#define atomic_xchg_acquire atomic_xchg +#define atomic_xchg_release atomic_xchg + +#else /* atomic_xchg_relaxed */ + +#ifndef atomic_xchg_acquire +#define atomic_xchg_acquire(...) \ + __atomic_op_acquire(atomic_xchg, __VA_ARGS__) +#endif + +#ifndef atomic_xchg_release +#define atomic_xchg_release(...) \ + __atomic_op_release(atomic_xchg, __VA_ARGS__) +#endif + +#ifndef atomic_xchg +#define atomic_xchg(...) \ + __atomic_op_fence(atomic_xchg, __VA_ARGS__) +#endif +#endif /* atomic_xchg_relaxed */ + +/* atomic_cmpxchg_relaxed */ +#ifndef atomic_cmpxchg_relaxed +#define atomic_cmpxchg_relaxed atomic_cmpxchg +#define atomic_cmpxchg_acquire atomic_cmpxchg +#define atomic_cmpxchg_release atomic_cmpxchg + +#else /* atomic_cmpxchg_relaxed */ + +#ifndef atomic_cmpxchg_acquire +#define atomic_cmpxchg_acquire(...) \ + __atomic_op_acquire(atomic_cmpxchg, __VA_ARGS__) +#endif + +#ifndef atomic_cmpxchg_release +#define atomic_cmpxchg_release(...) \ + __atomic_op_release(atomic_cmpxchg, __VA_ARGS__) +#endif + +#ifndef atomic_cmpxchg +#define atomic_cmpxchg(...) \ + __atomic_op_fence(atomic_cmpxchg, __VA_ARGS__) +#endif +#endif /* atomic_cmpxchg_relaxed */ + +#ifndef atomic64_read_acquire +#define atomic64_read_acquire(v) smp_load_acquire(&(v)->counter) +#endif + +#ifndef atomic64_set_release +#define atomic64_set_release(v, i) smp_store_release(&(v)->counter, (i)) +#endif + +/* atomic64_add_return_relaxed */ +#ifndef atomic64_add_return_relaxed +#define atomic64_add_return_relaxed atomic64_add_return +#define atomic64_add_return_acquire atomic64_add_return +#define atomic64_add_return_release atomic64_add_return + +#else /* atomic64_add_return_relaxed */ + +#ifndef atomic64_add_return_acquire +#define atomic64_add_return_acquire(...) \ + __atomic_op_acquire(atomic64_add_return, __VA_ARGS__) +#endif + +#ifndef atomic64_add_return_release +#define atomic64_add_return_release(...) \ + __atomic_op_release(atomic64_add_return, __VA_ARGS__) +#endif + +#ifndef atomic64_add_return +#define atomic64_add_return(...) \ + __atomic_op_fence(atomic64_add_return, __VA_ARGS__) +#endif +#endif /* atomic64_add_return_relaxed */ + +/* atomic64_sub_return_relaxed */ +#ifndef atomic64_sub_return_relaxed +#define atomic64_sub_return_relaxed atomic64_sub_return +#define atomic64_sub_return_acquire atomic64_sub_return +#define atomic64_sub_return_release atomic64_sub_return + +#else /* atomic64_sub_return_relaxed */ + +#ifndef atomic64_sub_return_acquire +#define atomic64_sub_return_acquire(...) \ + __atomic_op_acquire(atomic64_sub_return, __VA_ARGS__) +#endif + +#ifndef atomic64_sub_return_release +#define atomic64_sub_return_release(...) \ + __atomic_op_release(atomic64_sub_return, __VA_ARGS__) +#endif + +#ifndef atomic64_sub_return +#define atomic64_sub_return(...) \ + __atomic_op_fence(atomic64_sub_return, __VA_ARGS__) +#endif +#endif /* atomic64_sub_return_relaxed */ + +/* atomic64_xchg_relaxed */ +#ifndef atomic64_xchg_relaxed +#define atomic64_xchg_relaxed atomic64_xchg +#define atomic64_xchg_acquire atomic64_xchg +#define atomic64_xchg_release atomic64_xchg + +#else /* atomic64_xchg_relaxed */ + +#ifndef atomic64_xchg_acquire +#define atomic64_xchg_acquire(...) \ + __atomic_op_acquire(atomic64_xchg, __VA_ARGS__) +#endif + +#ifndef atomic64_xchg_release +#define atomic64_xchg_release(...) \ + __atomic_op_release(atomic64_xchg, __VA_ARGS__) +#endif + +#ifndef atomic64_xchg +#define atomic64_xchg(...) \ + __atomic_op_fence(atomic64_xchg, __VA_ARGS__) +#endif +#endif /* atomic64_xchg_relaxed */ + +/* atomic64_cmpxchg_relaxed */ +#ifndef atomic64_cmpxchg_relaxed +#define atomic64_cmpxchg_relaxed atomic64_cmpxchg +#define atomic64_cmpxchg_acquire atomic64_cmpxchg +#define atomic64_cmpxchg_release atomic64_cmpxchg + +#else /* atomic64_cmpxchg_relaxed */ + +#ifndef atomic64_cmpxchg_acquire +#define atomic64_cmpxchg_acquire(...) \ + __atomic_op_acquire(atomic64_cmpxchg, __VA_ARGS__) +#endif + +#ifndef atomic64_cmpxchg_release +#define atomic64_cmpxchg_release(...) \ + __atomic_op_release(atomic64_cmpxchg, __VA_ARGS__) +#endif + +#ifndef atomic64_cmpxchg +#define atomic64_cmpxchg(...) \ + __atomic_op_fence(atomic64_cmpxchg, __VA_ARGS__) +#endif +#endif /* atomic64_cmpxchg_relaxed */ + +/* cmpxchg_relaxed */ +#ifndef cmpxchg_relaxed +#define cmpxchg_relaxed cmpxchg +#define cmpxchg_acquire cmpxchg +#define cmpxchg_release cmpxchg + +#else /* cmpxchg_relaxed */ + +#ifndef cmpxchg_acquire +#define cmpxchg_acquire(...) \ + __atomic_op_acquire(cmpxchg, __VA_ARGS__) +#endif + +#ifndef cmpxchg_release +#define cmpxchg_release(...) \ + __atomic_op_release(cmpxchg, __VA_ARGS__) +#endif + +#ifndef cmpxchg +#define cmpxchg(...) \ + __atomic_op_fence(cmpxchg, __VA_ARGS__) +#endif +#endif /* cmpxchg_relaxed */ + +/* cmpxchg64_relaxed */ +#ifndef cmpxchg64_relaxed +#define cmpxchg64_relaxed cmpxchg64 +#define cmpxchg64_acquire cmpxchg64 +#define cmpxchg64_release cmpxchg64 + +#else /* cmpxchg64_relaxed */ + +#ifndef cmpxchg64_acquire +#define cmpxchg64_acquire(...) \ + __atomic_op_acquire(cmpxchg64, __VA_ARGS__) +#endif + +#ifndef cmpxchg64_release +#define cmpxchg64_release(...) \ + __atomic_op_release(cmpxchg64, __VA_ARGS__) +#endif + +#ifndef cmpxchg64 +#define cmpxchg64(...) \ + __atomic_op_fence(cmpxchg64, __VA_ARGS__) +#endif +#endif /* cmpxchg64_relaxed */ + +/* xchg_relaxed */ +#ifndef xchg_relaxed +#define xchg_relaxed xchg +#define xchg_acquire xchg +#define xchg_release xchg + +#else /* xchg_relaxed */ + +#ifndef xchg_acquire +#define xchg_acquire(...) __atomic_op_acquire(xchg, __VA_ARGS__) +#endif + +#ifndef xchg_release +#define xchg_release(...) __atomic_op_release(xchg, __VA_ARGS__) +#endif + +#ifndef xchg +#define xchg(...) __atomic_op_fence(xchg, __VA_ARGS__) +#endif +#endif /* xchg_relaxed */ /** * atomic_add_unless - add unless the number is already a given value @@ -28,6 +351,23 @@ static inline int atomic_add_unless(atomic_t *v, int a, int u) #define atomic_inc_not_zero(v) atomic_add_unless((v), 1, 0) #endif +#ifndef atomic_andnot +static inline void atomic_andnot(int i, atomic_t *v) +{ + atomic_and(~i, v); +} +#endif + +static inline __deprecated void atomic_clear_mask(unsigned int mask, atomic_t *v) +{ + atomic_andnot(mask, v); +} + +static inline __deprecated void atomic_set_mask(unsigned int mask, atomic_t *v) +{ + atomic_or(mask, v); +} + /** * atomic_inc_not_zero_hint - increment if not null * @v: pointer of type atomic_t @@ -111,21 +451,16 @@ static inline int atomic_dec_if_positive(atomic_t *v) } #endif -#ifndef CONFIG_ARCH_HAS_ATOMIC_OR -static inline void atomic_or(int i, atomic_t *v) -{ - int old; - int new; - - do { - old = atomic_read(v); - new = old | i; - } while (atomic_cmpxchg(v, old, new) != old); -} -#endif /* #ifndef CONFIG_ARCH_HAS_ATOMIC_OR */ - #include #ifdef CONFIG_GENERIC_ATOMIC64 #include #endif + +#ifndef atomic64_andnot +static inline void atomic64_andnot(long long i, atomic64_t *v) +{ + atomic64_and(~i, v); +} +#endif + #endif /* _LINUX_ATOMIC_H */ diff --git a/include/linux/basic_mmio_gpio.h b/include/linux/basic_mmio_gpio.h index 14eea946e6401c..ed3768f4ecc70f 100644 --- a/include/linux/basic_mmio_gpio.h +++ b/include/linux/basic_mmio_gpio.h @@ -75,5 +75,6 @@ int bgpio_init(struct bgpio_chip *bgc, struct device *dev, #define BGPIOF_UNREADABLE_REG_DIR BIT(2) /* reg_dir is unreadable */ #define BGPIOF_BIG_ENDIAN_BYTE_ORDER BIT(3) #define BGPIOF_READ_OUTPUT_REG_SET BIT(4) /* reg_set stores output value */ +#define BGPIOF_NO_OUTPUT BIT(5) /* only input */ #endif /* __BASIC_MMIO_GPIO_H */ diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 4b7b4ebaa633c4..e8130138f29d43 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -118,9 +118,8 @@ struct bio { #define BIO_USER_MAPPED 4 /* contains user pages */ #define BIO_NULL_MAPPED 5 /* contains invalid user pages */ #define BIO_QUIET 6 /* Make BIO Quiet */ -#define BIO_SNAP_STABLE 7 /* bio data must be snapshotted during write */ -#define BIO_CHAIN 8 /* chained bio, ->bi_remaining in effect */ -#define BIO_REFFED 9 /* bio has elevated ->bi_cnt */ +#define BIO_CHAIN 7 /* chained bio, ->bi_remaining in effect */ +#define BIO_REFFED 8 /* bio has elevated ->bi_cnt */ /* * Flags starting here get preserved by bio_reset() - this includes diff --git a/include/linux/compiler.h b/include/linux/compiler.h index e08a6ae7c0a422..c836eb2dc44d5b 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -252,7 +252,12 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s ({ union { typeof(x) __val; char __c[1]; } __u; __read_once_size(&(x), __u.__c, sizeof(x)); __u.__val; }) #define WRITE_ONCE(x, val) \ - ({ union { typeof(x) __val; char __c[1]; } __u = { .__val = (val) }; __write_once_size(&(x), __u.__c, sizeof(x)); __u.__val; }) +({ \ + union { typeof(x) __val; char __c[1]; } __u = \ + { .__val = (__force typeof(x)) (val) }; \ + __write_once_size(&(x), __u.__c, sizeof(x)); \ + __u.__val; \ +}) /** * READ_ONCE_CTRL - Read a value heading a control dependency diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h index 920408a21ffdcf..25c6324a0dd04a 100644 --- a/include/linux/f2fs_fs.h +++ b/include/linux/f2fs_fs.h @@ -417,15 +417,25 @@ typedef __le32 f2fs_hash_t; #define GET_DENTRY_SLOTS(x) ((x + F2FS_SLOT_LEN - 1) >> F2FS_SLOT_LEN_BITS) -/* the number of dentry in a block */ -#define NR_DENTRY_IN_BLOCK 214 - /* MAX level for dir lookup */ #define MAX_DIR_HASH_DEPTH 63 /* MAX buckets in one level of dir */ #define MAX_DIR_BUCKETS (1 << ((MAX_DIR_HASH_DEPTH / 2) - 1)) +/* + * space utilization of regular dentry and inline dentry + * regular dentry inline dentry + * bitmap 1 * 27 = 27 1 * 23 = 23 + * reserved 1 * 3 = 3 1 * 7 = 7 + * dentry 11 * 214 = 2354 11 * 182 = 2002 + * filename 8 * 214 = 1712 8 * 182 = 1456 + * total 4096 3488 + * + * Note: there are more reserved space in inline dentry than in regular + * dentry, when converting inline dentry we should handle this carefully. + */ +#define NR_DENTRY_IN_BLOCK 214 /* the number of dentry in a block */ #define SIZE_OF_DIR_ENTRY 11 /* by byte */ #define SIZE_OF_DENTRY_BITMAP ((NR_DENTRY_IN_BLOCK + BITS_PER_BYTE - 1) / \ BITS_PER_BYTE) diff --git a/include/linux/fsl_ifc.h b/include/linux/fsl_ifc.h index bf0321eabbda40..0023088b253b4d 100644 --- a/include/linux/fsl_ifc.h +++ b/include/linux/fsl_ifc.h @@ -841,9 +841,59 @@ struct fsl_ifc_ctrl { u32 nand_stat; wait_queue_head_t nand_wait; + bool little_endian; }; extern struct fsl_ifc_ctrl *fsl_ifc_ctrl_dev; +static inline u32 ifc_in32(void __iomem *addr) +{ + u32 val; + + if (fsl_ifc_ctrl_dev->little_endian) + val = ioread32(addr); + else + val = ioread32be(addr); + + return val; +} + +static inline u16 ifc_in16(void __iomem *addr) +{ + u16 val; + + if (fsl_ifc_ctrl_dev->little_endian) + val = ioread16(addr); + else + val = ioread16be(addr); + + return val; +} + +static inline u8 ifc_in8(void __iomem *addr) +{ + return ioread8(addr); +} + +static inline void ifc_out32(u32 val, void __iomem *addr) +{ + if (fsl_ifc_ctrl_dev->little_endian) + iowrite32(val, addr); + else + iowrite32be(val, addr); +} + +static inline void ifc_out16(u16 val, void __iomem *addr) +{ + if (fsl_ifc_ctrl_dev->little_endian) + iowrite16(val, addr); + else + iowrite16be(val, addr); +} + +static inline void ifc_out8(u8 val, void __iomem *addr) +{ + iowrite8(val, addr); +} #endif /* __ASM_FSL_IFC_H */ diff --git a/include/linux/gpio/consumer.h b/include/linux/gpio/consumer.h index adac255aee86e3..14cac67c2012b6 100644 --- a/include/linux/gpio/consumer.h +++ b/include/linux/gpio/consumer.h @@ -47,17 +47,17 @@ enum gpiod_flags { int gpiod_count(struct device *dev, const char *con_id); /* Acquire and dispose GPIOs */ -struct gpio_desc *__must_check __gpiod_get(struct device *dev, +struct gpio_desc *__must_check gpiod_get(struct device *dev, const char *con_id, enum gpiod_flags flags); -struct gpio_desc *__must_check __gpiod_get_index(struct device *dev, +struct gpio_desc *__must_check gpiod_get_index(struct device *dev, const char *con_id, unsigned int idx, enum gpiod_flags flags); -struct gpio_desc *__must_check __gpiod_get_optional(struct device *dev, +struct gpio_desc *__must_check gpiod_get_optional(struct device *dev, const char *con_id, enum gpiod_flags flags); -struct gpio_desc *__must_check __gpiod_get_index_optional(struct device *dev, +struct gpio_desc *__must_check gpiod_get_index_optional(struct device *dev, const char *con_id, unsigned int index, enum gpiod_flags flags); @@ -70,18 +70,18 @@ struct gpio_descs *__must_check gpiod_get_array_optional(struct device *dev, void gpiod_put(struct gpio_desc *desc); void gpiod_put_array(struct gpio_descs *descs); -struct gpio_desc *__must_check __devm_gpiod_get(struct device *dev, +struct gpio_desc *__must_check devm_gpiod_get(struct device *dev, const char *con_id, enum gpiod_flags flags); -struct gpio_desc *__must_check __devm_gpiod_get_index(struct device *dev, +struct gpio_desc *__must_check devm_gpiod_get_index(struct device *dev, const char *con_id, unsigned int idx, enum gpiod_flags flags); -struct gpio_desc *__must_check __devm_gpiod_get_optional(struct device *dev, +struct gpio_desc *__must_check devm_gpiod_get_optional(struct device *dev, const char *con_id, enum gpiod_flags flags); struct gpio_desc *__must_check -__devm_gpiod_get_index_optional(struct device *dev, const char *con_id, +devm_gpiod_get_index_optional(struct device *dev, const char *con_id, unsigned int index, enum gpiod_flags flags); struct gpio_descs *__must_check devm_gpiod_get_array(struct device *dev, const char *con_id, @@ -146,31 +146,31 @@ static inline int gpiod_count(struct device *dev, const char *con_id) return 0; } -static inline struct gpio_desc *__must_check __gpiod_get(struct device *dev, - const char *con_id, - enum gpiod_flags flags) +static inline struct gpio_desc *__must_check gpiod_get(struct device *dev, + const char *con_id, + enum gpiod_flags flags) { return ERR_PTR(-ENOSYS); } static inline struct gpio_desc *__must_check -__gpiod_get_index(struct device *dev, - const char *con_id, - unsigned int idx, - enum gpiod_flags flags) +gpiod_get_index(struct device *dev, + const char *con_id, + unsigned int idx, + enum gpiod_flags flags) { return ERR_PTR(-ENOSYS); } static inline struct gpio_desc *__must_check -__gpiod_get_optional(struct device *dev, const char *con_id, - enum gpiod_flags flags) +gpiod_get_optional(struct device *dev, const char *con_id, + enum gpiod_flags flags) { return ERR_PTR(-ENOSYS); } static inline struct gpio_desc *__must_check -__gpiod_get_index_optional(struct device *dev, const char *con_id, - unsigned int index, enum gpiod_flags flags) +gpiod_get_index_optional(struct device *dev, const char *con_id, + unsigned int index, enum gpiod_flags flags) { return ERR_PTR(-ENOSYS); } @@ -206,7 +206,7 @@ static inline void gpiod_put_array(struct gpio_descs *descs) } static inline struct gpio_desc *__must_check -__devm_gpiod_get(struct device *dev, +devm_gpiod_get(struct device *dev, const char *con_id, enum gpiod_flags flags) { @@ -214,7 +214,7 @@ __devm_gpiod_get(struct device *dev, } static inline struct gpio_desc *__must_check -__devm_gpiod_get_index(struct device *dev, +devm_gpiod_get_index(struct device *dev, const char *con_id, unsigned int idx, enum gpiod_flags flags) @@ -223,14 +223,14 @@ __devm_gpiod_get_index(struct device *dev, } static inline struct gpio_desc *__must_check -__devm_gpiod_get_optional(struct device *dev, const char *con_id, +devm_gpiod_get_optional(struct device *dev, const char *con_id, enum gpiod_flags flags) { return ERR_PTR(-ENOSYS); } static inline struct gpio_desc *__must_check -__devm_gpiod_get_index_optional(struct device *dev, const char *con_id, +devm_gpiod_get_index_optional(struct device *dev, const char *con_id, unsigned int index, enum gpiod_flags flags) { return ERR_PTR(-ENOSYS); @@ -424,42 +424,6 @@ static inline struct gpio_desc *devm_get_gpiod_from_child( #endif /* CONFIG_GPIOLIB */ -/* - * Vararg-hacks! This is done to transition the kernel to always pass - * the options flags argument to the below functions. During a transition - * phase these vararg macros make both old-and-newstyle code compile, - * but when all calls to the elder API are removed, these should go away - * and the __gpiod_get() etc functions above be renamed just gpiod_get() - * etc. - */ -#define __gpiod_get(dev, con_id, flags, ...) __gpiod_get(dev, con_id, flags) -#define gpiod_get(varargs...) __gpiod_get(varargs, GPIOD_ASIS) -#define __gpiod_get_index(dev, con_id, index, flags, ...) \ - __gpiod_get_index(dev, con_id, index, flags) -#define gpiod_get_index(varargs...) __gpiod_get_index(varargs, GPIOD_ASIS) -#define __gpiod_get_optional(dev, con_id, flags, ...) \ - __gpiod_get_optional(dev, con_id, flags) -#define gpiod_get_optional(varargs...) __gpiod_get_optional(varargs, GPIOD_ASIS) -#define __gpiod_get_index_optional(dev, con_id, index, flags, ...) \ - __gpiod_get_index_optional(dev, con_id, index, flags) -#define gpiod_get_index_optional(varargs...) \ - __gpiod_get_index_optional(varargs, GPIOD_ASIS) -#define __devm_gpiod_get(dev, con_id, flags, ...) \ - __devm_gpiod_get(dev, con_id, flags) -#define devm_gpiod_get(varargs...) __devm_gpiod_get(varargs, GPIOD_ASIS) -#define __devm_gpiod_get_index(dev, con_id, index, flags, ...) \ - __devm_gpiod_get_index(dev, con_id, index, flags) -#define devm_gpiod_get_index(varargs...) \ - __devm_gpiod_get_index(varargs, GPIOD_ASIS) -#define __devm_gpiod_get_optional(dev, con_id, flags, ...) \ - __devm_gpiod_get_optional(dev, con_id, flags) -#define devm_gpiod_get_optional(varargs...) \ - __devm_gpiod_get_optional(varargs, GPIOD_ASIS) -#define __devm_gpiod_get_index_optional(dev, con_id, index, flags, ...) \ - __devm_gpiod_get_index_optional(dev, con_id, index, flags) -#define devm_gpiod_get_index_optional(varargs...) \ - __devm_gpiod_get_index_optional(varargs, GPIOD_ASIS) - #if IS_ENABLED(CONFIG_GPIOLIB) && IS_ENABLED(CONFIG_GPIO_SYSFS) int gpiod_export(struct gpio_desc *desc, bool direction_may_change); diff --git a/include/linux/gpio/driver.h b/include/linux/gpio/driver.h index c8393cd4d44f2d..1aed31c5ffba33 100644 --- a/include/linux/gpio/driver.h +++ b/include/linux/gpio/driver.h @@ -6,6 +6,7 @@ #include #include #include +#include #include struct device; @@ -64,6 +65,17 @@ struct seq_file; * registers. * @irq_not_threaded: flag must be set if @can_sleep is set but the * IRQs don't need to be threaded + * @irqchip: GPIO IRQ chip impl, provided by GPIO driver + * @irqdomain: Interrupt translation domain; responsible for mapping + * between GPIO hwirq number and linux irq number + * @irq_base: first linux IRQ number assigned to GPIO IRQ chip (deprecated) + * @irq_handler: the irq handler to use (often a predefined irq core function) + * for GPIO IRQs, provided by GPIO driver + * @irq_default_type: default IRQ triggering type applied during GPIO driver + * initialization, provided by GPIO driver + * @irq_parent: GPIO IRQ chip parent/bank linux irq number, + * provided by GPIO driver + * @lock_key: per GPIO IRQ chip lockdep class * * A gpio_chip can help platforms abstract various sources of GPIOs so * they can all be accessed through a common programing interface. @@ -126,6 +138,7 @@ struct gpio_chip { irq_flow_handler_t irq_handler; unsigned int irq_default_type; int irq_parent; + struct lock_class_key *lock_key; #endif #if defined(CONFIG_OF_GPIO) @@ -171,11 +184,25 @@ void gpiochip_set_chained_irqchip(struct gpio_chip *gpiochip, int parent_irq, irq_flow_handler_t parent_handler); -int gpiochip_irqchip_add(struct gpio_chip *gpiochip, - struct irq_chip *irqchip, - unsigned int first_irq, - irq_flow_handler_t handler, - unsigned int type); +int _gpiochip_irqchip_add(struct gpio_chip *gpiochip, + struct irq_chip *irqchip, + unsigned int first_irq, + irq_flow_handler_t handler, + unsigned int type, + struct lock_class_key *lock_key); + +#ifdef CONFIG_LOCKDEP +#define gpiochip_irqchip_add(...) \ +( \ + ({ \ + static struct lock_class_key _key; \ + _gpiochip_irqchip_add(__VA_ARGS__, &_key); \ + }) \ +) +#else +#define gpiochip_irqchip_add(...) \ + _gpiochip_irqchip_add(__VA_ARGS__, NULL) +#endif #endif /* CONFIG_GPIOLIB_IRQCHIP */ diff --git a/include/linux/irqchip/mips-gic.h b/include/linux/irqchip/mips-gic.h index 9b1ad3734911bc..4e686160505048 100644 --- a/include/linux/irqchip/mips-gic.h +++ b/include/linux/irqchip/mips-gic.h @@ -41,12 +41,20 @@ /* Shared Global Counter */ #define GIC_SH_COUNTER_31_00_OFS 0x0010 +/* 64-bit counter register for CM3 */ +#define GIC_SH_COUNTER_OFS GIC_SH_COUNTER_31_00_OFS #define GIC_SH_COUNTER_63_32_OFS 0x0014 #define GIC_SH_REVISIONID_OFS 0x0020 /* Convert an interrupt number to a byte offset/bit for multi-word registers */ -#define GIC_INTR_OFS(intr) (((intr) / 32) * 4) -#define GIC_INTR_BIT(intr) ((intr) % 32) +#define GIC_INTR_OFS(intr) ({ \ + unsigned bits = mips_cm_is64 ? 64 : 32; \ + unsigned reg_idx = (intr) / bits; \ + unsigned reg_width = bits / 8; \ + \ + reg_idx * reg_width; \ +}) +#define GIC_INTR_BIT(intr) ((intr) % (mips_cm_is64 ? 64 : 32)) /* Polarity : Reset Value is always 0 */ #define GIC_SH_SET_POLARITY_OFS 0x0100 @@ -98,6 +106,8 @@ #define GIC_VPE_WD_COUNT0_OFS 0x0094 #define GIC_VPE_WD_INITIAL0_OFS 0x0098 #define GIC_VPE_COMPARE_LO_OFS 0x00a0 +/* 64-bit Compare register on CM3 */ +#define GIC_VPE_COMPARE_OFS GIC_VPE_COMPARE_LO_OFS #define GIC_VPE_COMPARE_HI_OFS 0x00a4 #define GIC_VPE_EIC_SHADOW_SET_BASE_OFS 0x0100 diff --git a/include/linux/jbd.h b/include/linux/jbd.h deleted file mode 100644 index d32615280be92b..00000000000000 --- a/include/linux/jbd.h +++ /dev/null @@ -1,1047 +0,0 @@ -/* - * linux/include/linux/jbd.h - * - * Written by Stephen C. Tweedie - * - * Copyright 1998-2000 Red Hat, Inc --- All Rights Reserved - * - * This file is part of the Linux kernel and is made available under - * the terms of the GNU General Public License, version 2, or at your - * option, any later version, incorporated herein by reference. - * - * Definitions for transaction data structures for the buffer cache - * filesystem journaling support. - */ - -#ifndef _LINUX_JBD_H -#define _LINUX_JBD_H - -/* Allow this file to be included directly into e2fsprogs */ -#ifndef __KERNEL__ -#include "jfs_compat.h" -#define JFS_DEBUG -#define jfs_debug jbd_debug -#else - -#include -#include -#include -#include -#include -#include -#include -#include - -#define journal_oom_retry 1 - -/* - * Define JBD_PARANOID_IOFAIL to cause a kernel BUG() if ext3 finds - * certain classes of error which can occur due to failed IOs. Under - * normal use we want ext3 to continue after such errors, because - * hardware _can_ fail, but for debugging purposes when running tests on - * known-good hardware we may want to trap these errors. - */ -#undef JBD_PARANOID_IOFAIL - -/* - * The default maximum commit age, in seconds. - */ -#define JBD_DEFAULT_MAX_COMMIT_AGE 5 - -#ifdef CONFIG_JBD_DEBUG -/* - * Define JBD_EXPENSIVE_CHECKING to enable more expensive internal - * consistency checks. By default we don't do this unless - * CONFIG_JBD_DEBUG is on. - */ -#define JBD_EXPENSIVE_CHECKING -extern u8 journal_enable_debug; - -void __jbd_debug(int level, const char *file, const char *func, - unsigned int line, const char *fmt, ...); - -#define jbd_debug(n, fmt, a...) \ - __jbd_debug((n), __FILE__, __func__, __LINE__, (fmt), ##a) -#else -#define jbd_debug(n, fmt, a...) /**/ -#endif - -static inline void *jbd_alloc(size_t size, gfp_t flags) -{ - return (void *)__get_free_pages(flags, get_order(size)); -} - -static inline void jbd_free(void *ptr, size_t size) -{ - free_pages((unsigned long)ptr, get_order(size)); -} - -#define JFS_MIN_JOURNAL_BLOCKS 1024 - - -/** - * typedef handle_t - The handle_t type represents a single atomic update being performed by some process. - * - * All filesystem modifications made by the process go - * through this handle. Recursive operations (such as quota operations) - * are gathered into a single update. - * - * The buffer credits field is used to account for journaled buffers - * being modified by the running process. To ensure that there is - * enough log space for all outstanding operations, we need to limit the - * number of outstanding buffers possible at any time. When the - * operation completes, any buffer credits not used are credited back to - * the transaction, so that at all times we know how many buffers the - * outstanding updates on a transaction might possibly touch. - * - * This is an opaque datatype. - **/ -typedef struct handle_s handle_t; /* Atomic operation type */ - - -/** - * typedef journal_t - The journal_t maintains all of the journaling state information for a single filesystem. - * - * journal_t is linked to from the fs superblock structure. - * - * We use the journal_t to keep track of all outstanding transaction - * activity on the filesystem, and to manage the state of the log - * writing process. - * - * This is an opaque datatype. - **/ -typedef struct journal_s journal_t; /* Journal control structure */ -#endif - -/* - * Internal structures used by the logging mechanism: - */ - -#define JFS_MAGIC_NUMBER 0xc03b3998U /* The first 4 bytes of /dev/random! */ - -/* - * On-disk structures - */ - -/* - * Descriptor block types: - */ - -#define JFS_DESCRIPTOR_BLOCK 1 -#define JFS_COMMIT_BLOCK 2 -#define JFS_SUPERBLOCK_V1 3 -#define JFS_SUPERBLOCK_V2 4 -#define JFS_REVOKE_BLOCK 5 - -/* - * Standard header for all descriptor blocks: - */ -typedef struct journal_header_s -{ - __be32 h_magic; - __be32 h_blocktype; - __be32 h_sequence; -} journal_header_t; - - -/* - * The block tag: used to describe a single buffer in the journal - */ -typedef struct journal_block_tag_s -{ - __be32 t_blocknr; /* The on-disk block number */ - __be32 t_flags; /* See below */ -} journal_block_tag_t; - -/* - * The revoke descriptor: used on disk to describe a series of blocks to - * be revoked from the log - */ -typedef struct journal_revoke_header_s -{ - journal_header_t r_header; - __be32 r_count; /* Count of bytes used in the block */ -} journal_revoke_header_t; - - -/* Definitions for the journal tag flags word: */ -#define JFS_FLAG_ESCAPE 1 /* on-disk block is escaped */ -#define JFS_FLAG_SAME_UUID 2 /* block has same uuid as previous */ -#define JFS_FLAG_DELETED 4 /* block deleted by this transaction */ -#define JFS_FLAG_LAST_TAG 8 /* last tag in this descriptor block */ - - -/* - * The journal superblock. All fields are in big-endian byte order. - */ -typedef struct journal_superblock_s -{ -/* 0x0000 */ - journal_header_t s_header; - -/* 0x000C */ - /* Static information describing the journal */ - __be32 s_blocksize; /* journal device blocksize */ - __be32 s_maxlen; /* total blocks in journal file */ - __be32 s_first; /* first block of log information */ - -/* 0x0018 */ - /* Dynamic information describing the current state of the log */ - __be32 s_sequence; /* first commit ID expected in log */ - __be32 s_start; /* blocknr of start of log */ - -/* 0x0020 */ - /* Error value, as set by journal_abort(). */ - __be32 s_errno; - -/* 0x0024 */ - /* Remaining fields are only valid in a version-2 superblock */ - __be32 s_feature_compat; /* compatible feature set */ - __be32 s_feature_incompat; /* incompatible feature set */ - __be32 s_feature_ro_compat; /* readonly-compatible feature set */ -/* 0x0030 */ - __u8 s_uuid[16]; /* 128-bit uuid for journal */ - -/* 0x0040 */ - __be32 s_nr_users; /* Nr of filesystems sharing log */ - - __be32 s_dynsuper; /* Blocknr of dynamic superblock copy*/ - -/* 0x0048 */ - __be32 s_max_transaction; /* Limit of journal blocks per trans.*/ - __be32 s_max_trans_data; /* Limit of data blocks per trans. */ - -/* 0x0050 */ - __u32 s_padding[44]; - -/* 0x0100 */ - __u8 s_users[16*48]; /* ids of all fs'es sharing the log */ -/* 0x0400 */ -} journal_superblock_t; - -#define JFS_HAS_COMPAT_FEATURE(j,mask) \ - ((j)->j_format_version >= 2 && \ - ((j)->j_superblock->s_feature_compat & cpu_to_be32((mask)))) -#define JFS_HAS_RO_COMPAT_FEATURE(j,mask) \ - ((j)->j_format_version >= 2 && \ - ((j)->j_superblock->s_feature_ro_compat & cpu_to_be32((mask)))) -#define JFS_HAS_INCOMPAT_FEATURE(j,mask) \ - ((j)->j_format_version >= 2 && \ - ((j)->j_superblock->s_feature_incompat & cpu_to_be32((mask)))) - -#define JFS_FEATURE_INCOMPAT_REVOKE 0x00000001 - -/* Features known to this kernel version: */ -#define JFS_KNOWN_COMPAT_FEATURES 0 -#define JFS_KNOWN_ROCOMPAT_FEATURES 0 -#define JFS_KNOWN_INCOMPAT_FEATURES JFS_FEATURE_INCOMPAT_REVOKE - -#ifdef __KERNEL__ - -#include -#include - -enum jbd_state_bits { - BH_JBD /* Has an attached ext3 journal_head */ - = BH_PrivateStart, - BH_JWrite, /* Being written to log (@@@ DEBUGGING) */ - BH_Freed, /* Has been freed (truncated) */ - BH_Revoked, /* Has been revoked from the log */ - BH_RevokeValid, /* Revoked flag is valid */ - BH_JBDDirty, /* Is dirty but journaled */ - BH_State, /* Pins most journal_head state */ - BH_JournalHead, /* Pins bh->b_private and jh->b_bh */ - BH_Unshadow, /* Dummy bit, for BJ_Shadow wakeup filtering */ - BH_JBDPrivateStart, /* First bit available for private use by FS */ -}; - -BUFFER_FNS(JBD, jbd) -BUFFER_FNS(JWrite, jwrite) -BUFFER_FNS(JBDDirty, jbddirty) -TAS_BUFFER_FNS(JBDDirty, jbddirty) -BUFFER_FNS(Revoked, revoked) -TAS_BUFFER_FNS(Revoked, revoked) -BUFFER_FNS(RevokeValid, revokevalid) -TAS_BUFFER_FNS(RevokeValid, revokevalid) -BUFFER_FNS(Freed, freed) - -#include - -#define J_ASSERT(assert) BUG_ON(!(assert)) - -#define J_ASSERT_BH(bh, expr) J_ASSERT(expr) -#define J_ASSERT_JH(jh, expr) J_ASSERT(expr) - -#if defined(JBD_PARANOID_IOFAIL) -#define J_EXPECT(expr, why...) J_ASSERT(expr) -#define J_EXPECT_BH(bh, expr, why...) J_ASSERT_BH(bh, expr) -#define J_EXPECT_JH(jh, expr, why...) J_ASSERT_JH(jh, expr) -#else -#define __journal_expect(expr, why...) \ - ({ \ - int val = (expr); \ - if (!val) { \ - printk(KERN_ERR \ - "EXT3-fs unexpected failure: %s;\n",# expr); \ - printk(KERN_ERR why "\n"); \ - } \ - val; \ - }) -#define J_EXPECT(expr, why...) __journal_expect(expr, ## why) -#define J_EXPECT_BH(bh, expr, why...) __journal_expect(expr, ## why) -#define J_EXPECT_JH(jh, expr, why...) __journal_expect(expr, ## why) -#endif - -struct jbd_revoke_table_s; - -/** - * struct handle_s - this is the concrete type associated with handle_t. - * @h_transaction: Which compound transaction is this update a part of? - * @h_buffer_credits: Number of remaining buffers we are allowed to dirty. - * @h_ref: Reference count on this handle - * @h_err: Field for caller's use to track errors through large fs operations - * @h_sync: flag for sync-on-close - * @h_jdata: flag to force data journaling - * @h_aborted: flag indicating fatal error on handle - * @h_lockdep_map: lockdep info for debugging lock problems - */ -struct handle_s -{ - /* Which compound transaction is this update a part of? */ - transaction_t *h_transaction; - - /* Number of remaining buffers we are allowed to dirty: */ - int h_buffer_credits; - - /* Reference count on this handle */ - int h_ref; - - /* Field for caller's use to track errors through large fs */ - /* operations */ - int h_err; - - /* Flags [no locking] */ - unsigned int h_sync: 1; /* sync-on-close */ - unsigned int h_jdata: 1; /* force data journaling */ - unsigned int h_aborted: 1; /* fatal error on handle */ - -#ifdef CONFIG_DEBUG_LOCK_ALLOC - struct lockdep_map h_lockdep_map; -#endif -}; - - -/* The transaction_t type is the guts of the journaling mechanism. It - * tracks a compound transaction through its various states: - * - * RUNNING: accepting new updates - * LOCKED: Updates still running but we don't accept new ones - * RUNDOWN: Updates are tidying up but have finished requesting - * new buffers to modify (state not used for now) - * FLUSH: All updates complete, but we are still writing to disk - * COMMIT: All data on disk, writing commit record - * FINISHED: We still have to keep the transaction for checkpointing. - * - * The transaction keeps track of all of the buffers modified by a - * running transaction, and all of the buffers committed but not yet - * flushed to home for finished transactions. - */ - -/* - * Lock ranking: - * - * j_list_lock - * ->jbd_lock_bh_journal_head() (This is "innermost") - * - * j_state_lock - * ->jbd_lock_bh_state() - * - * jbd_lock_bh_state() - * ->j_list_lock - * - * j_state_lock - * ->t_handle_lock - * - * j_state_lock - * ->j_list_lock (journal_unmap_buffer) - * - */ - -struct transaction_s -{ - /* Pointer to the journal for this transaction. [no locking] */ - journal_t *t_journal; - - /* Sequence number for this transaction [no locking] */ - tid_t t_tid; - - /* - * Transaction's current state - * [no locking - only kjournald alters this] - * [j_list_lock] guards transition of a transaction into T_FINISHED - * state and subsequent call of __journal_drop_transaction() - * FIXME: needs barriers - * KLUDGE: [use j_state_lock] - */ - enum { - T_RUNNING, - T_LOCKED, - T_FLUSH, - T_COMMIT, - T_COMMIT_RECORD, - T_FINISHED - } t_state; - - /* - * Where in the log does this transaction's commit start? [no locking] - */ - unsigned int t_log_start; - - /* Number of buffers on the t_buffers list [j_list_lock] */ - int t_nr_buffers; - - /* - * Doubly-linked circular list of all buffers reserved but not yet - * modified by this transaction [j_list_lock] - */ - struct journal_head *t_reserved_list; - - /* - * Doubly-linked circular list of all buffers under writeout during - * commit [j_list_lock] - */ - struct journal_head *t_locked_list; - - /* - * Doubly-linked circular list of all metadata buffers owned by this - * transaction [j_list_lock] - */ - struct journal_head *t_buffers; - - /* - * Doubly-linked circular list of all data buffers still to be - * flushed before this transaction can be committed [j_list_lock] - */ - struct journal_head *t_sync_datalist; - - /* - * Doubly-linked circular list of all forget buffers (superseded - * buffers which we can un-checkpoint once this transaction commits) - * [j_list_lock] - */ - struct journal_head *t_forget; - - /* - * Doubly-linked circular list of all buffers still to be flushed before - * this transaction can be checkpointed. [j_list_lock] - */ - struct journal_head *t_checkpoint_list; - - /* - * Doubly-linked circular list of all buffers submitted for IO while - * checkpointing. [j_list_lock] - */ - struct journal_head *t_checkpoint_io_list; - - /* - * Doubly-linked circular list of temporary buffers currently undergoing - * IO in the log [j_list_lock] - */ - struct journal_head *t_iobuf_list; - - /* - * Doubly-linked circular list of metadata buffers being shadowed by log - * IO. The IO buffers on the iobuf list and the shadow buffers on this - * list match each other one for one at all times. [j_list_lock] - */ - struct journal_head *t_shadow_list; - - /* - * Doubly-linked circular list of control buffers being written to the - * log. [j_list_lock] - */ - struct journal_head *t_log_list; - - /* - * Protects info related to handles - */ - spinlock_t t_handle_lock; - - /* - * Number of outstanding updates running on this transaction - * [t_handle_lock] - */ - int t_updates; - - /* - * Number of buffers reserved for use by all handles in this transaction - * handle but not yet modified. [t_handle_lock] - */ - int t_outstanding_credits; - - /* - * Forward and backward links for the circular list of all transactions - * awaiting checkpoint. [j_list_lock] - */ - transaction_t *t_cpnext, *t_cpprev; - - /* - * When will the transaction expire (become due for commit), in jiffies? - * [no locking] - */ - unsigned long t_expires; - - /* - * When this transaction started, in nanoseconds [no locking] - */ - ktime_t t_start_time; - - /* - * How many handles used this transaction? [t_handle_lock] - */ - int t_handle_count; -}; - -/** - * struct journal_s - this is the concrete type associated with journal_t. - * @j_flags: General journaling state flags - * @j_errno: Is there an outstanding uncleared error on the journal (from a - * prior abort)? - * @j_sb_buffer: First part of superblock buffer - * @j_superblock: Second part of superblock buffer - * @j_format_version: Version of the superblock format - * @j_state_lock: Protect the various scalars in the journal - * @j_barrier_count: Number of processes waiting to create a barrier lock - * @j_running_transaction: The current running transaction.. - * @j_committing_transaction: the transaction we are pushing to disk - * @j_checkpoint_transactions: a linked circular list of all transactions - * waiting for checkpointing - * @j_wait_transaction_locked: Wait queue for waiting for a locked transaction - * to start committing, or for a barrier lock to be released - * @j_wait_logspace: Wait queue for waiting for checkpointing to complete - * @j_wait_done_commit: Wait queue for waiting for commit to complete - * @j_wait_checkpoint: Wait queue to trigger checkpointing - * @j_wait_commit: Wait queue to trigger commit - * @j_wait_updates: Wait queue to wait for updates to complete - * @j_checkpoint_mutex: Mutex for locking against concurrent checkpoints - * @j_head: Journal head - identifies the first unused block in the journal - * @j_tail: Journal tail - identifies the oldest still-used block in the - * journal. - * @j_free: Journal free - how many free blocks are there in the journal? - * @j_first: The block number of the first usable block - * @j_last: The block number one beyond the last usable block - * @j_dev: Device where we store the journal - * @j_blocksize: blocksize for the location where we store the journal. - * @j_blk_offset: starting block offset for into the device where we store the - * journal - * @j_fs_dev: Device which holds the client fs. For internal journal this will - * be equal to j_dev - * @j_maxlen: Total maximum capacity of the journal region on disk. - * @j_list_lock: Protects the buffer lists and internal buffer state. - * @j_inode: Optional inode where we store the journal. If present, all journal - * block numbers are mapped into this inode via bmap(). - * @j_tail_sequence: Sequence number of the oldest transaction in the log - * @j_transaction_sequence: Sequence number of the next transaction to grant - * @j_commit_sequence: Sequence number of the most recently committed - * transaction - * @j_commit_request: Sequence number of the most recent transaction wanting - * commit - * @j_commit_waited: Sequence number of the most recent transaction someone - * is waiting for to commit. - * @j_uuid: Uuid of client object. - * @j_task: Pointer to the current commit thread for this journal - * @j_max_transaction_buffers: Maximum number of metadata buffers to allow in a - * single compound commit transaction - * @j_commit_interval: What is the maximum transaction lifetime before we begin - * a commit? - * @j_commit_timer: The timer used to wakeup the commit thread - * @j_revoke_lock: Protect the revoke table - * @j_revoke: The revoke table - maintains the list of revoked blocks in the - * current transaction. - * @j_revoke_table: alternate revoke tables for j_revoke - * @j_wbuf: array of buffer_heads for journal_commit_transaction - * @j_wbufsize: maximum number of buffer_heads allowed in j_wbuf, the - * number that will fit in j_blocksize - * @j_last_sync_writer: most recent pid which did a synchronous write - * @j_average_commit_time: the average amount of time in nanoseconds it - * takes to commit a transaction to the disk. - * @j_private: An opaque pointer to fs-private information. - */ - -struct journal_s -{ - /* General journaling state flags [j_state_lock] */ - unsigned long j_flags; - - /* - * Is there an outstanding uncleared error on the journal (from a prior - * abort)? [j_state_lock] - */ - int j_errno; - - /* The superblock buffer */ - struct buffer_head *j_sb_buffer; - journal_superblock_t *j_superblock; - - /* Version of the superblock format */ - int j_format_version; - - /* - * Protect the various scalars in the journal - */ - spinlock_t j_state_lock; - - /* - * Number of processes waiting to create a barrier lock [j_state_lock] - */ - int j_barrier_count; - - /* - * Transactions: The current running transaction... - * [j_state_lock] [caller holding open handle] - */ - transaction_t *j_running_transaction; - - /* - * the transaction we are pushing to disk - * [j_state_lock] [caller holding open handle] - */ - transaction_t *j_committing_transaction; - - /* - * ... and a linked circular list of all transactions waiting for - * checkpointing. [j_list_lock] - */ - transaction_t *j_checkpoint_transactions; - - /* - * Wait queue for waiting for a locked transaction to start committing, - * or for a barrier lock to be released - */ - wait_queue_head_t j_wait_transaction_locked; - - /* Wait queue for waiting for checkpointing to complete */ - wait_queue_head_t j_wait_logspace; - - /* Wait queue for waiting for commit to complete */ - wait_queue_head_t j_wait_done_commit; - - /* Wait queue to trigger checkpointing */ - wait_queue_head_t j_wait_checkpoint; - - /* Wait queue to trigger commit */ - wait_queue_head_t j_wait_commit; - - /* Wait queue to wait for updates to complete */ - wait_queue_head_t j_wait_updates; - - /* Semaphore for locking against concurrent checkpoints */ - struct mutex j_checkpoint_mutex; - - /* - * Journal head: identifies the first unused block in the journal. - * [j_state_lock] - */ - unsigned int j_head; - - /* - * Journal tail: identifies the oldest still-used block in the journal. - * [j_state_lock] - */ - unsigned int j_tail; - - /* - * Journal free: how many free blocks are there in the journal? - * [j_state_lock] - */ - unsigned int j_free; - - /* - * Journal start and end: the block numbers of the first usable block - * and one beyond the last usable block in the journal. [j_state_lock] - */ - unsigned int j_first; - unsigned int j_last; - - /* - * Device, blocksize and starting block offset for the location where we - * store the journal. - */ - struct block_device *j_dev; - int j_blocksize; - unsigned int j_blk_offset; - - /* - * Device which holds the client fs. For internal journal this will be - * equal to j_dev. - */ - struct block_device *j_fs_dev; - - /* Total maximum capacity of the journal region on disk. */ - unsigned int j_maxlen; - - /* - * Protects the buffer lists and internal buffer state. - */ - spinlock_t j_list_lock; - - /* Optional inode where we store the journal. If present, all */ - /* journal block numbers are mapped into this inode via */ - /* bmap(). */ - struct inode *j_inode; - - /* - * Sequence number of the oldest transaction in the log [j_state_lock] - */ - tid_t j_tail_sequence; - - /* - * Sequence number of the next transaction to grant [j_state_lock] - */ - tid_t j_transaction_sequence; - - /* - * Sequence number of the most recently committed transaction - * [j_state_lock]. - */ - tid_t j_commit_sequence; - - /* - * Sequence number of the most recent transaction wanting commit - * [j_state_lock] - */ - tid_t j_commit_request; - - /* - * Sequence number of the most recent transaction someone is waiting - * for to commit. - * [j_state_lock] - */ - tid_t j_commit_waited; - - /* - * Journal uuid: identifies the object (filesystem, LVM volume etc) - * backed by this journal. This will eventually be replaced by an array - * of uuids, allowing us to index multiple devices within a single - * journal and to perform atomic updates across them. - */ - __u8 j_uuid[16]; - - /* Pointer to the current commit thread for this journal */ - struct task_struct *j_task; - - /* - * Maximum number of metadata buffers to allow in a single compound - * commit transaction - */ - int j_max_transaction_buffers; - - /* - * What is the maximum transaction lifetime before we begin a commit? - */ - unsigned long j_commit_interval; - - /* The timer used to wakeup the commit thread: */ - struct timer_list j_commit_timer; - - /* - * The revoke table: maintains the list of revoked blocks in the - * current transaction. [j_revoke_lock] - */ - spinlock_t j_revoke_lock; - struct jbd_revoke_table_s *j_revoke; - struct jbd_revoke_table_s *j_revoke_table[2]; - - /* - * array of bhs for journal_commit_transaction - */ - struct buffer_head **j_wbuf; - int j_wbufsize; - - /* - * this is the pid of the last person to run a synchronous operation - * through the journal. - */ - pid_t j_last_sync_writer; - - /* - * the average amount of time in nanoseconds it takes to commit a - * transaction to the disk. [j_state_lock] - */ - u64 j_average_commit_time; - - /* - * An opaque pointer to fs-private information. ext3 puts its - * superblock pointer here - */ - void *j_private; -}; - -/* - * Journal flag definitions - */ -#define JFS_UNMOUNT 0x001 /* Journal thread is being destroyed */ -#define JFS_ABORT 0x002 /* Journaling has been aborted for errors. */ -#define JFS_ACK_ERR 0x004 /* The errno in the sb has been acked */ -#define JFS_FLUSHED 0x008 /* The journal superblock has been flushed */ -#define JFS_LOADED 0x010 /* The journal superblock has been loaded */ -#define JFS_BARRIER 0x020 /* Use IDE barriers */ -#define JFS_ABORT_ON_SYNCDATA_ERR 0x040 /* Abort the journal on file - * data write error in ordered - * mode */ - -/* - * Function declarations for the journaling transaction and buffer - * management - */ - -/* Filing buffers */ -extern void journal_unfile_buffer(journal_t *, struct journal_head *); -extern void __journal_unfile_buffer(struct journal_head *); -extern void __journal_refile_buffer(struct journal_head *); -extern void journal_refile_buffer(journal_t *, struct journal_head *); -extern void __journal_file_buffer(struct journal_head *, transaction_t *, int); -extern void __journal_free_buffer(struct journal_head *bh); -extern void journal_file_buffer(struct journal_head *, transaction_t *, int); -extern void __journal_clean_data_list(transaction_t *transaction); - -/* Log buffer allocation */ -extern struct journal_head * journal_get_descriptor_buffer(journal_t *); -int journal_next_log_block(journal_t *, unsigned int *); - -/* Commit management */ -extern void journal_commit_transaction(journal_t *); - -/* Checkpoint list management */ -int __journal_clean_checkpoint_list(journal_t *journal); -int __journal_remove_checkpoint(struct journal_head *); -void __journal_insert_checkpoint(struct journal_head *, transaction_t *); - -/* Buffer IO */ -extern int -journal_write_metadata_buffer(transaction_t *transaction, - struct journal_head *jh_in, - struct journal_head **jh_out, - unsigned int blocknr); - -/* Transaction locking */ -extern void __wait_on_journal (journal_t *); - -/* - * Journal locking. - * - * We need to lock the journal during transaction state changes so that nobody - * ever tries to take a handle on the running transaction while we are in the - * middle of moving it to the commit phase. j_state_lock does this. - * - * Note that the locking is completely interrupt unsafe. We never touch - * journal structures from interrupts. - */ - -static inline handle_t *journal_current_handle(void) -{ - return current->journal_info; -} - -/* The journaling code user interface: - * - * Create and destroy handles - * Register buffer modifications against the current transaction. - */ - -extern handle_t *journal_start(journal_t *, int nblocks); -extern int journal_restart (handle_t *, int nblocks); -extern int journal_extend (handle_t *, int nblocks); -extern int journal_get_write_access(handle_t *, struct buffer_head *); -extern int journal_get_create_access (handle_t *, struct buffer_head *); -extern int journal_get_undo_access(handle_t *, struct buffer_head *); -extern int journal_dirty_data (handle_t *, struct buffer_head *); -extern int journal_dirty_metadata (handle_t *, struct buffer_head *); -extern void journal_release_buffer (handle_t *, struct buffer_head *); -extern int journal_forget (handle_t *, struct buffer_head *); -extern void journal_sync_buffer (struct buffer_head *); -extern void journal_invalidatepage(journal_t *, - struct page *, unsigned int, unsigned int); -extern int journal_try_to_free_buffers(journal_t *, struct page *, gfp_t); -extern int journal_stop(handle_t *); -extern int journal_flush (journal_t *); -extern void journal_lock_updates (journal_t *); -extern void journal_unlock_updates (journal_t *); - -extern journal_t * journal_init_dev(struct block_device *bdev, - struct block_device *fs_dev, - int start, int len, int bsize); -extern journal_t * journal_init_inode (struct inode *); -extern int journal_update_format (journal_t *); -extern int journal_check_used_features - (journal_t *, unsigned long, unsigned long, unsigned long); -extern int journal_check_available_features - (journal_t *, unsigned long, unsigned long, unsigned long); -extern int journal_set_features - (journal_t *, unsigned long, unsigned long, unsigned long); -extern int journal_create (journal_t *); -extern int journal_load (journal_t *journal); -extern int journal_destroy (journal_t *); -extern int journal_recover (journal_t *journal); -extern int journal_wipe (journal_t *, int); -extern int journal_skip_recovery (journal_t *); -extern void journal_update_sb_log_tail (journal_t *, tid_t, unsigned int, - int); -extern void journal_abort (journal_t *, int); -extern int journal_errno (journal_t *); -extern void journal_ack_err (journal_t *); -extern int journal_clear_err (journal_t *); -extern int journal_bmap(journal_t *, unsigned int, unsigned int *); -extern int journal_force_commit(journal_t *); - -/* - * journal_head management - */ -struct journal_head *journal_add_journal_head(struct buffer_head *bh); -struct journal_head *journal_grab_journal_head(struct buffer_head *bh); -void journal_put_journal_head(struct journal_head *jh); - -/* - * handle management - */ -extern struct kmem_cache *jbd_handle_cache; - -static inline handle_t *jbd_alloc_handle(gfp_t gfp_flags) -{ - return kmem_cache_zalloc(jbd_handle_cache, gfp_flags); -} - -static inline void jbd_free_handle(handle_t *handle) -{ - kmem_cache_free(jbd_handle_cache, handle); -} - -/* Primary revoke support */ -#define JOURNAL_REVOKE_DEFAULT_HASH 256 -extern int journal_init_revoke(journal_t *, int); -extern void journal_destroy_revoke_caches(void); -extern int journal_init_revoke_caches(void); - -extern void journal_destroy_revoke(journal_t *); -extern int journal_revoke (handle_t *, - unsigned int, struct buffer_head *); -extern int journal_cancel_revoke(handle_t *, struct journal_head *); -extern void journal_write_revoke_records(journal_t *, - transaction_t *, int); - -/* Recovery revoke support */ -extern int journal_set_revoke(journal_t *, unsigned int, tid_t); -extern int journal_test_revoke(journal_t *, unsigned int, tid_t); -extern void journal_clear_revoke(journal_t *); -extern void journal_switch_revoke_table(journal_t *journal); -extern void journal_clear_buffer_revoked_flags(journal_t *journal); - -/* - * The log thread user interface: - * - * Request space in the current transaction, and force transaction commit - * transitions on demand. - */ - -int __log_space_left(journal_t *); /* Called with journal locked */ -int log_start_commit(journal_t *journal, tid_t tid); -int __log_start_commit(journal_t *journal, tid_t tid); -int journal_start_commit(journal_t *journal, tid_t *tid); -int journal_force_commit_nested(journal_t *journal); -int log_wait_commit(journal_t *journal, tid_t tid); -int log_do_checkpoint(journal_t *journal); -int journal_trans_will_send_data_barrier(journal_t *journal, tid_t tid); - -void __log_wait_for_space(journal_t *journal); -extern void __journal_drop_transaction(journal_t *, transaction_t *); -extern int cleanup_journal_tail(journal_t *); - -/* - * is_journal_abort - * - * Simple test wrapper function to test the JFS_ABORT state flag. This - * bit, when set, indicates that we have had a fatal error somewhere, - * either inside the journaling layer or indicated to us by the client - * (eg. ext3), and that we and should not commit any further - * transactions. - */ - -static inline int is_journal_aborted(journal_t *journal) -{ - return journal->j_flags & JFS_ABORT; -} - -static inline int is_handle_aborted(handle_t *handle) -{ - if (handle->h_aborted) - return 1; - return is_journal_aborted(handle->h_transaction->t_journal); -} - -static inline void journal_abort_handle(handle_t *handle) -{ - handle->h_aborted = 1; -} - -#endif /* __KERNEL__ */ - -/* Comparison functions for transaction IDs: perform comparisons using - * modulo arithmetic so that they work over sequence number wraps. */ - -static inline int tid_gt(tid_t x, tid_t y) -{ - int difference = (x - y); - return (difference > 0); -} - -static inline int tid_geq(tid_t x, tid_t y) -{ - int difference = (x - y); - return (difference >= 0); -} - -extern int journal_blocks_per_page(struct inode *inode); - -/* - * Return the minimum number of blocks which must be free in the journal - * before a new transaction may be started. Must be called under j_state_lock. - */ -static inline int jbd_space_needed(journal_t *journal) -{ - int nblocks = journal->j_max_transaction_buffers; - if (journal->j_committing_transaction) - nblocks += journal->j_committing_transaction-> - t_outstanding_credits; - return nblocks; -} - -/* - * Definitions which augment the buffer_head layer - */ - -/* journaling buffer types */ -#define BJ_None 0 /* Not journaled */ -#define BJ_SyncData 1 /* Normal data: flush before commit */ -#define BJ_Metadata 2 /* Normal journaled metadata */ -#define BJ_Forget 3 /* Buffer superseded by this transaction */ -#define BJ_IO 4 /* Buffer is for temporary IO use */ -#define BJ_Shadow 5 /* Buffer contents being shadowed to the log */ -#define BJ_LogCtl 6 /* Buffer contains log descriptors */ -#define BJ_Reserved 7 /* Buffer is reserved for access by journal */ -#define BJ_Locked 8 /* Locked for I/O during commit */ -#define BJ_Types 9 - -extern int jbd_blocks_per_page(struct inode *inode); - -#ifdef __KERNEL__ - -#define buffer_trace_init(bh) do {} while (0) -#define print_buffer_fields(bh) do {} while (0) -#define print_buffer_trace(bh) do {} while (0) -#define BUFFER_TRACE(bh, info) do {} while (0) -#define BUFFER_TRACE2(bh, bh2, info) do {} while (0) -#define JBUFFER_TRACE(jh, info) do {} while (0) - -#endif /* __KERNEL__ */ - -#endif /* _LINUX_JBD_H */ diff --git a/include/linux/jbd2.h b/include/linux/jbd2.h index edb640ae9a948b..df07e78487d560 100644 --- a/include/linux/jbd2.h +++ b/include/linux/jbd2.h @@ -29,6 +29,7 @@ #include #include #include +#include #include #endif @@ -336,7 +337,45 @@ BUFFER_FNS(Freed, freed) BUFFER_FNS(Shadow, shadow) BUFFER_FNS(Verified, verified) -#include +static inline struct buffer_head *jh2bh(struct journal_head *jh) +{ + return jh->b_bh; +} + +static inline struct journal_head *bh2jh(struct buffer_head *bh) +{ + return bh->b_private; +} + +static inline void jbd_lock_bh_state(struct buffer_head *bh) +{ + bit_spin_lock(BH_State, &bh->b_state); +} + +static inline int jbd_trylock_bh_state(struct buffer_head *bh) +{ + return bit_spin_trylock(BH_State, &bh->b_state); +} + +static inline int jbd_is_locked_bh_state(struct buffer_head *bh) +{ + return bit_spin_is_locked(BH_State, &bh->b_state); +} + +static inline void jbd_unlock_bh_state(struct buffer_head *bh) +{ + bit_spin_unlock(BH_State, &bh->b_state); +} + +static inline void jbd_lock_bh_journal_head(struct buffer_head *bh) +{ + bit_spin_lock(BH_JournalHead, &bh->b_state); +} + +static inline void jbd_unlock_bh_journal_head(struct buffer_head *bh) +{ + bit_spin_unlock(BH_JournalHead, &bh->b_state); +} #define J_ASSERT(assert) BUG_ON(!(assert)) @@ -1042,8 +1081,9 @@ void jbd2_update_log_tail(journal_t *journal, tid_t tid, unsigned long block); extern void jbd2_journal_commit_transaction(journal_t *); /* Checkpoint list management */ -void __jbd2_journal_clean_checkpoint_list(journal_t *journal); +void __jbd2_journal_clean_checkpoint_list(journal_t *journal, bool destroy); int __jbd2_journal_remove_checkpoint(struct journal_head *); +void jbd2_journal_destroy_checkpoint(journal_t *journal); void __jbd2_journal_insert_checkpoint(struct journal_head *, transaction_t *); diff --git a/include/linux/jbd_common.h b/include/linux/jbd_common.h deleted file mode 100644 index 3dc53432355f5e..00000000000000 --- a/include/linux/jbd_common.h +++ /dev/null @@ -1,46 +0,0 @@ -#ifndef _LINUX_JBD_STATE_H -#define _LINUX_JBD_STATE_H - -#include - -static inline struct buffer_head *jh2bh(struct journal_head *jh) -{ - return jh->b_bh; -} - -static inline struct journal_head *bh2jh(struct buffer_head *bh) -{ - return bh->b_private; -} - -static inline void jbd_lock_bh_state(struct buffer_head *bh) -{ - bit_spin_lock(BH_State, &bh->b_state); -} - -static inline int jbd_trylock_bh_state(struct buffer_head *bh) -{ - return bit_spin_trylock(BH_State, &bh->b_state); -} - -static inline int jbd_is_locked_bh_state(struct buffer_head *bh) -{ - return bit_spin_is_locked(BH_State, &bh->b_state); -} - -static inline void jbd_unlock_bh_state(struct buffer_head *bh) -{ - bit_spin_unlock(BH_State, &bh->b_state); -} - -static inline void jbd_lock_bh_journal_head(struct buffer_head *bh) -{ - bit_spin_lock(BH_JournalHead, &bh->b_state); -} - -static inline void jbd_unlock_bh_journal_head(struct buffer_head *bh) -{ - bit_spin_unlock(BH_JournalHead, &bh->b_state); -} - -#endif diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index f4de473f226b8a..7f653e8f669000 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -7,17 +7,52 @@ * Copyright (C) 2009-2012 Jason Baron * Copyright (C) 2011-2012 Peter Zijlstra * + * DEPRECATED API: + * + * The use of 'struct static_key' directly, is now DEPRECATED. In addition + * static_key_{true,false}() is also DEPRECATED. IE DO NOT use the following: + * + * struct static_key false = STATIC_KEY_INIT_FALSE; + * struct static_key true = STATIC_KEY_INIT_TRUE; + * static_key_true() + * static_key_false() + * + * The updated API replacements are: + * + * DEFINE_STATIC_KEY_TRUE(key); + * DEFINE_STATIC_KEY_FALSE(key); + * static_key_likely() + * statick_key_unlikely() + * * Jump labels provide an interface to generate dynamic branches using - * self-modifying code. Assuming toolchain and architecture support, the result - * of a "if (static_key_false(&key))" statement is an unconditional branch (which - * defaults to false - and the true block is placed out of line). + * self-modifying code. Assuming toolchain and architecture support, if we + * define a "key" that is initially false via "DEFINE_STATIC_KEY_FALSE(key)", + * an "if (static_branch_unlikely(&key))" statement is an unconditional branch + * (which defaults to false - and the true block is placed out of line). + * Similarly, we can define an initially true key via + * "DEFINE_STATIC_KEY_TRUE(key)", and use it in the same + * "if (static_branch_unlikely(&key))", in which case we will generate an + * unconditional branch to the out-of-line true branch. Keys that are + * initially true or false can be using in both static_branch_unlikely() + * and static_branch_likely() statements. + * + * At runtime we can change the branch target by setting the key + * to true via a call to static_branch_enable(), or false using + * static_branch_disable(). If the direction of the branch is switched by + * these calls then we run-time modify the branch target via a + * no-op -> jump or jump -> no-op conversion. For example, for an + * initially false key that is used in an "if (static_branch_unlikely(&key))" + * statement, setting the key to true requires us to patch in a jump + * to the out-of-line of true branch. * - * However at runtime we can change the branch target using - * static_key_slow_{inc,dec}(). These function as a 'reference' count on the key - * object, and for as long as there are references all branches referring to - * that particular key will point to the (out of line) true block. + * In addtion to static_branch_{enable,disable}, we can also reference count + * the key or branch direction via static_branch_{inc,dec}. Thus, + * static_branch_inc() can be thought of as a 'make more true' and + * static_branch_dec() as a 'make more false'. The inc()/dec() + * interface is meant to be used exclusively from the inc()/dec() for a given + * key. * - * Since this relies on modifying code, the static_key_slow_{inc,dec}() functions + * Since this relies on modifying code, the branch modifying functions * must be considered absolute slow paths (machine wide synchronization etc.). * OTOH, since the affected branches are unconditional, their runtime overhead * will be absolutely minimal, esp. in the default (off) case where the total @@ -29,20 +64,10 @@ * cause significant performance degradation. Struct static_key_deferred and * static_key_slow_dec_deferred() provide for this. * - * Lacking toolchain and or architecture support, jump labels fall back to a simple - * conditional branch. - * - * struct static_key my_key = STATIC_KEY_INIT_TRUE; - * - * if (static_key_true(&my_key)) { - * } + * Lacking toolchain and or architecture support, static keys fall back to a + * simple conditional branch. * - * will result in the true case being in-line and starts the key with a single - * reference. Mixing static_key_true() and static_key_false() on the same key is not - * allowed. - * - * Not initializing the key (static data is initialized to 0s anyway) is the - * same as using STATIC_KEY_INIT_FALSE. + * Additional babbling in: Documentation/static-keys.txt */ #if defined(CC_HAVE_ASM_GOTO) && defined(CONFIG_JUMP_LABEL) @@ -86,8 +111,8 @@ struct static_key { #ifndef __ASSEMBLY__ enum jump_label_type { - JUMP_LABEL_DISABLE = 0, - JUMP_LABEL_ENABLE, + JUMP_LABEL_NOP = 0, + JUMP_LABEL_JMP, }; struct module; @@ -101,33 +126,18 @@ static inline int static_key_count(struct static_key *key) #ifdef HAVE_JUMP_LABEL -#define JUMP_LABEL_TYPE_FALSE_BRANCH 0UL -#define JUMP_LABEL_TYPE_TRUE_BRANCH 1UL -#define JUMP_LABEL_TYPE_MASK 1UL - -static -inline struct jump_entry *jump_label_get_entries(struct static_key *key) -{ - return (struct jump_entry *)((unsigned long)key->entries - & ~JUMP_LABEL_TYPE_MASK); -} - -static inline bool jump_label_get_branch_default(struct static_key *key) -{ - if (((unsigned long)key->entries & JUMP_LABEL_TYPE_MASK) == - JUMP_LABEL_TYPE_TRUE_BRANCH) - return true; - return false; -} +#define JUMP_TYPE_FALSE 0UL +#define JUMP_TYPE_TRUE 1UL +#define JUMP_TYPE_MASK 1UL static __always_inline bool static_key_false(struct static_key *key) { - return arch_static_branch(key); + return arch_static_branch(key, false); } static __always_inline bool static_key_true(struct static_key *key) { - return !static_key_false(key); + return !arch_static_branch(key, true); } extern struct jump_entry __start___jump_table[]; @@ -145,12 +155,12 @@ extern void static_key_slow_inc(struct static_key *key); extern void static_key_slow_dec(struct static_key *key); extern void jump_label_apply_nops(struct module *mod); -#define STATIC_KEY_INIT_TRUE ((struct static_key) \ +#define STATIC_KEY_INIT_TRUE \ { .enabled = ATOMIC_INIT(1), \ - .entries = (void *)JUMP_LABEL_TYPE_TRUE_BRANCH }) -#define STATIC_KEY_INIT_FALSE ((struct static_key) \ + .entries = (void *)JUMP_TYPE_TRUE } +#define STATIC_KEY_INIT_FALSE \ { .enabled = ATOMIC_INIT(0), \ - .entries = (void *)JUMP_LABEL_TYPE_FALSE_BRANCH }) + .entries = (void *)JUMP_TYPE_FALSE } #else /* !HAVE_JUMP_LABEL */ @@ -198,10 +208,8 @@ static inline int jump_label_apply_nops(struct module *mod) return 0; } -#define STATIC_KEY_INIT_TRUE ((struct static_key) \ - { .enabled = ATOMIC_INIT(1) }) -#define STATIC_KEY_INIT_FALSE ((struct static_key) \ - { .enabled = ATOMIC_INIT(0) }) +#define STATIC_KEY_INIT_TRUE { .enabled = ATOMIC_INIT(1) } +#define STATIC_KEY_INIT_FALSE { .enabled = ATOMIC_INIT(0) } #endif /* HAVE_JUMP_LABEL */ @@ -213,6 +221,157 @@ static inline bool static_key_enabled(struct static_key *key) return static_key_count(key) > 0; } +static inline void static_key_enable(struct static_key *key) +{ + int count = static_key_count(key); + + WARN_ON_ONCE(count < 0 || count > 1); + + if (!count) + static_key_slow_inc(key); +} + +static inline void static_key_disable(struct static_key *key) +{ + int count = static_key_count(key); + + WARN_ON_ONCE(count < 0 || count > 1); + + if (count) + static_key_slow_dec(key); +} + +/* -------------------------------------------------------------------------- */ + +/* + * Two type wrappers around static_key, such that we can use compile time + * type differentiation to emit the right code. + * + * All the below code is macros in order to play type games. + */ + +struct static_key_true { + struct static_key key; +}; + +struct static_key_false { + struct static_key key; +}; + +#define STATIC_KEY_TRUE_INIT (struct static_key_true) { .key = STATIC_KEY_INIT_TRUE, } +#define STATIC_KEY_FALSE_INIT (struct static_key_false){ .key = STATIC_KEY_INIT_FALSE, } + +#define DEFINE_STATIC_KEY_TRUE(name) \ + struct static_key_true name = STATIC_KEY_TRUE_INIT + +#define DEFINE_STATIC_KEY_FALSE(name) \ + struct static_key_false name = STATIC_KEY_FALSE_INIT + +#ifdef HAVE_JUMP_LABEL + +/* + * Combine the right initial value (type) with the right branch order + * to generate the desired result. + * + * + * type\branch| likely (1) | unlikely (0) + * -----------+-----------------------+------------------ + * | | + * true (1) | ... | ... + * | NOP | JMP L + * | | 1: ... + * | L: ... | + * | | + * | | L: + * | | jmp 1b + * | | + * -----------+-----------------------+------------------ + * | | + * false (0) | ... | ... + * | JMP L | NOP + * | | 1: ... + * | L: ... | + * | | + * | | L: + * | | jmp 1b + * | | + * -----------+-----------------------+------------------ + * + * The initial value is encoded in the LSB of static_key::entries, + * type: 0 = false, 1 = true. + * + * The branch type is encoded in the LSB of jump_entry::key, + * branch: 0 = unlikely, 1 = likely. + * + * This gives the following logic table: + * + * enabled type branch instuction + * -----------------------------+----------- + * 0 0 0 | NOP + * 0 0 1 | JMP + * 0 1 0 | NOP + * 0 1 1 | JMP + * + * 1 0 0 | JMP + * 1 0 1 | NOP + * 1 1 0 | JMP + * 1 1 1 | NOP + * + * Which gives the following functions: + * + * dynamic: instruction = enabled ^ branch + * static: instruction = type ^ branch + * + * See jump_label_type() / jump_label_init_type(). + */ + +extern bool ____wrong_branch_error(void); + +#define static_branch_likely(x) \ +({ \ + bool branch; \ + if (__builtin_types_compatible_p(typeof(*x), struct static_key_true)) \ + branch = !arch_static_branch(&(x)->key, true); \ + else if (__builtin_types_compatible_p(typeof(*x), struct static_key_false)) \ + branch = !arch_static_branch_jump(&(x)->key, true); \ + else \ + branch = ____wrong_branch_error(); \ + branch; \ +}) + +#define static_branch_unlikely(x) \ +({ \ + bool branch; \ + if (__builtin_types_compatible_p(typeof(*x), struct static_key_true)) \ + branch = arch_static_branch_jump(&(x)->key, false); \ + else if (__builtin_types_compatible_p(typeof(*x), struct static_key_false)) \ + branch = arch_static_branch(&(x)->key, false); \ + else \ + branch = ____wrong_branch_error(); \ + branch; \ +}) + +#else /* !HAVE_JUMP_LABEL */ + +#define static_branch_likely(x) likely(static_key_enabled(&(x)->key)) +#define static_branch_unlikely(x) unlikely(static_key_enabled(&(x)->key)) + +#endif /* HAVE_JUMP_LABEL */ + +/* + * Advanced usage; refcount, branch is enabled when: count != 0 + */ + +#define static_branch_inc(x) static_key_slow_inc(&(x)->key) +#define static_branch_dec(x) static_key_slow_dec(&(x)->key) + +/* + * Normal usage; boolean enable/disable. + */ + +#define static_branch_enable(x) static_key_enable(&(x)->key) +#define static_branch_disable(x) static_key_disable(&(x)->key) + #endif /* _LINUX_JUMP_LABEL_H */ #endif /* __ASSEMBLY__ */ diff --git a/include/linux/llist.h b/include/linux/llist.h index fbf10a0bc0957c..fd4ca0b4fe0f72 100644 --- a/include/linux/llist.h +++ b/include/linux/llist.h @@ -55,8 +55,8 @@ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ +#include #include -#include struct llist_head { struct llist_node *first; diff --git a/include/linux/of_gpio.h b/include/linux/of_gpio.h index 69dbe312b11b23..f3191828f03736 100644 --- a/include/linux/of_gpio.h +++ b/include/linux/of_gpio.h @@ -54,7 +54,7 @@ extern int of_mm_gpiochip_add(struct device_node *np, struct of_mm_gpio_chip *mm_gc); extern void of_mm_gpiochip_remove(struct of_mm_gpio_chip *mm_gc); -extern void of_gpiochip_add(struct gpio_chip *gc); +extern int of_gpiochip_add(struct gpio_chip *gc); extern void of_gpiochip_remove(struct gpio_chip *gc); extern int of_gpio_simple_xlate(struct gpio_chip *gc, const struct of_phandle_args *gpiospec, @@ -76,7 +76,7 @@ static inline int of_gpio_simple_xlate(struct gpio_chip *gc, return -ENOSYS; } -static inline void of_gpiochip_add(struct gpio_chip *gc) { } +static inline int of_gpiochip_add(struct gpio_chip *gc) { return 0; } static inline void of_gpiochip_remove(struct gpio_chip *gc) { } #endif /* CONFIG_OF_GPIO */ diff --git a/arch/arm/include/asm/pmu.h b/include/linux/perf/arm_pmu.h similarity index 98% rename from arch/arm/include/asm/pmu.h rename to include/linux/perf/arm_pmu.h index 3fc87dfd77e61f..bfa673bb822d43 100644 --- a/arch/arm/include/asm/pmu.h +++ b/include/linux/perf/arm_pmu.h @@ -30,7 +30,7 @@ struct arm_pmu_platdata { irq_handler_t pmu_handler); }; -#ifdef CONFIG_HW_PERF_EVENTS +#ifdef CONFIG_ARM_PMU /* * The ARMv7 CPU PMU supports up to 32 event counters. @@ -149,6 +149,6 @@ int arm_pmu_device_probe(struct platform_device *pdev, const struct of_device_id *of_table, const struct pmu_probe_info *probe_table); -#endif /* CONFIG_HW_PERF_EVENTS */ +#endif /* CONFIG_ARM_PMU */ #endif /* __ARM_PMU_H__ */ diff --git a/include/linux/platform_data/gpio-em.h b/include/linux/platform_data/gpio-em.h deleted file mode 100644 index 7c5a519d2dcde2..00000000000000 --- a/include/linux/platform_data/gpio-em.h +++ /dev/null @@ -1,11 +0,0 @@ -#ifndef __GPIO_EM_H__ -#define __GPIO_EM_H__ - -struct gpio_em_config { - unsigned int gpio_base; - unsigned int irq_base; - unsigned int number_of_pins; - const char *pctl_name; -}; - -#endif /* __GPIO_EM_H__ */ diff --git a/include/linux/psci.h b/include/linux/psci.h new file mode 100644 index 00000000000000..a682fcc91c3321 --- /dev/null +++ b/include/linux/psci.h @@ -0,0 +1,52 @@ +/* + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * Copyright (C) 2015 ARM Limited + */ + +#ifndef __LINUX_PSCI_H +#define __LINUX_PSCI_H + +#include +#include + +#define PSCI_POWER_STATE_TYPE_STANDBY 0 +#define PSCI_POWER_STATE_TYPE_POWER_DOWN 1 + +bool psci_tos_resident_on(int cpu); + +struct psci_operations { + int (*cpu_suspend)(u32 state, unsigned long entry_point); + int (*cpu_off)(u32 state); + int (*cpu_on)(unsigned long cpuid, unsigned long entry_point); + int (*migrate)(unsigned long cpuid); + int (*affinity_info)(unsigned long target_affinity, + unsigned long lowest_affinity_level); + int (*migrate_info_type)(void); +}; + +extern struct psci_operations psci_ops; + +#if defined(CONFIG_ARM_PSCI_FW) +int __init psci_dt_init(void); +#else +static inline int psci_dt_init(void) { return 0; } +#endif + +#if defined(CONFIG_ARM_PSCI_FW) && defined(CONFIG_ACPI) +int __init psci_acpi_init(void); +bool __init acpi_psci_present(void); +bool __init acpi_psci_use_hvc(void); +#else +static inline int psci_acpi_init(void) { return 0; } +static inline bool acpi_psci_present(void) { return false; } +#endif + +#endif /* __LINUX_PSCI_H */ diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h index 77ca6601ff258b..7a57c28eb5e708 100644 --- a/include/linux/quotaops.h +++ b/include/linux/quotaops.h @@ -43,7 +43,7 @@ void inode_claim_rsv_space(struct inode *inode, qsize_t number); void inode_sub_rsv_space(struct inode *inode, qsize_t number); void inode_reclaim_rsv_space(struct inode *inode, qsize_t number); -void dquot_initialize(struct inode *inode); +int dquot_initialize(struct inode *inode); void dquot_drop(struct inode *inode); struct dquot *dqget(struct super_block *sb, struct kqid qid); static inline struct dquot *dqgrab(struct dquot *dquot) @@ -200,8 +200,9 @@ static inline int sb_has_quota_active(struct super_block *sb, int type) return 0; } -static inline void dquot_initialize(struct inode *inode) +static inline int dquot_initialize(struct inode *inode) { + return 0; } static inline void dquot_drop(struct inode *inode) diff --git a/include/misc/cxl.h b/include/misc/cxl.h index 7a6c1d6cc1732e..f2ffe5bd720dea 100644 --- a/include/misc/cxl.h +++ b/include/misc/cxl.h @@ -200,4 +200,14 @@ unsigned int cxl_fd_poll(struct file *file, struct poll_table_struct *poll); ssize_t cxl_fd_read(struct file *file, char __user *buf, size_t count, loff_t *off); +/* + * For EEH, a driver may want to assert a PERST will reload the same image + * from flash into the FPGA. + * + * This is a property of the entire adapter, not a single AFU, so drivers + * should set this property with care! + */ +void cxl_perst_reloads_same_image(struct cxl_afu *afu, + bool perst_reloads_same_image); + #endif /* _MISC_CXL_H */ diff --git a/include/trace/events/ext3.h b/include/trace/events/ext3.h deleted file mode 100644 index fc733d28117a43..00000000000000 --- a/include/trace/events/ext3.h +++ /dev/null @@ -1,866 +0,0 @@ -#undef TRACE_SYSTEM -#define TRACE_SYSTEM ext3 - -#if !defined(_TRACE_EXT3_H) || defined(TRACE_HEADER_MULTI_READ) -#define _TRACE_EXT3_H - -#include - -TRACE_EVENT(ext3_free_inode, - TP_PROTO(struct inode *inode), - - TP_ARGS(inode), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( umode_t, mode ) - __field( uid_t, uid ) - __field( gid_t, gid ) - __field( blkcnt_t, blocks ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->mode = inode->i_mode; - __entry->uid = i_uid_read(inode); - __entry->gid = i_gid_read(inode); - __entry->blocks = inode->i_blocks; - ), - - TP_printk("dev %d,%d ino %lu mode 0%o uid %u gid %u blocks %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->mode, __entry->uid, __entry->gid, - (unsigned long) __entry->blocks) -); - -TRACE_EVENT(ext3_request_inode, - TP_PROTO(struct inode *dir, int mode), - - TP_ARGS(dir, mode), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, dir ) - __field( umode_t, mode ) - ), - - TP_fast_assign( - __entry->dev = dir->i_sb->s_dev; - __entry->dir = dir->i_ino; - __entry->mode = mode; - ), - - TP_printk("dev %d,%d dir %lu mode 0%o", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->dir, __entry->mode) -); - -TRACE_EVENT(ext3_allocate_inode, - TP_PROTO(struct inode *inode, struct inode *dir, int mode), - - TP_ARGS(inode, dir, mode), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( ino_t, dir ) - __field( umode_t, mode ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->dir = dir->i_ino; - __entry->mode = mode; - ), - - TP_printk("dev %d,%d ino %lu dir %lu mode 0%o", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - (unsigned long) __entry->dir, __entry->mode) -); - -TRACE_EVENT(ext3_evict_inode, - TP_PROTO(struct inode *inode), - - TP_ARGS(inode), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( int, nlink ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->nlink = inode->i_nlink; - ), - - TP_printk("dev %d,%d ino %lu nlink %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, __entry->nlink) -); - -TRACE_EVENT(ext3_drop_inode, - TP_PROTO(struct inode *inode, int drop), - - TP_ARGS(inode, drop), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( int, drop ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->drop = drop; - ), - - TP_printk("dev %d,%d ino %lu drop %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, __entry->drop) -); - -TRACE_EVENT(ext3_mark_inode_dirty, - TP_PROTO(struct inode *inode, unsigned long IP), - - TP_ARGS(inode, IP), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field(unsigned long, ip ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->ip = IP; - ), - - TP_printk("dev %d,%d ino %lu caller %pS", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, (void *)__entry->ip) -); - -TRACE_EVENT(ext3_write_begin, - TP_PROTO(struct inode *inode, loff_t pos, unsigned int len, - unsigned int flags), - - TP_ARGS(inode, pos, len, flags), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( loff_t, pos ) - __field( unsigned int, len ) - __field( unsigned int, flags ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->pos = pos; - __entry->len = len; - __entry->flags = flags; - ), - - TP_printk("dev %d,%d ino %lu pos %llu len %u flags %u", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - (unsigned long long) __entry->pos, __entry->len, - __entry->flags) -); - -DECLARE_EVENT_CLASS(ext3__write_end, - TP_PROTO(struct inode *inode, loff_t pos, unsigned int len, - unsigned int copied), - - TP_ARGS(inode, pos, len, copied), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( loff_t, pos ) - __field( unsigned int, len ) - __field( unsigned int, copied ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->pos = pos; - __entry->len = len; - __entry->copied = copied; - ), - - TP_printk("dev %d,%d ino %lu pos %llu len %u copied %u", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - (unsigned long long) __entry->pos, __entry->len, - __entry->copied) -); - -DEFINE_EVENT(ext3__write_end, ext3_ordered_write_end, - - TP_PROTO(struct inode *inode, loff_t pos, unsigned int len, - unsigned int copied), - - TP_ARGS(inode, pos, len, copied) -); - -DEFINE_EVENT(ext3__write_end, ext3_writeback_write_end, - - TP_PROTO(struct inode *inode, loff_t pos, unsigned int len, - unsigned int copied), - - TP_ARGS(inode, pos, len, copied) -); - -DEFINE_EVENT(ext3__write_end, ext3_journalled_write_end, - - TP_PROTO(struct inode *inode, loff_t pos, unsigned int len, - unsigned int copied), - - TP_ARGS(inode, pos, len, copied) -); - -DECLARE_EVENT_CLASS(ext3__page_op, - TP_PROTO(struct page *page), - - TP_ARGS(page), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( pgoff_t, index ) - - ), - - TP_fast_assign( - __entry->index = page->index; - __entry->ino = page->mapping->host->i_ino; - __entry->dev = page->mapping->host->i_sb->s_dev; - ), - - TP_printk("dev %d,%d ino %lu page_index %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, __entry->index) -); - -DEFINE_EVENT(ext3__page_op, ext3_ordered_writepage, - - TP_PROTO(struct page *page), - - TP_ARGS(page) -); - -DEFINE_EVENT(ext3__page_op, ext3_writeback_writepage, - - TP_PROTO(struct page *page), - - TP_ARGS(page) -); - -DEFINE_EVENT(ext3__page_op, ext3_journalled_writepage, - - TP_PROTO(struct page *page), - - TP_ARGS(page) -); - -DEFINE_EVENT(ext3__page_op, ext3_readpage, - - TP_PROTO(struct page *page), - - TP_ARGS(page) -); - -DEFINE_EVENT(ext3__page_op, ext3_releasepage, - - TP_PROTO(struct page *page), - - TP_ARGS(page) -); - -TRACE_EVENT(ext3_invalidatepage, - TP_PROTO(struct page *page, unsigned int offset, unsigned int length), - - TP_ARGS(page, offset, length), - - TP_STRUCT__entry( - __field( pgoff_t, index ) - __field( unsigned int, offset ) - __field( unsigned int, length ) - __field( ino_t, ino ) - __field( dev_t, dev ) - - ), - - TP_fast_assign( - __entry->index = page->index; - __entry->offset = offset; - __entry->length = length; - __entry->ino = page->mapping->host->i_ino; - __entry->dev = page->mapping->host->i_sb->s_dev; - ), - - TP_printk("dev %d,%d ino %lu page_index %lu offset %u length %u", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->index, __entry->offset, __entry->length) -); - -TRACE_EVENT(ext3_discard_blocks, - TP_PROTO(struct super_block *sb, unsigned long blk, - unsigned long count), - - TP_ARGS(sb, blk, count), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( unsigned long, blk ) - __field( unsigned long, count ) - - ), - - TP_fast_assign( - __entry->dev = sb->s_dev; - __entry->blk = blk; - __entry->count = count; - ), - - TP_printk("dev %d,%d blk %lu count %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->blk, __entry->count) -); - -TRACE_EVENT(ext3_request_blocks, - TP_PROTO(struct inode *inode, unsigned long goal, - unsigned long count), - - TP_ARGS(inode, goal, count), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( unsigned long, count ) - __field( unsigned long, goal ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->count = count; - __entry->goal = goal; - ), - - TP_printk("dev %d,%d ino %lu count %lu goal %lu ", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->count, __entry->goal) -); - -TRACE_EVENT(ext3_allocate_blocks, - TP_PROTO(struct inode *inode, unsigned long goal, - unsigned long count, unsigned long block), - - TP_ARGS(inode, goal, count, block), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( unsigned long, block ) - __field( unsigned long, count ) - __field( unsigned long, goal ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->block = block; - __entry->count = count; - __entry->goal = goal; - ), - - TP_printk("dev %d,%d ino %lu count %lu block %lu goal %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->count, __entry->block, - __entry->goal) -); - -TRACE_EVENT(ext3_free_blocks, - TP_PROTO(struct inode *inode, unsigned long block, - unsigned long count), - - TP_ARGS(inode, block, count), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( umode_t, mode ) - __field( unsigned long, block ) - __field( unsigned long, count ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->mode = inode->i_mode; - __entry->block = block; - __entry->count = count; - ), - - TP_printk("dev %d,%d ino %lu mode 0%o block %lu count %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->mode, __entry->block, __entry->count) -); - -TRACE_EVENT(ext3_sync_file_enter, - TP_PROTO(struct file *file, int datasync), - - TP_ARGS(file, datasync), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( ino_t, parent ) - __field( int, datasync ) - ), - - TP_fast_assign( - struct dentry *dentry = file->f_path.dentry; - - __entry->dev = d_inode(dentry)->i_sb->s_dev; - __entry->ino = d_inode(dentry)->i_ino; - __entry->datasync = datasync; - __entry->parent = d_inode(dentry->d_parent)->i_ino; - ), - - TP_printk("dev %d,%d ino %lu parent %ld datasync %d ", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - (unsigned long) __entry->parent, __entry->datasync) -); - -TRACE_EVENT(ext3_sync_file_exit, - TP_PROTO(struct inode *inode, int ret), - - TP_ARGS(inode, ret), - - TP_STRUCT__entry( - __field( int, ret ) - __field( ino_t, ino ) - __field( dev_t, dev ) - ), - - TP_fast_assign( - __entry->ret = ret; - __entry->ino = inode->i_ino; - __entry->dev = inode->i_sb->s_dev; - ), - - TP_printk("dev %d,%d ino %lu ret %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->ret) -); - -TRACE_EVENT(ext3_sync_fs, - TP_PROTO(struct super_block *sb, int wait), - - TP_ARGS(sb, wait), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( int, wait ) - - ), - - TP_fast_assign( - __entry->dev = sb->s_dev; - __entry->wait = wait; - ), - - TP_printk("dev %d,%d wait %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->wait) -); - -TRACE_EVENT(ext3_rsv_window_add, - TP_PROTO(struct super_block *sb, - struct ext3_reserve_window_node *rsv_node), - - TP_ARGS(sb, rsv_node), - - TP_STRUCT__entry( - __field( unsigned long, start ) - __field( unsigned long, end ) - __field( dev_t, dev ) - ), - - TP_fast_assign( - __entry->dev = sb->s_dev; - __entry->start = rsv_node->rsv_window._rsv_start; - __entry->end = rsv_node->rsv_window._rsv_end; - ), - - TP_printk("dev %d,%d start %lu end %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->start, __entry->end) -); - -TRACE_EVENT(ext3_discard_reservation, - TP_PROTO(struct inode *inode, - struct ext3_reserve_window_node *rsv_node), - - TP_ARGS(inode, rsv_node), - - TP_STRUCT__entry( - __field( unsigned long, start ) - __field( unsigned long, end ) - __field( ino_t, ino ) - __field( dev_t, dev ) - ), - - TP_fast_assign( - __entry->start = rsv_node->rsv_window._rsv_start; - __entry->end = rsv_node->rsv_window._rsv_end; - __entry->ino = inode->i_ino; - __entry->dev = inode->i_sb->s_dev; - ), - - TP_printk("dev %d,%d ino %lu start %lu end %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long)__entry->ino, __entry->start, - __entry->end) -); - -TRACE_EVENT(ext3_alloc_new_reservation, - TP_PROTO(struct super_block *sb, unsigned long goal), - - TP_ARGS(sb, goal), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( unsigned long, goal ) - ), - - TP_fast_assign( - __entry->dev = sb->s_dev; - __entry->goal = goal; - ), - - TP_printk("dev %d,%d goal %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->goal) -); - -TRACE_EVENT(ext3_reserved, - TP_PROTO(struct super_block *sb, unsigned long block, - struct ext3_reserve_window_node *rsv_node), - - TP_ARGS(sb, block, rsv_node), - - TP_STRUCT__entry( - __field( unsigned long, block ) - __field( unsigned long, start ) - __field( unsigned long, end ) - __field( dev_t, dev ) - ), - - TP_fast_assign( - __entry->block = block; - __entry->start = rsv_node->rsv_window._rsv_start; - __entry->end = rsv_node->rsv_window._rsv_end; - __entry->dev = sb->s_dev; - ), - - TP_printk("dev %d,%d block %lu, start %lu end %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->block, __entry->start, __entry->end) -); - -TRACE_EVENT(ext3_forget, - TP_PROTO(struct inode *inode, int is_metadata, unsigned long block), - - TP_ARGS(inode, is_metadata, block), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( ino_t, ino ) - __field( umode_t, mode ) - __field( int, is_metadata ) - __field( unsigned long, block ) - ), - - TP_fast_assign( - __entry->dev = inode->i_sb->s_dev; - __entry->ino = inode->i_ino; - __entry->mode = inode->i_mode; - __entry->is_metadata = is_metadata; - __entry->block = block; - ), - - TP_printk("dev %d,%d ino %lu mode 0%o is_metadata %d block %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->mode, __entry->is_metadata, __entry->block) -); - -TRACE_EVENT(ext3_read_block_bitmap, - TP_PROTO(struct super_block *sb, unsigned int group), - - TP_ARGS(sb, group), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( __u32, group ) - - ), - - TP_fast_assign( - __entry->dev = sb->s_dev; - __entry->group = group; - ), - - TP_printk("dev %d,%d group %u", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->group) -); - -TRACE_EVENT(ext3_direct_IO_enter, - TP_PROTO(struct inode *inode, loff_t offset, unsigned long len, int rw), - - TP_ARGS(inode, offset, len, rw), - - TP_STRUCT__entry( - __field( ino_t, ino ) - __field( dev_t, dev ) - __field( loff_t, pos ) - __field( unsigned long, len ) - __field( int, rw ) - ), - - TP_fast_assign( - __entry->ino = inode->i_ino; - __entry->dev = inode->i_sb->s_dev; - __entry->pos = offset; - __entry->len = len; - __entry->rw = rw; - ), - - TP_printk("dev %d,%d ino %lu pos %llu len %lu rw %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - (unsigned long long) __entry->pos, __entry->len, - __entry->rw) -); - -TRACE_EVENT(ext3_direct_IO_exit, - TP_PROTO(struct inode *inode, loff_t offset, unsigned long len, - int rw, int ret), - - TP_ARGS(inode, offset, len, rw, ret), - - TP_STRUCT__entry( - __field( ino_t, ino ) - __field( dev_t, dev ) - __field( loff_t, pos ) - __field( unsigned long, len ) - __field( int, rw ) - __field( int, ret ) - ), - - TP_fast_assign( - __entry->ino = inode->i_ino; - __entry->dev = inode->i_sb->s_dev; - __entry->pos = offset; - __entry->len = len; - __entry->rw = rw; - __entry->ret = ret; - ), - - TP_printk("dev %d,%d ino %lu pos %llu len %lu rw %d ret %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - (unsigned long long) __entry->pos, __entry->len, - __entry->rw, __entry->ret) -); - -TRACE_EVENT(ext3_unlink_enter, - TP_PROTO(struct inode *parent, struct dentry *dentry), - - TP_ARGS(parent, dentry), - - TP_STRUCT__entry( - __field( ino_t, parent ) - __field( ino_t, ino ) - __field( loff_t, size ) - __field( dev_t, dev ) - ), - - TP_fast_assign( - __entry->parent = parent->i_ino; - __entry->ino = d_inode(dentry)->i_ino; - __entry->size = d_inode(dentry)->i_size; - __entry->dev = d_inode(dentry)->i_sb->s_dev; - ), - - TP_printk("dev %d,%d ino %lu size %lld parent %ld", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - (unsigned long long)__entry->size, - (unsigned long) __entry->parent) -); - -TRACE_EVENT(ext3_unlink_exit, - TP_PROTO(struct dentry *dentry, int ret), - - TP_ARGS(dentry, ret), - - TP_STRUCT__entry( - __field( ino_t, ino ) - __field( dev_t, dev ) - __field( int, ret ) - ), - - TP_fast_assign( - __entry->ino = d_inode(dentry)->i_ino; - __entry->dev = d_inode(dentry)->i_sb->s_dev; - __entry->ret = ret; - ), - - TP_printk("dev %d,%d ino %lu ret %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->ret) -); - -DECLARE_EVENT_CLASS(ext3__truncate, - TP_PROTO(struct inode *inode), - - TP_ARGS(inode), - - TP_STRUCT__entry( - __field( ino_t, ino ) - __field( dev_t, dev ) - __field( blkcnt_t, blocks ) - ), - - TP_fast_assign( - __entry->ino = inode->i_ino; - __entry->dev = inode->i_sb->s_dev; - __entry->blocks = inode->i_blocks; - ), - - TP_printk("dev %d,%d ino %lu blocks %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, (unsigned long) __entry->blocks) -); - -DEFINE_EVENT(ext3__truncate, ext3_truncate_enter, - - TP_PROTO(struct inode *inode), - - TP_ARGS(inode) -); - -DEFINE_EVENT(ext3__truncate, ext3_truncate_exit, - - TP_PROTO(struct inode *inode), - - TP_ARGS(inode) -); - -TRACE_EVENT(ext3_get_blocks_enter, - TP_PROTO(struct inode *inode, unsigned long lblk, - unsigned long len, int create), - - TP_ARGS(inode, lblk, len, create), - - TP_STRUCT__entry( - __field( ino_t, ino ) - __field( dev_t, dev ) - __field( unsigned long, lblk ) - __field( unsigned long, len ) - __field( int, create ) - ), - - TP_fast_assign( - __entry->ino = inode->i_ino; - __entry->dev = inode->i_sb->s_dev; - __entry->lblk = lblk; - __entry->len = len; - __entry->create = create; - ), - - TP_printk("dev %d,%d ino %lu lblk %lu len %lu create %u", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->lblk, __entry->len, __entry->create) -); - -TRACE_EVENT(ext3_get_blocks_exit, - TP_PROTO(struct inode *inode, unsigned long lblk, - unsigned long pblk, unsigned long len, int ret), - - TP_ARGS(inode, lblk, pblk, len, ret), - - TP_STRUCT__entry( - __field( ino_t, ino ) - __field( dev_t, dev ) - __field( unsigned long, lblk ) - __field( unsigned long, pblk ) - __field( unsigned long, len ) - __field( int, ret ) - ), - - TP_fast_assign( - __entry->ino = inode->i_ino; - __entry->dev = inode->i_sb->s_dev; - __entry->lblk = lblk; - __entry->pblk = pblk; - __entry->len = len; - __entry->ret = ret; - ), - - TP_printk("dev %d,%d ino %lu lblk %lu pblk %lu len %lu ret %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino, - __entry->lblk, __entry->pblk, - __entry->len, __entry->ret) -); - -TRACE_EVENT(ext3_load_inode, - TP_PROTO(struct inode *inode), - - TP_ARGS(inode), - - TP_STRUCT__entry( - __field( ino_t, ino ) - __field( dev_t, dev ) - ), - - TP_fast_assign( - __entry->ino = inode->i_ino; - __entry->dev = inode->i_sb->s_dev; - ), - - TP_printk("dev %d,%d ino %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - (unsigned long) __entry->ino) -); - -#endif /* _TRACE_EXT3_H */ - -/* This part must be outside protection */ -#include diff --git a/include/trace/events/f2fs.h b/include/trace/events/f2fs.h index 04856a2d8c82d7..a01946514b5ac5 100644 --- a/include/trace/events/f2fs.h +++ b/include/trace/events/f2fs.h @@ -1099,11 +1099,11 @@ TRACE_EVENT(f2fs_lookup_extent_tree_start, TRACE_EVENT_CONDITION(f2fs_lookup_extent_tree_end, TP_PROTO(struct inode *inode, unsigned int pgofs, - struct extent_node *en), + struct extent_info *ei), - TP_ARGS(inode, pgofs, en), + TP_ARGS(inode, pgofs, ei), - TP_CONDITION(en), + TP_CONDITION(ei), TP_STRUCT__entry( __field(dev_t, dev) @@ -1118,9 +1118,9 @@ TRACE_EVENT_CONDITION(f2fs_lookup_extent_tree_end, __entry->dev = inode->i_sb->s_dev; __entry->ino = inode->i_ino; __entry->pgofs = pgofs; - __entry->fofs = en->ei.fofs; - __entry->blk = en->ei.blk; - __entry->len = en->ei.len; + __entry->fofs = ei->fofs; + __entry->blk = ei->blk; + __entry->len = ei->len; ), TP_printk("dev = (%d,%d), ino = %lu, pgofs = %u, " diff --git a/include/trace/events/jbd.h b/include/trace/events/jbd.h deleted file mode 100644 index da6f2591c25e1a..00000000000000 --- a/include/trace/events/jbd.h +++ /dev/null @@ -1,194 +0,0 @@ -#undef TRACE_SYSTEM -#define TRACE_SYSTEM jbd - -#if !defined(_TRACE_JBD_H) || defined(TRACE_HEADER_MULTI_READ) -#define _TRACE_JBD_H - -#include -#include - -TRACE_EVENT(jbd_checkpoint, - - TP_PROTO(journal_t *journal, int result), - - TP_ARGS(journal, result), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( int, result ) - ), - - TP_fast_assign( - __entry->dev = journal->j_fs_dev->bd_dev; - __entry->result = result; - ), - - TP_printk("dev %d,%d result %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->result) -); - -DECLARE_EVENT_CLASS(jbd_commit, - - TP_PROTO(journal_t *journal, transaction_t *commit_transaction), - - TP_ARGS(journal, commit_transaction), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( int, transaction ) - ), - - TP_fast_assign( - __entry->dev = journal->j_fs_dev->bd_dev; - __entry->transaction = commit_transaction->t_tid; - ), - - TP_printk("dev %d,%d transaction %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->transaction) -); - -DEFINE_EVENT(jbd_commit, jbd_start_commit, - - TP_PROTO(journal_t *journal, transaction_t *commit_transaction), - - TP_ARGS(journal, commit_transaction) -); - -DEFINE_EVENT(jbd_commit, jbd_commit_locking, - - TP_PROTO(journal_t *journal, transaction_t *commit_transaction), - - TP_ARGS(journal, commit_transaction) -); - -DEFINE_EVENT(jbd_commit, jbd_commit_flushing, - - TP_PROTO(journal_t *journal, transaction_t *commit_transaction), - - TP_ARGS(journal, commit_transaction) -); - -DEFINE_EVENT(jbd_commit, jbd_commit_logging, - - TP_PROTO(journal_t *journal, transaction_t *commit_transaction), - - TP_ARGS(journal, commit_transaction) -); - -TRACE_EVENT(jbd_drop_transaction, - - TP_PROTO(journal_t *journal, transaction_t *commit_transaction), - - TP_ARGS(journal, commit_transaction), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( int, transaction ) - ), - - TP_fast_assign( - __entry->dev = journal->j_fs_dev->bd_dev; - __entry->transaction = commit_transaction->t_tid; - ), - - TP_printk("dev %d,%d transaction %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->transaction) -); - -TRACE_EVENT(jbd_end_commit, - TP_PROTO(journal_t *journal, transaction_t *commit_transaction), - - TP_ARGS(journal, commit_transaction), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( int, transaction ) - __field( int, head ) - ), - - TP_fast_assign( - __entry->dev = journal->j_fs_dev->bd_dev; - __entry->transaction = commit_transaction->t_tid; - __entry->head = journal->j_tail_sequence; - ), - - TP_printk("dev %d,%d transaction %d head %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->transaction, __entry->head) -); - -TRACE_EVENT(jbd_do_submit_data, - TP_PROTO(journal_t *journal, transaction_t *commit_transaction), - - TP_ARGS(journal, commit_transaction), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( int, transaction ) - ), - - TP_fast_assign( - __entry->dev = journal->j_fs_dev->bd_dev; - __entry->transaction = commit_transaction->t_tid; - ), - - TP_printk("dev %d,%d transaction %d", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->transaction) -); - -TRACE_EVENT(jbd_cleanup_journal_tail, - - TP_PROTO(journal_t *journal, tid_t first_tid, - unsigned long block_nr, unsigned long freed), - - TP_ARGS(journal, first_tid, block_nr, freed), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( tid_t, tail_sequence ) - __field( tid_t, first_tid ) - __field(unsigned long, block_nr ) - __field(unsigned long, freed ) - ), - - TP_fast_assign( - __entry->dev = journal->j_fs_dev->bd_dev; - __entry->tail_sequence = journal->j_tail_sequence; - __entry->first_tid = first_tid; - __entry->block_nr = block_nr; - __entry->freed = freed; - ), - - TP_printk("dev %d,%d from %u to %u offset %lu freed %lu", - MAJOR(__entry->dev), MINOR(__entry->dev), - __entry->tail_sequence, __entry->first_tid, - __entry->block_nr, __entry->freed) -); - -TRACE_EVENT(journal_write_superblock, - TP_PROTO(journal_t *journal, int write_op), - - TP_ARGS(journal, write_op), - - TP_STRUCT__entry( - __field( dev_t, dev ) - __field( int, write_op ) - ), - - TP_fast_assign( - __entry->dev = journal->j_fs_dev->bd_dev; - __entry->write_op = write_op; - ), - - TP_printk("dev %d,%d write_op %x", MAJOR(__entry->dev), - MINOR(__entry->dev), __entry->write_op) -); - -#endif /* _TRACE_JBD_H */ - -/* This part must be outside protection */ -#include diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h index d3475e1f15ec19..1f977dd4c370a8 100644 --- a/include/uapi/linux/audit.h +++ b/include/uapi/linux/audit.h @@ -382,6 +382,9 @@ enum { #define AUDIT_ARCH_SHEL64 (EM_SH|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE) #define AUDIT_ARCH_SPARC (EM_SPARC) #define AUDIT_ARCH_SPARC64 (EM_SPARCV9|__AUDIT_ARCH_64BIT) +#define AUDIT_ARCH_TILEGX (EM_TILEGX|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE) +#define AUDIT_ARCH_TILEGX32 (EM_TILEGX|__AUDIT_ARCH_LE) +#define AUDIT_ARCH_TILEPRO (EM_TILEPRO|__AUDIT_ARCH_LE) #define AUDIT_ARCH_X86_64 (EM_X86_64|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE) #define AUDIT_PERM_EXEC 1 diff --git a/include/uapi/linux/dlm_device.h b/include/uapi/linux/dlm_device.h index 3060783c419154..df56c8ff076986 100644 --- a/include/uapi/linux/dlm_device.h +++ b/include/uapi/linux/dlm_device.h @@ -26,7 +26,7 @@ /* Version of the device interface */ #define DLM_DEVICE_VERSION_MAJOR 6 #define DLM_DEVICE_VERSION_MINOR 0 -#define DLM_DEVICE_VERSION_PATCH 1 +#define DLM_DEVICE_VERSION_PATCH 2 /* struct passed to the lock write */ struct dlm_lock_params { diff --git a/include/uapi/linux/elf-em.h b/include/uapi/linux/elf-em.h index b08829667ed75d..3429a3ba382b64 100644 --- a/include/uapi/linux/elf-em.h +++ b/include/uapi/linux/elf-em.h @@ -38,6 +38,8 @@ #define EM_ALTERA_NIOS2 113 /* Altera Nios II soft-core processor */ #define EM_TI_C6000 140 /* TI C6X DSPs */ #define EM_AARCH64 183 /* ARM 64 bit */ +#define EM_TILEPRO 188 /* Tilera TILEPro */ +#define EM_TILEGX 191 /* Tilera TILE-Gx */ #define EM_FRV 0x5441 /* Fujitsu FR-V */ #define EM_AVR32 0x18ad /* Atmel AVR32 */ diff --git a/include/uapi/misc/cxl.h b/include/uapi/misc/cxl.h index 99a8ca15fe648c..1e889aa8a36e8c 100644 --- a/include/uapi/misc/cxl.h +++ b/include/uapi/misc/cxl.h @@ -29,8 +29,10 @@ struct cxl_ioctl_start_work { #define CXL_START_WORK_AMR 0x0000000000000001ULL #define CXL_START_WORK_NUM_IRQS 0x0000000000000002ULL +#define CXL_START_WORK_ERR_FF 0x0000000000000004ULL #define CXL_START_WORK_ALL (CXL_START_WORK_AMR |\ - CXL_START_WORK_NUM_IRQS) + CXL_START_WORK_NUM_IRQS |\ + CXL_START_WORK_ERR_FF) /* Possible modes that an afu can be in */ diff --git a/kernel/futex.c b/kernel/futex.c index c4a182f5357eb4..6e443efc65f41e 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -64,6 +64,7 @@ #include #include #include +#include #include @@ -258,6 +259,66 @@ static unsigned long __read_mostly futex_hashsize; static struct futex_hash_bucket *futex_queues; +/* + * Fault injections for futexes. + */ +#ifdef CONFIG_FAIL_FUTEX + +static struct { + struct fault_attr attr; + + u32 ignore_private; +} fail_futex = { + .attr = FAULT_ATTR_INITIALIZER, + .ignore_private = 0, +}; + +static int __init setup_fail_futex(char *str) +{ + return setup_fault_attr(&fail_futex.attr, str); +} +__setup("fail_futex=", setup_fail_futex); + +static bool should_fail_futex(bool fshared) +{ + if (fail_futex.ignore_private && !fshared) + return false; + + return should_fail(&fail_futex.attr, 1); +} + +#ifdef CONFIG_FAULT_INJECTION_DEBUG_FS + +static int __init fail_futex_debugfs(void) +{ + umode_t mode = S_IFREG | S_IRUSR | S_IWUSR; + struct dentry *dir; + + dir = fault_create_debugfs_attr("fail_futex", NULL, + &fail_futex.attr); + if (IS_ERR(dir)) + return PTR_ERR(dir); + + if (!debugfs_create_bool("ignore-private", mode, dir, + &fail_futex.ignore_private)) { + debugfs_remove_recursive(dir); + return -ENOMEM; + } + + return 0; +} + +late_initcall(fail_futex_debugfs); + +#endif /* CONFIG_FAULT_INJECTION_DEBUG_FS */ + +#else +static inline bool should_fail_futex(bool fshared) +{ + return false; +} +#endif /* CONFIG_FAIL_FUTEX */ + static inline void futex_get_mm(union futex_key *key) { atomic_inc(&key->private.mm->mm_count); @@ -413,6 +474,9 @@ get_futex_key(u32 __user *uaddr, int fshared, union futex_key *key, int rw) if (unlikely(!access_ok(rw, uaddr, sizeof(u32)))) return -EFAULT; + if (unlikely(should_fail_futex(fshared))) + return -EFAULT; + /* * PROCESS_PRIVATE futexes are fast. * As the mm cannot disappear under us and the 'key' only needs @@ -428,6 +492,10 @@ get_futex_key(u32 __user *uaddr, int fshared, union futex_key *key, int rw) } again: + /* Ignore any VERIFY_READ mapping (futex common case) */ + if (unlikely(should_fail_futex(fshared))) + return -EFAULT; + err = get_user_pages_fast(address, 1, 1, &page); /* * If write access is not required (eg. FUTEX_WAIT), try @@ -516,7 +584,7 @@ get_futex_key(u32 __user *uaddr, int fshared, union futex_key *key, int rw) * A RO anonymous page will never change and thus doesn't make * sense for futex operations. */ - if (ro) { + if (unlikely(should_fail_futex(fshared)) || ro) { err = -EFAULT; goto out; } @@ -974,6 +1042,9 @@ static int lock_pi_update_atomic(u32 __user *uaddr, u32 uval, u32 newval) { u32 uninitialized_var(curval); + if (unlikely(should_fail_futex(true))) + return -EFAULT; + if (unlikely(cmpxchg_futex_value_locked(&curval, uaddr, uval, newval))) return -EFAULT; @@ -1015,12 +1086,18 @@ static int futex_lock_pi_atomic(u32 __user *uaddr, struct futex_hash_bucket *hb, if (get_futex_value_locked(&uval, uaddr)) return -EFAULT; + if (unlikely(should_fail_futex(true))) + return -EFAULT; + /* * Detect deadlocks. */ if ((unlikely((uval & FUTEX_TID_MASK) == vpid))) return -EDEADLK; + if ((unlikely(should_fail_futex(true)))) + return -EDEADLK; + /* * Lookup existing state first. If it exists, try to attach to * its pi_state. @@ -1155,6 +1232,9 @@ static int wake_futex_pi(u32 __user *uaddr, u32 uval, struct futex_q *this, */ newval = FUTEX_WAITERS | task_pid_vnr(new_owner); + if (unlikely(should_fail_futex(true))) + ret = -EFAULT; + if (cmpxchg_futex_value_locked(&curval, uaddr, uval, newval)) ret = -EFAULT; else if (curval != uval) @@ -1457,6 +1537,9 @@ static int futex_proxy_trylock_atomic(u32 __user *pifutex, if (get_futex_value_locked(&curval, pifutex)) return -EFAULT; + if (unlikely(should_fail_futex(true))) + return -EFAULT; + /* * Find the top_waiter and determine if there are additional waiters. * If the caller intends to requeue more than 1 waiter to pifutex, @@ -2268,8 +2351,11 @@ static long futex_wait_restart(struct restart_block *restart) /* * Userspace tried a 0 -> TID atomic transition of the futex value * and failed. The kernel side here does the whole locking operation: - * if there are waiters then it will block, it does PI, etc. (Due to - * races the kernel might see a 0 value of the futex too.) + * if there are waiters then it will block as a consequence of relying + * on rt-mutexes, it does PI, etc. (Due to races the kernel might see + * a 0 value of the futex too.). + * + * Also serves as futex trylock_pi()'ing, and due semantics. */ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, ktime_t *time, int trylock) @@ -2300,6 +2386,10 @@ static int futex_lock_pi(u32 __user *uaddr, unsigned int flags, ret = futex_lock_pi_atomic(uaddr, hb, &q.key, &q.pi_state, current, 0); if (unlikely(ret)) { + /* + * Atomic work succeeded and we got the lock, + * or failed. Either way, we do _not_ block. + */ switch (ret) { case 1: /* We got the lock. */ @@ -2530,7 +2620,7 @@ int handle_early_requeue_pi_wakeup(struct futex_hash_bucket *hb, * futex_wait_requeue_pi() - Wait on uaddr and take uaddr2 * @uaddr: the futex we initially wait on (non-pi) * @flags: futex flags (FLAGS_SHARED, FLAGS_CLOCKRT, etc.), they must be - * the same type, no requeueing from private to shared, etc. + * the same type, no requeueing from private to shared, etc. * @val: the expected value of uaddr * @abs_time: absolute timeout * @bitset: 32 bit wakeup bitset set by userspace, defaults to all @@ -3005,6 +3095,8 @@ SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val, if (utime && (cmd == FUTEX_WAIT || cmd == FUTEX_LOCK_PI || cmd == FUTEX_WAIT_BITSET || cmd == FUTEX_WAIT_REQUEUE_PI)) { + if (unlikely(should_fail_futex(!(op & FUTEX_PRIVATE_FLAG)))) + return -EFAULT; if (copy_from_user(&ts, utime, sizeof(ts)) != 0) return -EFAULT; if (!timespec_valid(&ts)) diff --git a/kernel/jump_label.c b/kernel/jump_label.c index 52ebaca1b9fc16..f7dd15d537f9b1 100644 --- a/kernel/jump_label.c +++ b/kernel/jump_label.c @@ -54,7 +54,7 @@ jump_label_sort_entries(struct jump_entry *start, struct jump_entry *stop) sort(start, size, sizeof(struct jump_entry), jump_label_cmp, NULL); } -static void jump_label_update(struct static_key *key, int enable); +static void jump_label_update(struct static_key *key); void static_key_slow_inc(struct static_key *key) { @@ -63,13 +63,8 @@ void static_key_slow_inc(struct static_key *key) return; jump_label_lock(); - if (atomic_read(&key->enabled) == 0) { - if (!jump_label_get_branch_default(key)) - jump_label_update(key, JUMP_LABEL_ENABLE); - else - jump_label_update(key, JUMP_LABEL_DISABLE); - } - atomic_inc(&key->enabled); + if (atomic_inc_return(&key->enabled) == 1) + jump_label_update(key); jump_label_unlock(); } EXPORT_SYMBOL_GPL(static_key_slow_inc); @@ -87,10 +82,7 @@ static void __static_key_slow_dec(struct static_key *key, atomic_inc(&key->enabled); schedule_delayed_work(work, rate_limit); } else { - if (!jump_label_get_branch_default(key)) - jump_label_update(key, JUMP_LABEL_DISABLE); - else - jump_label_update(key, JUMP_LABEL_ENABLE); + jump_label_update(key); } jump_label_unlock(); } @@ -149,7 +141,7 @@ static int __jump_label_text_reserved(struct jump_entry *iter_start, return 0; } -/* +/* * Update code which is definitely not currently executing. * Architectures which need heavyweight synchronization to modify * running code can override this to make the non-live update case @@ -158,37 +150,54 @@ static int __jump_label_text_reserved(struct jump_entry *iter_start, void __weak __init_or_module arch_jump_label_transform_static(struct jump_entry *entry, enum jump_label_type type) { - arch_jump_label_transform(entry, type); + arch_jump_label_transform(entry, type); +} + +static inline struct jump_entry *static_key_entries(struct static_key *key) +{ + return (struct jump_entry *)((unsigned long)key->entries & ~JUMP_TYPE_MASK); +} + +static inline bool static_key_type(struct static_key *key) +{ + return (unsigned long)key->entries & JUMP_TYPE_MASK; +} + +static inline struct static_key *jump_entry_key(struct jump_entry *entry) +{ + return (struct static_key *)((unsigned long)entry->key & ~1UL); +} + +static bool jump_entry_branch(struct jump_entry *entry) +{ + return (unsigned long)entry->key & 1UL; +} + +static enum jump_label_type jump_label_type(struct jump_entry *entry) +{ + struct static_key *key = jump_entry_key(entry); + bool enabled = static_key_enabled(key); + bool branch = jump_entry_branch(entry); + + /* See the comment in linux/jump_label.h */ + return enabled ^ branch; } static void __jump_label_update(struct static_key *key, struct jump_entry *entry, - struct jump_entry *stop, int enable) + struct jump_entry *stop) { - for (; (entry < stop) && - (entry->key == (jump_label_t)(unsigned long)key); - entry++) { + for (; (entry < stop) && (jump_entry_key(entry) == key); entry++) { /* * entry->code set to 0 invalidates module init text sections * kernel_text_address() verifies we are not in core kernel * init code, see jump_label_invalidate_module_init(). */ if (entry->code && kernel_text_address(entry->code)) - arch_jump_label_transform(entry, enable); + arch_jump_label_transform(entry, jump_label_type(entry)); } } -static enum jump_label_type jump_label_type(struct static_key *key) -{ - bool true_branch = jump_label_get_branch_default(key); - bool state = static_key_enabled(key); - - if ((!true_branch && state) || (true_branch && !state)) - return JUMP_LABEL_ENABLE; - - return JUMP_LABEL_DISABLE; -} - void __init jump_label_init(void) { struct jump_entry *iter_start = __start___jump_table; @@ -202,8 +211,11 @@ void __init jump_label_init(void) for (iter = iter_start; iter < iter_stop; iter++) { struct static_key *iterk; - iterk = (struct static_key *)(unsigned long)iter->key; - arch_jump_label_transform_static(iter, jump_label_type(iterk)); + /* rewrite NOPs */ + if (jump_label_type(iter) == JUMP_LABEL_NOP) + arch_jump_label_transform_static(iter, JUMP_LABEL_NOP); + + iterk = jump_entry_key(iter); if (iterk == key) continue; @@ -222,6 +234,16 @@ void __init jump_label_init(void) #ifdef CONFIG_MODULES +static enum jump_label_type jump_label_init_type(struct jump_entry *entry) +{ + struct static_key *key = jump_entry_key(entry); + bool type = static_key_type(key); + bool branch = jump_entry_branch(entry); + + /* See the comment in linux/jump_label.h */ + return type ^ branch; +} + struct static_key_mod { struct static_key_mod *next; struct jump_entry *entries; @@ -243,17 +265,15 @@ static int __jump_label_mod_text_reserved(void *start, void *end) start, end); } -static void __jump_label_mod_update(struct static_key *key, int enable) +static void __jump_label_mod_update(struct static_key *key) { - struct static_key_mod *mod = key->next; + struct static_key_mod *mod; - while (mod) { + for (mod = key->next; mod; mod = mod->next) { struct module *m = mod->mod; __jump_label_update(key, mod->entries, - m->jump_entries + m->num_jump_entries, - enable); - mod = mod->next; + m->jump_entries + m->num_jump_entries); } } @@ -276,7 +296,9 @@ void jump_label_apply_nops(struct module *mod) return; for (iter = iter_start; iter < iter_stop; iter++) { - arch_jump_label_transform_static(iter, JUMP_LABEL_DISABLE); + /* Only write NOPs for arch_branch_static(). */ + if (jump_label_init_type(iter) == JUMP_LABEL_NOP) + arch_jump_label_transform_static(iter, JUMP_LABEL_NOP); } } @@ -297,7 +319,7 @@ static int jump_label_add_module(struct module *mod) for (iter = iter_start; iter < iter_stop; iter++) { struct static_key *iterk; - iterk = (struct static_key *)(unsigned long)iter->key; + iterk = jump_entry_key(iter); if (iterk == key) continue; @@ -318,8 +340,9 @@ static int jump_label_add_module(struct module *mod) jlm->next = key->next; key->next = jlm; - if (jump_label_type(key) == JUMP_LABEL_ENABLE) - __jump_label_update(key, iter, iter_stop, JUMP_LABEL_ENABLE); + /* Only update if we've changed from our initial state */ + if (jump_label_type(iter) != jump_label_init_type(iter)) + __jump_label_update(key, iter, iter_stop); } return 0; @@ -334,10 +357,10 @@ static void jump_label_del_module(struct module *mod) struct static_key_mod *jlm, **prev; for (iter = iter_start; iter < iter_stop; iter++) { - if (iter->key == (jump_label_t)(unsigned long)key) + if (jump_entry_key(iter) == key) continue; - key = (struct static_key *)(unsigned long)iter->key; + key = jump_entry_key(iter); if (within_module(iter->key, mod)) continue; @@ -439,14 +462,14 @@ int jump_label_text_reserved(void *start, void *end) return ret; } -static void jump_label_update(struct static_key *key, int enable) +static void jump_label_update(struct static_key *key) { struct jump_entry *stop = __stop___jump_table; - struct jump_entry *entry = jump_label_get_entries(key); + struct jump_entry *entry = static_key_entries(key); #ifdef CONFIG_MODULES struct module *mod; - __jump_label_mod_update(key, enable); + __jump_label_mod_update(key); preempt_disable(); mod = __module_address((unsigned long)key); @@ -456,7 +479,44 @@ static void jump_label_update(struct static_key *key, int enable) #endif /* if there are no users, entry can be NULL */ if (entry) - __jump_label_update(key, entry, stop, enable); + __jump_label_update(key, entry, stop); } -#endif +#ifdef CONFIG_STATIC_KEYS_SELFTEST +static DEFINE_STATIC_KEY_TRUE(sk_true); +static DEFINE_STATIC_KEY_FALSE(sk_false); + +static __init int jump_label_test(void) +{ + int i; + + for (i = 0; i < 2; i++) { + WARN_ON(static_key_enabled(&sk_true.key) != true); + WARN_ON(static_key_enabled(&sk_false.key) != false); + + WARN_ON(!static_branch_likely(&sk_true)); + WARN_ON(!static_branch_unlikely(&sk_true)); + WARN_ON(static_branch_likely(&sk_false)); + WARN_ON(static_branch_unlikely(&sk_false)); + + static_branch_disable(&sk_true); + static_branch_enable(&sk_false); + + WARN_ON(static_key_enabled(&sk_true.key) == true); + WARN_ON(static_key_enabled(&sk_false.key) == false); + + WARN_ON(static_branch_likely(&sk_true)); + WARN_ON(static_branch_unlikely(&sk_true)); + WARN_ON(!static_branch_likely(&sk_false)); + WARN_ON(!static_branch_unlikely(&sk_false)); + + static_branch_enable(&sk_true); + static_branch_disable(&sk_false); + } + + return 0; +} +late_initcall(jump_label_test); +#endif /* STATIC_KEYS_SELFTEST */ + +#endif /* HAVE_JUMP_LABEL */ diff --git a/kernel/locking/Makefile b/kernel/locking/Makefile index 7dd5c9918e4c24..36942047ffc080 100644 --- a/kernel/locking/Makefile +++ b/kernel/locking/Makefile @@ -20,7 +20,6 @@ obj-$(CONFIG_PROVE_LOCKING) += spinlock.o obj-$(CONFIG_QUEUED_SPINLOCKS) += qspinlock.o obj-$(CONFIG_RT_MUTEXES) += rtmutex.o obj-$(CONFIG_DEBUG_RT_MUTEXES) += rtmutex-debug.o -obj-$(CONFIG_RT_MUTEX_TESTER) += rtmutex-tester.o obj-$(CONFIG_DEBUG_SPINLOCK) += spinlock.o obj-$(CONFIG_DEBUG_SPINLOCK) += spinlock_debug.o obj-$(CONFIG_RWSEM_GENERIC_SPINLOCK) += rwsem-spinlock.o diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c index 6c5da483966bde..f17a3e3b355079 100644 --- a/kernel/locking/qrwlock.c +++ b/kernel/locking/qrwlock.c @@ -55,27 +55,29 @@ rspin_until_writer_unlock(struct qrwlock *lock, u32 cnts) { while ((cnts & _QW_WMASK) == _QW_LOCKED) { cpu_relax_lowlatency(); - cnts = smp_load_acquire((u32 *)&lock->cnts); + cnts = atomic_read_acquire(&lock->cnts); } } /** - * queue_read_lock_slowpath - acquire read lock of a queue rwlock + * queued_read_lock_slowpath - acquire read lock of a queue rwlock * @lock: Pointer to queue rwlock structure + * @cnts: Current qrwlock lock value */ -void queue_read_lock_slowpath(struct qrwlock *lock) +void queued_read_lock_slowpath(struct qrwlock *lock, u32 cnts) { - u32 cnts; - /* * Readers come here when they cannot get the lock without waiting */ if (unlikely(in_interrupt())) { /* - * Readers in interrupt context will spin until the lock is - * available without waiting in the queue. + * Readers in interrupt context will get the lock immediately + * if the writer is just waiting (not holding the lock yet). + * The rspin_until_writer_unlock() function returns immediately + * in this case. Otherwise, they will spin (with ACQUIRE + * semantics) until the lock is available without waiting in + * the queue. */ - cnts = smp_load_acquire((u32 *)&lock->cnts); rspin_until_writer_unlock(lock, cnts); return; } @@ -87,16 +89,11 @@ void queue_read_lock_slowpath(struct qrwlock *lock) arch_spin_lock(&lock->lock); /* - * At the head of the wait queue now, wait until the writer state - * goes to 0 and then try to increment the reader count and get - * the lock. It is possible that an incoming writer may steal the - * lock in the interim, so it is necessary to check the writer byte - * to make sure that the write lock isn't taken. + * The ACQUIRE semantics of the following spinning code ensure + * that accesses can't leak upwards out of our subsequent critical + * section in the case that the lock is currently held for write. */ - while (atomic_read(&lock->cnts) & _QW_WMASK) - cpu_relax_lowlatency(); - - cnts = atomic_add_return(_QR_BIAS, &lock->cnts) - _QR_BIAS; + cnts = atomic_add_return_acquire(_QR_BIAS, &lock->cnts) - _QR_BIAS; rspin_until_writer_unlock(lock, cnts); /* @@ -104,13 +101,13 @@ void queue_read_lock_slowpath(struct qrwlock *lock) */ arch_spin_unlock(&lock->lock); } -EXPORT_SYMBOL(queue_read_lock_slowpath); +EXPORT_SYMBOL(queued_read_lock_slowpath); /** - * queue_write_lock_slowpath - acquire write lock of a queue rwlock + * queued_write_lock_slowpath - acquire write lock of a queue rwlock * @lock : Pointer to queue rwlock structure */ -void queue_write_lock_slowpath(struct qrwlock *lock) +void queued_write_lock_slowpath(struct qrwlock *lock) { u32 cnts; @@ -119,7 +116,7 @@ void queue_write_lock_slowpath(struct qrwlock *lock) /* Try to acquire the lock directly if no reader is present */ if (!atomic_read(&lock->cnts) && - (atomic_cmpxchg(&lock->cnts, 0, _QW_LOCKED) == 0)) + (atomic_cmpxchg_acquire(&lock->cnts, 0, _QW_LOCKED) == 0)) goto unlock; /* @@ -130,7 +127,7 @@ void queue_write_lock_slowpath(struct qrwlock *lock) struct __qrwlock *l = (struct __qrwlock *)lock; if (!READ_ONCE(l->wmode) && - (cmpxchg(&l->wmode, 0, _QW_WAITING) == 0)) + (cmpxchg_relaxed(&l->wmode, 0, _QW_WAITING) == 0)) break; cpu_relax_lowlatency(); @@ -140,8 +137,8 @@ void queue_write_lock_slowpath(struct qrwlock *lock) for (;;) { cnts = atomic_read(&lock->cnts); if ((cnts == _QW_WAITING) && - (atomic_cmpxchg(&lock->cnts, _QW_WAITING, - _QW_LOCKED) == _QW_WAITING)) + (atomic_cmpxchg_acquire(&lock->cnts, _QW_WAITING, + _QW_LOCKED) == _QW_WAITING)) break; cpu_relax_lowlatency(); @@ -149,4 +146,4 @@ void queue_write_lock_slowpath(struct qrwlock *lock) unlock: arch_spin_unlock(&lock->lock); } -EXPORT_SYMBOL(queue_write_lock_slowpath); +EXPORT_SYMBOL(queued_write_lock_slowpath); diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c index 38c49202d532b3..337c8818541d33 100644 --- a/kernel/locking/qspinlock.c +++ b/kernel/locking/qspinlock.c @@ -239,8 +239,8 @@ static __always_inline void set_locked(struct qspinlock *lock) static __always_inline void __pv_init_node(struct mcs_spinlock *node) { } static __always_inline void __pv_wait_node(struct mcs_spinlock *node) { } -static __always_inline void __pv_kick_node(struct mcs_spinlock *node) { } - +static __always_inline void __pv_kick_node(struct qspinlock *lock, + struct mcs_spinlock *node) { } static __always_inline void __pv_wait_head(struct qspinlock *lock, struct mcs_spinlock *node) { } @@ -440,7 +440,7 @@ void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val) cpu_relax(); arch_mcs_spin_unlock_contended(&next->locked); - pv_kick_node(next); + pv_kick_node(lock, next); release: /* diff --git a/kernel/locking/qspinlock_paravirt.h b/kernel/locking/qspinlock_paravirt.h index df19ae4debd09c..c8e6e9a596f513 100644 --- a/kernel/locking/qspinlock_paravirt.h +++ b/kernel/locking/qspinlock_paravirt.h @@ -22,9 +22,14 @@ #define _Q_SLOW_VAL (3U << _Q_LOCKED_OFFSET) +/* + * Queue node uses: vcpu_running & vcpu_halted. + * Queue head uses: vcpu_running & vcpu_hashed. + */ enum vcpu_state { vcpu_running = 0, - vcpu_halted, + vcpu_halted, /* Used only in pv_wait_node */ + vcpu_hashed, /* = pv_hash'ed + vcpu_halted */ }; struct pv_node { @@ -153,7 +158,8 @@ static void pv_init_node(struct mcs_spinlock *node) /* * Wait for node->locked to become true, halt the vcpu after a short spin. - * pv_kick_node() is used to wake the vcpu again. + * pv_kick_node() is used to set _Q_SLOW_VAL and fill in hash table on its + * behalf. */ static void pv_wait_node(struct mcs_spinlock *node) { @@ -172,9 +178,9 @@ static void pv_wait_node(struct mcs_spinlock *node) * * [S] pn->state = vcpu_halted [S] next->locked = 1 * MB MB - * [L] pn->locked [RmW] pn->state = vcpu_running + * [L] pn->locked [RmW] pn->state = vcpu_hashed * - * Matches the xchg() from pv_kick_node(). + * Matches the cmpxchg() from pv_kick_node(). */ smp_store_mb(pn->state, vcpu_halted); @@ -182,9 +188,10 @@ static void pv_wait_node(struct mcs_spinlock *node) pv_wait(&pn->state, vcpu_halted); /* - * Reset the vCPU state to avoid unncessary CPU kicking + * If pv_kick_node() changed us to vcpu_hashed, retain that value + * so that pv_wait_head() knows to not also try to hash this lock. */ - WRITE_ONCE(pn->state, vcpu_running); + cmpxchg(&pn->state, vcpu_halted, vcpu_running); /* * If the locked flag is still not set after wakeup, it is a @@ -194,6 +201,7 @@ static void pv_wait_node(struct mcs_spinlock *node) * MCS lock will be released soon. */ } + /* * By now our node->locked should be 1 and our caller will not actually * spin-wait for it. We do however rely on our caller to do a @@ -202,24 +210,35 @@ static void pv_wait_node(struct mcs_spinlock *node) } /* - * Called after setting next->locked = 1, used to wake those stuck in - * pv_wait_node(). + * Called after setting next->locked = 1 when we're the lock owner. + * + * Instead of waking the waiters stuck in pv_wait_node() advance their state such + * that they're waiting in pv_wait_head(), this avoids a wake/sleep cycle. */ -static void pv_kick_node(struct mcs_spinlock *node) +static void pv_kick_node(struct qspinlock *lock, struct mcs_spinlock *node) { struct pv_node *pn = (struct pv_node *)node; + struct __qspinlock *l = (void *)lock; /* - * Note that because node->locked is already set, this actual - * mcs_spinlock entry could be re-used already. + * If the vCPU is indeed halted, advance its state to match that of + * pv_wait_node(). If OTOH this fails, the vCPU was running and will + * observe its next->locked value and advance itself. * - * This should be fine however, kicking people for no reason is - * harmless. + * Matches with smp_store_mb() and cmpxchg() in pv_wait_node() + */ + if (cmpxchg(&pn->state, vcpu_halted, vcpu_hashed) != vcpu_halted) + return; + + /* + * Put the lock into the hash table and set the _Q_SLOW_VAL. * - * See the comment in pv_wait_node(). + * As this is the same vCPU that will check the _Q_SLOW_VAL value and + * the hash table later on at unlock time, no atomic instruction is + * needed. */ - if (xchg(&pn->state, vcpu_running) == vcpu_halted) - pv_kick(pn->cpu); + WRITE_ONCE(l->locked, _Q_SLOW_VAL); + (void)pv_hash(lock, pn); } /* @@ -233,6 +252,13 @@ static void pv_wait_head(struct qspinlock *lock, struct mcs_spinlock *node) struct qspinlock **lp = NULL; int loop; + /* + * If pv_kick_node() already advanced our state, we don't need to + * insert ourselves into the hash table anymore. + */ + if (READ_ONCE(pn->state) == vcpu_hashed) + lp = (struct qspinlock **)1; + for (;;) { for (loop = SPIN_THRESHOLD; loop; loop--) { if (!READ_ONCE(l->locked)) @@ -240,17 +266,22 @@ static void pv_wait_head(struct qspinlock *lock, struct mcs_spinlock *node) cpu_relax(); } - WRITE_ONCE(pn->state, vcpu_halted); if (!lp) { /* ONCE */ + WRITE_ONCE(pn->state, vcpu_hashed); lp = pv_hash(lock, pn); + /* - * lp must be set before setting _Q_SLOW_VAL + * We must hash before setting _Q_SLOW_VAL, such that + * when we observe _Q_SLOW_VAL in __pv_queued_spin_unlock() + * we'll be sure to be able to observe our hash entry. * - * [S] lp = lock [RmW] l = l->locked = 0 - * MB MB - * [S] l->locked = _Q_SLOW_VAL [L] lp + * [S] pn->state + * [S] [Rmw] l->locked == _Q_SLOW_VAL + * MB RMB + * [RmW] l->locked = _Q_SLOW_VAL [L] + * [L] pn->state * - * Matches the cmpxchg() in __pv_queued_spin_unlock(). + * Matches the smp_rmb() in __pv_queued_spin_unlock(). */ if (!cmpxchg(&l->locked, _Q_LOCKED_VAL, _Q_SLOW_VAL)) { /* @@ -287,23 +318,33 @@ __visible void __pv_queued_spin_unlock(struct qspinlock *lock) { struct __qspinlock *l = (void *)lock; struct pv_node *node; - u8 lockval = cmpxchg(&l->locked, _Q_LOCKED_VAL, 0); + u8 locked; /* * We must not unlock if SLOW, because in that case we must first * unhash. Otherwise it would be possible to have multiple @lock * entries, which would be BAD. */ - if (likely(lockval == _Q_LOCKED_VAL)) + locked = cmpxchg(&l->locked, _Q_LOCKED_VAL, 0); + if (likely(locked == _Q_LOCKED_VAL)) return; - if (unlikely(lockval != _Q_SLOW_VAL)) { - if (debug_locks_silent) - return; - WARN(1, "pvqspinlock: lock %p has corrupted value 0x%x!\n", lock, atomic_read(&lock->val)); + if (unlikely(locked != _Q_SLOW_VAL)) { + WARN(!debug_locks_silent, + "pvqspinlock: lock 0x%lx has corrupted value 0x%x!\n", + (unsigned long)lock, atomic_read(&lock->val)); return; } + /* + * A failed cmpxchg doesn't provide any memory-ordering guarantees, + * so we need a barrier to order the read of the node data in + * pv_unhash *after* we've read the lock being _Q_SLOW_VAL. + * + * Matches the cmpxchg() in pv_wait_head() setting _Q_SLOW_VAL. + */ + smp_rmb(); + /* * Since the above failed to release, this must be the SLOW path. * Therefore start by looking up the blocked node and unhashing it. @@ -319,8 +360,11 @@ __visible void __pv_queued_spin_unlock(struct qspinlock *lock) /* * At this point the memory pointed at by lock can be freed/reused, * however we can still use the pv_node to kick the CPU. + * The other vCPU may not really be halted, but kicking an active + * vCPU is harmless other than the additional latency in completing + * the unlock. */ - if (READ_ONCE(node->state) == vcpu_halted) + if (READ_ONCE(node->state) == vcpu_hashed) pv_kick(node->cpu); } /* diff --git a/kernel/locking/rtmutex-tester.c b/kernel/locking/rtmutex-tester.c deleted file mode 100644 index 1d96dd0d93c1aa..00000000000000 --- a/kernel/locking/rtmutex-tester.c +++ /dev/null @@ -1,420 +0,0 @@ -/* - * RT-Mutex-tester: scriptable tester for rt mutexes - * - * started by Thomas Gleixner: - * - * Copyright (C) 2006, Timesys Corp., Thomas Gleixner - * - */ -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "rtmutex.h" - -#define MAX_RT_TEST_THREADS 8 -#define MAX_RT_TEST_MUTEXES 8 - -static spinlock_t rttest_lock; -static atomic_t rttest_event; - -struct test_thread_data { - int opcode; - int opdata; - int mutexes[MAX_RT_TEST_MUTEXES]; - int event; - struct device dev; -}; - -static struct test_thread_data thread_data[MAX_RT_TEST_THREADS]; -static struct task_struct *threads[MAX_RT_TEST_THREADS]; -static struct rt_mutex mutexes[MAX_RT_TEST_MUTEXES]; - -enum test_opcodes { - RTTEST_NOP = 0, - RTTEST_SCHEDOT, /* 1 Sched other, data = nice */ - RTTEST_SCHEDRT, /* 2 Sched fifo, data = prio */ - RTTEST_LOCK, /* 3 Lock uninterruptible, data = lockindex */ - RTTEST_LOCKNOWAIT, /* 4 Lock uninterruptible no wait in wakeup, data = lockindex */ - RTTEST_LOCKINT, /* 5 Lock interruptible, data = lockindex */ - RTTEST_LOCKINTNOWAIT, /* 6 Lock interruptible no wait in wakeup, data = lockindex */ - RTTEST_LOCKCONT, /* 7 Continue locking after the wakeup delay */ - RTTEST_UNLOCK, /* 8 Unlock, data = lockindex */ - /* 9, 10 - reserved for BKL commemoration */ - RTTEST_SIGNAL = 11, /* 11 Signal other test thread, data = thread id */ - RTTEST_RESETEVENT = 98, /* 98 Reset event counter */ - RTTEST_RESET = 99, /* 99 Reset all pending operations */ -}; - -static int handle_op(struct test_thread_data *td, int lockwakeup) -{ - int i, id, ret = -EINVAL; - - switch(td->opcode) { - - case RTTEST_NOP: - return 0; - - case RTTEST_LOCKCONT: - td->mutexes[td->opdata] = 1; - td->event = atomic_add_return(1, &rttest_event); - return 0; - - case RTTEST_RESET: - for (i = 0; i < MAX_RT_TEST_MUTEXES; i++) { - if (td->mutexes[i] == 4) { - rt_mutex_unlock(&mutexes[i]); - td->mutexes[i] = 0; - } - } - return 0; - - case RTTEST_RESETEVENT: - atomic_set(&rttest_event, 0); - return 0; - - default: - if (lockwakeup) - return ret; - } - - switch(td->opcode) { - - case RTTEST_LOCK: - case RTTEST_LOCKNOWAIT: - id = td->opdata; - if (id < 0 || id >= MAX_RT_TEST_MUTEXES) - return ret; - - td->mutexes[id] = 1; - td->event = atomic_add_return(1, &rttest_event); - rt_mutex_lock(&mutexes[id]); - td->event = atomic_add_return(1, &rttest_event); - td->mutexes[id] = 4; - return 0; - - case RTTEST_LOCKINT: - case RTTEST_LOCKINTNOWAIT: - id = td->opdata; - if (id < 0 || id >= MAX_RT_TEST_MUTEXES) - return ret; - - td->mutexes[id] = 1; - td->event = atomic_add_return(1, &rttest_event); - ret = rt_mutex_lock_interruptible(&mutexes[id], 0); - td->event = atomic_add_return(1, &rttest_event); - td->mutexes[id] = ret ? 0 : 4; - return ret ? -EINTR : 0; - - case RTTEST_UNLOCK: - id = td->opdata; - if (id < 0 || id >= MAX_RT_TEST_MUTEXES || td->mutexes[id] != 4) - return ret; - - td->event = atomic_add_return(1, &rttest_event); - rt_mutex_unlock(&mutexes[id]); - td->event = atomic_add_return(1, &rttest_event); - td->mutexes[id] = 0; - return 0; - - default: - break; - } - return ret; -} - -/* - * Schedule replacement for rtsem_down(). Only called for threads with - * PF_MUTEX_TESTER set. - * - * This allows us to have finegrained control over the event flow. - * - */ -void schedule_rt_mutex_test(struct rt_mutex *mutex) -{ - int tid, op, dat; - struct test_thread_data *td; - - /* We have to lookup the task */ - for (tid = 0; tid < MAX_RT_TEST_THREADS; tid++) { - if (threads[tid] == current) - break; - } - - BUG_ON(tid == MAX_RT_TEST_THREADS); - - td = &thread_data[tid]; - - op = td->opcode; - dat = td->opdata; - - switch (op) { - case RTTEST_LOCK: - case RTTEST_LOCKINT: - case RTTEST_LOCKNOWAIT: - case RTTEST_LOCKINTNOWAIT: - if (mutex != &mutexes[dat]) - break; - - if (td->mutexes[dat] != 1) - break; - - td->mutexes[dat] = 2; - td->event = atomic_add_return(1, &rttest_event); - break; - - default: - break; - } - - schedule(); - - - switch (op) { - case RTTEST_LOCK: - case RTTEST_LOCKINT: - if (mutex != &mutexes[dat]) - return; - - if (td->mutexes[dat] != 2) - return; - - td->mutexes[dat] = 3; - td->event = atomic_add_return(1, &rttest_event); - break; - - case RTTEST_LOCKNOWAIT: - case RTTEST_LOCKINTNOWAIT: - if (mutex != &mutexes[dat]) - return; - - if (td->mutexes[dat] != 2) - return; - - td->mutexes[dat] = 1; - td->event = atomic_add_return(1, &rttest_event); - return; - - default: - return; - } - - td->opcode = 0; - - for (;;) { - set_current_state(TASK_INTERRUPTIBLE); - - if (td->opcode > 0) { - int ret; - - set_current_state(TASK_RUNNING); - ret = handle_op(td, 1); - set_current_state(TASK_INTERRUPTIBLE); - if (td->opcode == RTTEST_LOCKCONT) - break; - td->opcode = ret; - } - - /* Wait for the next command to be executed */ - schedule(); - } - - /* Restore previous command and data */ - td->opcode = op; - td->opdata = dat; -} - -static int test_func(void *data) -{ - struct test_thread_data *td = data; - int ret; - - current->flags |= PF_MUTEX_TESTER; - set_freezable(); - allow_signal(SIGHUP); - - for(;;) { - - set_current_state(TASK_INTERRUPTIBLE); - - if (td->opcode > 0) { - set_current_state(TASK_RUNNING); - ret = handle_op(td, 0); - set_current_state(TASK_INTERRUPTIBLE); - td->opcode = ret; - } - - /* Wait for the next command to be executed */ - schedule(); - try_to_freeze(); - - if (signal_pending(current)) - flush_signals(current); - - if(kthread_should_stop()) - break; - } - return 0; -} - -/** - * sysfs_test_command - interface for test commands - * @dev: thread reference - * @buf: command for actual step - * @count: length of buffer - * - * command syntax: - * - * opcode:data - */ -static ssize_t sysfs_test_command(struct device *dev, struct device_attribute *attr, - const char *buf, size_t count) -{ - struct sched_param schedpar; - struct test_thread_data *td; - char cmdbuf[32]; - int op, dat, tid, ret; - - td = container_of(dev, struct test_thread_data, dev); - tid = td->dev.id; - - /* strings from sysfs write are not 0 terminated! */ - if (count >= sizeof(cmdbuf)) - return -EINVAL; - - /* strip of \n: */ - if (buf[count-1] == '\n') - count--; - if (count < 1) - return -EINVAL; - - memcpy(cmdbuf, buf, count); - cmdbuf[count] = 0; - - if (sscanf(cmdbuf, "%d:%d", &op, &dat) != 2) - return -EINVAL; - - switch (op) { - case RTTEST_SCHEDOT: - schedpar.sched_priority = 0; - ret = sched_setscheduler(threads[tid], SCHED_NORMAL, &schedpar); - if (ret) - return ret; - set_user_nice(current, 0); - break; - - case RTTEST_SCHEDRT: - schedpar.sched_priority = dat; - ret = sched_setscheduler(threads[tid], SCHED_FIFO, &schedpar); - if (ret) - return ret; - break; - - case RTTEST_SIGNAL: - send_sig(SIGHUP, threads[tid], 0); - break; - - default: - if (td->opcode > 0) - return -EBUSY; - td->opdata = dat; - td->opcode = op; - wake_up_process(threads[tid]); - } - - return count; -} - -/** - * sysfs_test_status - sysfs interface for rt tester - * @dev: thread to query - * @buf: char buffer to be filled with thread status info - */ -static ssize_t sysfs_test_status(struct device *dev, struct device_attribute *attr, - char *buf) -{ - struct test_thread_data *td; - struct task_struct *tsk; - char *curr = buf; - int i; - - td = container_of(dev, struct test_thread_data, dev); - tsk = threads[td->dev.id]; - - spin_lock(&rttest_lock); - - curr += sprintf(curr, - "O: %4d, E:%8d, S: 0x%08lx, P: %4d, N: %4d, B: %p, M:", - td->opcode, td->event, tsk->state, - (MAX_RT_PRIO - 1) - tsk->prio, - (MAX_RT_PRIO - 1) - tsk->normal_prio, - tsk->pi_blocked_on); - - for (i = MAX_RT_TEST_MUTEXES - 1; i >=0 ; i--) - curr += sprintf(curr, "%d", td->mutexes[i]); - - spin_unlock(&rttest_lock); - - curr += sprintf(curr, ", T: %p, R: %p\n", tsk, - mutexes[td->dev.id].owner); - - return curr - buf; -} - -static DEVICE_ATTR(status, S_IRUSR, sysfs_test_status, NULL); -static DEVICE_ATTR(command, S_IWUSR, NULL, sysfs_test_command); - -static struct bus_type rttest_subsys = { - .name = "rttest", - .dev_name = "rttest", -}; - -static int init_test_thread(int id) -{ - thread_data[id].dev.bus = &rttest_subsys; - thread_data[id].dev.id = id; - - threads[id] = kthread_run(test_func, &thread_data[id], "rt-test-%d", id); - if (IS_ERR(threads[id])) - return PTR_ERR(threads[id]); - - return device_register(&thread_data[id].dev); -} - -static int init_rttest(void) -{ - int ret, i; - - spin_lock_init(&rttest_lock); - - for (i = 0; i < MAX_RT_TEST_MUTEXES; i++) - rt_mutex_init(&mutexes[i]); - - ret = subsys_system_register(&rttest_subsys, NULL); - if (ret) - return ret; - - for (i = 0; i < MAX_RT_TEST_THREADS; i++) { - ret = init_test_thread(i); - if (ret) - break; - ret = device_create_file(&thread_data[i].dev, &dev_attr_status); - if (ret) - break; - ret = device_create_file(&thread_data[i].dev, &dev_attr_command); - if (ret) - break; - } - - printk("Initializing RT-Tester: %s\n", ret ? "Failed" : "OK" ); - - return ret; -} - -device_initcall(init_rttest); diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 5674b073473c2f..7781d801212fa3 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -1120,7 +1120,7 @@ __rt_mutex_slowlock(struct rt_mutex *lock, int state, debug_rt_mutex_print_deadlock(waiter); - schedule_rt_mutex(lock); + schedule(); raw_spin_lock(&lock->wait_lock); set_current_state(state); diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h index 7844f8f0e6391e..4f5f83c7d2d351 100644 --- a/kernel/locking/rtmutex_common.h +++ b/kernel/locking/rtmutex_common.h @@ -14,28 +14,6 @@ #include -/* - * The rtmutex in kernel tester is independent of rtmutex debugging. We - * call schedule_rt_mutex_test() instead of schedule() for the tasks which - * belong to the tester. That way we can delay the wakeup path of those - * threads to provoke lock stealing and testing of complex boosting scenarios. - */ -#ifdef CONFIG_RT_MUTEX_TESTER - -extern void schedule_rt_mutex_test(struct rt_mutex *lock); - -#define schedule_rt_mutex(_lock) \ - do { \ - if (!(current->flags & PF_MUTEX_TESTER)) \ - schedule(); \ - else \ - schedule_rt_mutex_test(_lock); \ - } while (0) - -#else -# define schedule_rt_mutex(_lock) schedule() -#endif - /* * This is the control structure for tasks blocked on a rt_mutex, * which is allocated on the kernel stack on of the blocked task. diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d8420c233ff762..3595403921bd5b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -164,14 +164,12 @@ struct static_key sched_feat_keys[__SCHED_FEAT_NR] = { static void sched_feat_disable(int i) { - if (static_key_enabled(&sched_feat_keys[i])) - static_key_slow_dec(&sched_feat_keys[i]); + static_key_disable(&sched_feat_keys[i]); } static void sched_feat_enable(int i) { - if (!static_key_enabled(&sched_feat_keys[i])) - static_key_slow_inc(&sched_feat_keys[i]); + static_key_enable(&sched_feat_keys[i]); } #else static void sched_feat_disable(int i) { }; diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 3e0b662cae0961..ab76b99adc857f 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -916,12 +916,6 @@ config DEBUG_RT_MUTEXES This allows rt mutex semantics violations and rt mutex related deadlocks (lockups) to be detected and reported automatically. -config RT_MUTEX_TESTER - bool "Built-in scriptable tester for rt-mutexes" - depends on DEBUG_KERNEL && RT_MUTEXES && BROKEN - help - This option enables a rt-mutex tester. - config DEBUG_SPINLOCK bool "Spinlock and rw-lock debugging: basic checks" depends on DEBUG_KERNEL @@ -1528,6 +1522,13 @@ config FAIL_MMC_REQUEST and to test how the mmc host driver handles retries from the block device. +config FAIL_FUTEX + bool "Fault-injection capability for futexes" + select DEBUG_FS + depends on FAULT_INJECTION && FUTEX + help + Provide fault-injection capability for futexes. + config FAULT_INJECTION_DEBUG_FS bool "Debugfs entries for fault-injection capabilities" depends on FAULT_INJECTION && SYSFS && DEBUG_FS @@ -1826,6 +1827,15 @@ config MEMTEST memtest=17, mean do 17 test patterns. If you are unsure how to answer this question, answer N. +config TEST_STATIC_KEYS + tristate "Test static keys" + default n + depends on m + help + Test the static key interfaces. + + If unsure, say N. + source "samples/Kconfig" source "lib/Kconfig.kgdb" diff --git a/lib/Makefile b/lib/Makefile index f2610061bfa4df..f01c558bf80db6 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -39,6 +39,8 @@ obj-$(CONFIG_TEST_KSTRTOX) += test-kstrtox.o obj-$(CONFIG_TEST_LKM) += test_module.o obj-$(CONFIG_TEST_RHASHTABLE) += test_rhashtable.o obj-$(CONFIG_TEST_USER_COPY) += test_user_copy.o +obj-$(CONFIG_TEST_STATIC_KEYS) += test_static_keys.o +obj-$(CONFIG_TEST_STATIC_KEYS) += test_static_key_base.o ifeq ($(CONFIG_DEBUG_KOBJECT),y) CFLAGS_kobject.o += -DDEBUG diff --git a/lib/atomic64.c b/lib/atomic64.c index 1298c05ef52848..2886ebac656772 100644 --- a/lib/atomic64.c +++ b/lib/atomic64.c @@ -102,6 +102,9 @@ EXPORT_SYMBOL(atomic64_##op##_return); ATOMIC64_OPS(add, +=) ATOMIC64_OPS(sub, -=) +ATOMIC64_OP(and, &=) +ATOMIC64_OP(or, |=) +ATOMIC64_OP(xor, ^=) #undef ATOMIC64_OPS #undef ATOMIC64_OP_RETURN diff --git a/lib/atomic64_test.c b/lib/atomic64_test.c index 0211d30d8c390f..83c33a5bcffb1a 100644 --- a/lib/atomic64_test.c +++ b/lib/atomic64_test.c @@ -16,8 +16,39 @@ #include #include +#define TEST(bit, op, c_op, val) \ +do { \ + atomic##bit##_set(&v, v0); \ + r = v0; \ + atomic##bit##_##op(val, &v); \ + r c_op val; \ + WARN(atomic##bit##_read(&v) != r, "%Lx != %Lx\n", \ + (unsigned long long)atomic##bit##_read(&v), \ + (unsigned long long)r); \ +} while (0) + +static __init void test_atomic(void) +{ + int v0 = 0xaaa31337; + int v1 = 0xdeadbeef; + int onestwos = 0x11112222; + int one = 1; + + atomic_t v; + int r; + + TEST(, add, +=, onestwos); + TEST(, add, +=, -one); + TEST(, sub, -=, onestwos); + TEST(, sub, -=, -one); + TEST(, or, |=, v1); + TEST(, and, &=, v1); + TEST(, xor, ^=, v1); + TEST(, andnot, &= ~, v1); +} + #define INIT(c) do { atomic64_set(&v, c); r = c; } while (0) -static __init int test_atomic64(void) +static __init void test_atomic64(void) { long long v0 = 0xaaa31337c001d00dLL; long long v1 = 0xdeadbeefdeafcafeLL; @@ -34,15 +65,14 @@ static __init int test_atomic64(void) BUG_ON(v.counter != r); BUG_ON(atomic64_read(&v) != r); - INIT(v0); - atomic64_add(onestwos, &v); - r += onestwos; - BUG_ON(v.counter != r); - - INIT(v0); - atomic64_add(-one, &v); - r += -one; - BUG_ON(v.counter != r); + TEST(64, add, +=, onestwos); + TEST(64, add, +=, -one); + TEST(64, sub, -=, onestwos); + TEST(64, sub, -=, -one); + TEST(64, or, |=, v1); + TEST(64, and, &=, v1); + TEST(64, xor, ^=, v1); + TEST(64, andnot, &= ~, v1); INIT(v0); r += onestwos; @@ -54,16 +84,6 @@ static __init int test_atomic64(void) BUG_ON(atomic64_add_return(-one, &v) != r); BUG_ON(v.counter != r); - INIT(v0); - atomic64_sub(onestwos, &v); - r -= onestwos; - BUG_ON(v.counter != r); - - INIT(v0); - atomic64_sub(-one, &v); - r -= -one; - BUG_ON(v.counter != r); - INIT(v0); r -= onestwos; BUG_ON(atomic64_sub_return(onestwos, &v) != r); @@ -147,6 +167,12 @@ static __init int test_atomic64(void) BUG_ON(!atomic64_inc_not_zero(&v)); r += one; BUG_ON(v.counter != r); +} + +static __init int test_atomics(void) +{ + test_atomic(); + test_atomic64(); #ifdef CONFIG_X86 pr_info("passed for %s platform %s CX8 and %s SSE\n", @@ -166,4 +192,4 @@ static __init int test_atomic64(void) return 0; } -core_initcall(test_atomic64); +core_initcall(test_atomics); diff --git a/lib/lockref.c b/lib/lockref.c index 494994bf17c8ec..5a92189ad711af 100644 --- a/lib/lockref.c +++ b/lib/lockref.c @@ -3,14 +3,6 @@ #if USE_CMPXCHG_LOCKREF -/* - * Allow weakly-ordered memory architectures to provide barrier-less - * cmpxchg semantics for lockref updates. - */ -#ifndef cmpxchg64_relaxed -# define cmpxchg64_relaxed cmpxchg64 -#endif - /* * Note that the "cmpxchg()" reloads the "old" value for the * failure case. diff --git a/lib/test_static_key_base.c b/lib/test_static_key_base.c new file mode 100644 index 00000000000000..729447aea02f4d --- /dev/null +++ b/lib/test_static_key_base.c @@ -0,0 +1,68 @@ +/* + * Kernel module for testing static keys. + * + * Copyright 2015 Akamai Technologies Inc. All Rights Reserved + * + * Authors: + * Jason Baron + * + * This software is licensed under the terms of the GNU General Public + * License version 2, as published by the Free Software Foundation, and + * may be copied, distributed, and modified under those terms. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include + +/* old keys */ +struct static_key base_old_true_key = STATIC_KEY_INIT_TRUE; +EXPORT_SYMBOL_GPL(base_old_true_key); +struct static_key base_inv_old_true_key = STATIC_KEY_INIT_TRUE; +EXPORT_SYMBOL_GPL(base_inv_old_true_key); +struct static_key base_old_false_key = STATIC_KEY_INIT_FALSE; +EXPORT_SYMBOL_GPL(base_old_false_key); +struct static_key base_inv_old_false_key = STATIC_KEY_INIT_FALSE; +EXPORT_SYMBOL_GPL(base_inv_old_false_key); + +/* new keys */ +DEFINE_STATIC_KEY_TRUE(base_true_key); +EXPORT_SYMBOL_GPL(base_true_key); +DEFINE_STATIC_KEY_TRUE(base_inv_true_key); +EXPORT_SYMBOL_GPL(base_inv_true_key); +DEFINE_STATIC_KEY_FALSE(base_false_key); +EXPORT_SYMBOL_GPL(base_false_key); +DEFINE_STATIC_KEY_FALSE(base_inv_false_key); +EXPORT_SYMBOL_GPL(base_inv_false_key); + +static void invert_key(struct static_key *key) +{ + if (static_key_enabled(key)) + static_key_disable(key); + else + static_key_enable(key); +} + +static int __init test_static_key_base_init(void) +{ + invert_key(&base_inv_old_true_key); + invert_key(&base_inv_old_false_key); + invert_key(&base_inv_true_key.key); + invert_key(&base_inv_false_key.key); + + return 0; +} + +static void __exit test_static_key_base_exit(void) +{ +} + +module_init(test_static_key_base_init); +module_exit(test_static_key_base_exit); + +MODULE_AUTHOR("Jason Baron "); +MODULE_LICENSE("GPL"); diff --git a/lib/test_static_keys.c b/lib/test_static_keys.c new file mode 100644 index 00000000000000..c61b299e367ffa --- /dev/null +++ b/lib/test_static_keys.c @@ -0,0 +1,225 @@ +/* + * Kernel module for testing static keys. + * + * Copyright 2015 Akamai Technologies Inc. All Rights Reserved + * + * Authors: + * Jason Baron + * + * This software is licensed under the terms of the GNU General Public + * License version 2, as published by the Free Software Foundation, and + * may be copied, distributed, and modified under those terms. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include +#include + +/* old keys */ +struct static_key old_true_key = STATIC_KEY_INIT_TRUE; +struct static_key old_false_key = STATIC_KEY_INIT_FALSE; + +/* new api */ +DEFINE_STATIC_KEY_TRUE(true_key); +DEFINE_STATIC_KEY_FALSE(false_key); + +/* external */ +extern struct static_key base_old_true_key; +extern struct static_key base_inv_old_true_key; +extern struct static_key base_old_false_key; +extern struct static_key base_inv_old_false_key; + +/* new api */ +extern struct static_key_true base_true_key; +extern struct static_key_true base_inv_true_key; +extern struct static_key_false base_false_key; +extern struct static_key_false base_inv_false_key; + + +struct test_key { + bool init_state; + struct static_key *key; + bool (*test_key)(void); +}; + +#define test_key_func(key, branch) \ + ({bool func(void) { return branch(key); } func; }) + +static void invert_key(struct static_key *key) +{ + if (static_key_enabled(key)) + static_key_disable(key); + else + static_key_enable(key); +} + +static void invert_keys(struct test_key *keys, int size) +{ + struct static_key *previous = NULL; + int i; + + for (i = 0; i < size; i++) { + if (previous != keys[i].key) { + invert_key(keys[i].key); + previous = keys[i].key; + } + } +} + +static int verify_keys(struct test_key *keys, int size, bool invert) +{ + int i; + bool ret, init; + + for (i = 0; i < size; i++) { + ret = static_key_enabled(keys[i].key); + init = keys[i].init_state; + if (ret != (invert ? !init : init)) + return -EINVAL; + ret = keys[i].test_key(); + if (static_key_enabled(keys[i].key)) { + if (!ret) + return -EINVAL; + } else { + if (ret) + return -EINVAL; + } + } + return 0; +} + +static int __init test_static_key_init(void) +{ + int ret; + int size; + + struct test_key static_key_tests[] = { + /* internal keys - old keys */ + { + .init_state = true, + .key = &old_true_key, + .test_key = test_key_func(&old_true_key, static_key_true), + }, + { + .init_state = false, + .key = &old_false_key, + .test_key = test_key_func(&old_false_key, static_key_false), + }, + /* internal keys - new keys */ + { + .init_state = true, + .key = &true_key.key, + .test_key = test_key_func(&true_key, static_branch_likely), + }, + { + .init_state = true, + .key = &true_key.key, + .test_key = test_key_func(&true_key, static_branch_unlikely), + }, + { + .init_state = false, + .key = &false_key.key, + .test_key = test_key_func(&false_key, static_branch_likely), + }, + { + .init_state = false, + .key = &false_key.key, + .test_key = test_key_func(&false_key, static_branch_unlikely), + }, + /* external keys - old keys */ + { + .init_state = true, + .key = &base_old_true_key, + .test_key = test_key_func(&base_old_true_key, static_key_true), + }, + { + .init_state = false, + .key = &base_inv_old_true_key, + .test_key = test_key_func(&base_inv_old_true_key, static_key_true), + }, + { + .init_state = false, + .key = &base_old_false_key, + .test_key = test_key_func(&base_old_false_key, static_key_false), + }, + { + .init_state = true, + .key = &base_inv_old_false_key, + .test_key = test_key_func(&base_inv_old_false_key, static_key_false), + }, + /* external keys - new keys */ + { + .init_state = true, + .key = &base_true_key.key, + .test_key = test_key_func(&base_true_key, static_branch_likely), + }, + { + .init_state = true, + .key = &base_true_key.key, + .test_key = test_key_func(&base_true_key, static_branch_unlikely), + }, + { + .init_state = false, + .key = &base_inv_true_key.key, + .test_key = test_key_func(&base_inv_true_key, static_branch_likely), + }, + { + .init_state = false, + .key = &base_inv_true_key.key, + .test_key = test_key_func(&base_inv_true_key, static_branch_unlikely), + }, + { + .init_state = false, + .key = &base_false_key.key, + .test_key = test_key_func(&base_false_key, static_branch_likely), + }, + { + .init_state = false, + .key = &base_false_key.key, + .test_key = test_key_func(&base_false_key, static_branch_unlikely), + }, + { + .init_state = true, + .key = &base_inv_false_key.key, + .test_key = test_key_func(&base_inv_false_key, static_branch_likely), + }, + { + .init_state = true, + .key = &base_inv_false_key.key, + .test_key = test_key_func(&base_inv_false_key, static_branch_unlikely), + }, + }; + + size = ARRAY_SIZE(static_key_tests); + + ret = verify_keys(static_key_tests, size, false); + if (ret) + goto out; + + invert_keys(static_key_tests, size); + ret = verify_keys(static_key_tests, size, true); + if (ret) + goto out; + + invert_keys(static_key_tests, size); + ret = verify_keys(static_key_tests, size, false); + if (ret) + goto out; + return 0; +out: + return ret; +} + +static void __exit test_static_key_exit(void) +{ +} + +module_init(test_static_key_init); +module_exit(test_static_key_exit); + +MODULE_AUTHOR("Jason Baron "); +MODULE_LICENSE("GPL"); diff --git a/mm/Kconfig b/mm/Kconfig index e79de2bd12cd04..d4e6495a720f4a 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -299,15 +299,9 @@ config BOUNCE # On the 'tile' arch, USB OHCI needs the bounce pool since tilegx will often # have more than 4GB of memory, but we don't currently use the IOTLB to present # a 32-bit address to OHCI. So we need to use a bounce pool instead. -# -# We also use the bounce pool to provide stable page writes for jbd. jbd -# initiates buffer writeback without locking the page or setting PG_writeback, -# and fixing that behavior (a second time; jbd2 doesn't have this problem) is -# a major rework effort. Instead, use the bounce buffer to snapshot pages -# (until jbd goes away). The only jbd user is ext3. config NEED_BOUNCE_POOL bool - default y if (TILE && USB_OHCI_HCD) || (BLK_DEV_INTEGRITY && JBD) + default y if TILE && USB_OHCI_HCD config NR_QUICK int diff --git a/scripts/rt-tester/check-all.sh b/scripts/rt-tester/check-all.sh deleted file mode 100644 index 6b5c83baf14887..00000000000000 --- a/scripts/rt-tester/check-all.sh +++ /dev/null @@ -1,21 +0,0 @@ - - -function testit () -{ - printf "%-30s: " $1 - ./rt-tester.py $1 | grep Pass -} - -testit t2-l1-2rt-sameprio.tst -testit t2-l1-pi.tst -testit t2-l1-signal.tst -#testit t2-l2-2rt-deadlock.tst -testit t3-l1-pi-1rt.tst -testit t3-l1-pi-2rt.tst -testit t3-l1-pi-3rt.tst -testit t3-l1-pi-signal.tst -testit t3-l1-pi-steal.tst -testit t3-l2-pi.tst -testit t4-l2-pi-deboost.tst -testit t5-l4-pi-boost-deboost.tst -testit t5-l4-pi-boost-deboost-setsched.tst diff --git a/scripts/rt-tester/rt-tester.py b/scripts/rt-tester/rt-tester.py deleted file mode 100755 index 6d916c2a45a550..00000000000000 --- a/scripts/rt-tester/rt-tester.py +++ /dev/null @@ -1,218 +0,0 @@ -#!/usr/bin/python -# -# rt-mutex tester -# -# (C) 2006 Thomas Gleixner -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License version 2 as -# published by the Free Software Foundation. -# -import os -import sys -import getopt -import shutil -import string - -# Globals -quiet = 0 -test = 0 -comments = 0 - -sysfsprefix = "/sys/devices/system/rttest/rttest" -statusfile = "/status" -commandfile = "/command" - -# Command opcodes -cmd_opcodes = { - "schedother" : "1", - "schedfifo" : "2", - "lock" : "3", - "locknowait" : "4", - "lockint" : "5", - "lockintnowait" : "6", - "lockcont" : "7", - "unlock" : "8", - "signal" : "11", - "resetevent" : "98", - "reset" : "99", - } - -test_opcodes = { - "prioeq" : ["P" , "eq" , None], - "priolt" : ["P" , "lt" , None], - "priogt" : ["P" , "gt" , None], - "nprioeq" : ["N" , "eq" , None], - "npriolt" : ["N" , "lt" , None], - "npriogt" : ["N" , "gt" , None], - "unlocked" : ["M" , "eq" , 0], - "trylock" : ["M" , "eq" , 1], - "blocked" : ["M" , "eq" , 2], - "blockedwake" : ["M" , "eq" , 3], - "locked" : ["M" , "eq" , 4], - "opcodeeq" : ["O" , "eq" , None], - "opcodelt" : ["O" , "lt" , None], - "opcodegt" : ["O" , "gt" , None], - "eventeq" : ["E" , "eq" , None], - "eventlt" : ["E" , "lt" , None], - "eventgt" : ["E" , "gt" , None], - } - -# Print usage information -def usage(): - print "rt-tester.py <-c -h -q -t> " - print " -c display comments after first command" - print " -h help" - print " -q quiet mode" - print " -t test mode (syntax check)" - print " testfile: read test specification from testfile" - print " otherwise from stdin" - return - -# Print progress when not in quiet mode -def progress(str): - if not quiet: - print str - -# Analyse a status value -def analyse(val, top, arg): - - intval = int(val) - - if top[0] == "M": - intval = intval / (10 ** int(arg)) - intval = intval % 10 - argval = top[2] - elif top[0] == "O": - argval = int(cmd_opcodes.get(arg, arg)) - else: - argval = int(arg) - - # progress("%d %s %d" %(intval, top[1], argval)) - - if top[1] == "eq" and intval == argval: - return 1 - if top[1] == "lt" and intval < argval: - return 1 - if top[1] == "gt" and intval > argval: - return 1 - return 0 - -# Parse the commandline -try: - (options, arguments) = getopt.getopt(sys.argv[1:],'chqt') -except getopt.GetoptError, ex: - usage() - sys.exit(1) - -# Parse commandline options -for option, value in options: - if option == "-c": - comments = 1 - elif option == "-q": - quiet = 1 - elif option == "-t": - test = 1 - elif option == '-h': - usage() - sys.exit(0) - -# Select the input source -if arguments: - try: - fd = open(arguments[0]) - except Exception,ex: - sys.stderr.write("File not found %s\n" %(arguments[0])) - sys.exit(1) -else: - fd = sys.stdin - -linenr = 0 - -# Read the test patterns -while 1: - - linenr = linenr + 1 - line = fd.readline() - if not len(line): - break - - line = line.strip() - parts = line.split(":") - - if not parts or len(parts) < 1: - continue - - if len(parts[0]) == 0: - continue - - if parts[0].startswith("#"): - if comments > 1: - progress(line) - continue - - if comments == 1: - comments = 2 - - progress(line) - - cmd = parts[0].strip().lower() - opc = parts[1].strip().lower() - tid = parts[2].strip() - dat = parts[3].strip() - - try: - # Test or wait for a status value - if cmd == "t" or cmd == "w": - testop = test_opcodes[opc] - - fname = "%s%s%s" %(sysfsprefix, tid, statusfile) - if test: - print fname - continue - - while 1: - query = 1 - fsta = open(fname, 'r') - status = fsta.readline().strip() - fsta.close() - stat = status.split(",") - for s in stat: - s = s.strip() - if s.startswith(testop[0]): - # Separate status value - val = s[2:].strip() - query = analyse(val, testop, dat) - break - if query or cmd == "t": - break - - progress(" " + status) - - if not query: - sys.stderr.write("Test failed in line %d\n" %(linenr)) - sys.exit(1) - - # Issue a command to the tester - elif cmd == "c": - cmdnr = cmd_opcodes[opc] - # Build command string and sys filename - cmdstr = "%s:%s" %(cmdnr, dat) - fname = "%s%s%s" %(sysfsprefix, tid, commandfile) - if test: - print fname - continue - fcmd = open(fname, 'w') - fcmd.write(cmdstr) - fcmd.close() - - except Exception,ex: - sys.stderr.write(str(ex)) - sys.stderr.write("\nSyntax error in line %d\n" %(linenr)) - if not test: - fd.close() - sys.exit(1) - -# Normal exit pass -print "Pass" -sys.exit(0) diff --git a/scripts/rt-tester/t2-l1-2rt-sameprio.tst b/scripts/rt-tester/t2-l1-2rt-sameprio.tst deleted file mode 100644 index 3710c8b2090d5f..00000000000000 --- a/scripts/rt-tester/t2-l1-2rt-sameprio.tst +++ /dev/null @@ -1,94 +0,0 @@ -# -# RT-Mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal 0 -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 2 threads 1 lock -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedfifo: 0: 80 -C: schedfifo: 1: 80 - -# T0 lock L0 -C: locknowait: 0: 0 -C: locknowait: 1: 0 -W: locked: 0: 0 -W: blocked: 1: 0 -T: prioeq: 0: 80 - -# T0 unlock L0 -C: unlock: 0: 0 -W: locked: 1: 0 - -# Verify T0 -W: unlocked: 0: 0 -T: prioeq: 0: 80 - -# Unlock -C: unlock: 1: 0 -W: unlocked: 1: 0 - -# T1,T0 lock L0 -C: locknowait: 1: 0 -C: locknowait: 0: 0 -W: locked: 1: 0 -W: blocked: 0: 0 -T: prioeq: 1: 80 - -# T1 unlock L0 -C: unlock: 1: 0 -W: locked: 0: 0 - -# Verify T1 -W: unlocked: 1: 0 -T: prioeq: 1: 80 - -# Unlock and exit -C: unlock: 0: 0 -W: unlocked: 0: 0 - diff --git a/scripts/rt-tester/t2-l1-pi.tst b/scripts/rt-tester/t2-l1-pi.tst deleted file mode 100644 index b4cc95975adbb4..00000000000000 --- a/scripts/rt-tester/t2-l1-pi.tst +++ /dev/null @@ -1,77 +0,0 @@ -# -# RT-Mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal 0 -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 2 threads 1 lock with priority inversion -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedother: 0: 0 -C: schedfifo: 1: 80 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L0 -C: locknowait: 1: 0 -W: blocked: 1: 0 -T: prioeq: 0: 80 - -# T0 unlock L0 -C: unlock: 0: 0 -W: locked: 1: 0 - -# Verify T1 -W: unlocked: 0: 0 -T: priolt: 0: 1 - -# Unlock and exit -C: unlock: 1: 0 -W: unlocked: 1: 0 - diff --git a/scripts/rt-tester/t2-l1-signal.tst b/scripts/rt-tester/t2-l1-signal.tst deleted file mode 100644 index 1b57376cc1f7ca..00000000000000 --- a/scripts/rt-tester/t2-l1-signal.tst +++ /dev/null @@ -1,72 +0,0 @@ -# -# RT-Mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal 0 -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 2 threads 1 lock with priority inversion -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedother: 0: 0 -C: schedother: 1: 0 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L0 -C: lockintnowait: 1: 0 -W: blocked: 1: 0 - -# Interrupt T1 -C: signal: 1: 0 -W: unlocked: 1: 0 -T: opcodeeq: 1: -4 - -# Unlock and exit -C: unlock: 0: 0 -W: unlocked: 0: 0 diff --git a/scripts/rt-tester/t2-l2-2rt-deadlock.tst b/scripts/rt-tester/t2-l2-2rt-deadlock.tst deleted file mode 100644 index 68b10629b6f4d1..00000000000000 --- a/scripts/rt-tester/t2-l2-2rt-deadlock.tst +++ /dev/null @@ -1,84 +0,0 @@ -# -# RT-Mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal 0 -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 2 threads 2 lock -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedfifo: 0: 80 -C: schedfifo: 1: 80 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L1 -C: locknowait: 1: 1 -W: locked: 1: 1 - -# T0 lock L1 -C: lockintnowait: 0: 1 -W: blocked: 0: 1 - -# T1 lock L0 -C: lockintnowait: 1: 0 -W: blocked: 1: 0 - -# Make deadlock go away -C: signal: 1: 0 -W: unlocked: 1: 0 -C: signal: 0: 0 -W: unlocked: 0: 1 - -# Unlock and exit -C: unlock: 0: 0 -W: unlocked: 0: 0 -C: unlock: 1: 1 -W: unlocked: 1: 1 - diff --git a/scripts/rt-tester/t3-l1-pi-1rt.tst b/scripts/rt-tester/t3-l1-pi-1rt.tst deleted file mode 100644 index 8e6c8b11ae563a..00000000000000 --- a/scripts/rt-tester/t3-l1-pi-1rt.tst +++ /dev/null @@ -1,87 +0,0 @@ -# -# rt-mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal thread to signal (0-7) -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 3 threads 1 lock PI -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedother: 0: 0 -C: schedother: 1: 0 -C: schedfifo: 2: 82 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L0 -C: locknowait: 1: 0 -W: blocked: 1: 0 -T: priolt: 0: 1 - -# T2 lock L0 -C: locknowait: 2: 0 -W: blocked: 2: 0 -T: prioeq: 0: 82 - -# T0 unlock L0 -C: unlock: 0: 0 - -# Wait until T2 got the lock -W: locked: 2: 0 -W: unlocked: 0: 0 -T: priolt: 0: 1 - -# T2 unlock L0 -C: unlock: 2: 0 - -W: unlocked: 2: 0 -W: locked: 1: 0 - -C: unlock: 1: 0 -W: unlocked: 1: 0 diff --git a/scripts/rt-tester/t3-l1-pi-2rt.tst b/scripts/rt-tester/t3-l1-pi-2rt.tst deleted file mode 100644 index 69c2212fc52089..00000000000000 --- a/scripts/rt-tester/t3-l1-pi-2rt.tst +++ /dev/null @@ -1,88 +0,0 @@ -# -# rt-mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal thread to signal (0-7) -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 3 threads 1 lock PI -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedother: 0: 0 -C: schedfifo: 1: 81 -C: schedfifo: 2: 82 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L0 -C: locknowait: 1: 0 -W: blocked: 1: 0 -T: prioeq: 0: 81 - -# T2 lock L0 -C: locknowait: 2: 0 -W: blocked: 2: 0 -T: prioeq: 0: 82 -T: prioeq: 1: 81 - -# T0 unlock L0 -C: unlock: 0: 0 - -# Wait until T2 got the lock -W: locked: 2: 0 -W: unlocked: 0: 0 -T: priolt: 0: 1 - -# T2 unlock L0 -C: unlock: 2: 0 - -W: unlocked: 2: 0 -W: locked: 1: 0 - -C: unlock: 1: 0 -W: unlocked: 1: 0 diff --git a/scripts/rt-tester/t3-l1-pi-3rt.tst b/scripts/rt-tester/t3-l1-pi-3rt.tst deleted file mode 100644 index 9b0f1eb26a88a9..00000000000000 --- a/scripts/rt-tester/t3-l1-pi-3rt.tst +++ /dev/null @@ -1,87 +0,0 @@ -# -# rt-mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal thread to signal (0-7) -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 3 threads 1 lock PI -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedfifo: 0: 80 -C: schedfifo: 1: 81 -C: schedfifo: 2: 82 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L0 -C: locknowait: 1: 0 -W: blocked: 1: 0 -T: prioeq: 0: 81 - -# T2 lock L0 -C: locknowait: 2: 0 -W: blocked: 2: 0 -T: prioeq: 0: 82 - -# T0 unlock L0 -C: unlock: 0: 0 - -# Wait until T2 got the lock -W: locked: 2: 0 -W: unlocked: 0: 0 -T: prioeq: 0: 80 - -# T2 unlock L0 -C: unlock: 2: 0 - -W: locked: 1: 0 -W: unlocked: 2: 0 - -C: unlock: 1: 0 -W: unlocked: 1: 0 diff --git a/scripts/rt-tester/t3-l1-pi-signal.tst b/scripts/rt-tester/t3-l1-pi-signal.tst deleted file mode 100644 index 39ec74ab06ee76..00000000000000 --- a/scripts/rt-tester/t3-l1-pi-signal.tst +++ /dev/null @@ -1,93 +0,0 @@ -# -# rt-mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal thread to signal (0-7) -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# Reset event counter -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set priorities -C: schedother: 0: 0 -C: schedfifo: 1: 80 -C: schedfifo: 2: 81 - -# T0 lock L0 -C: lock: 0: 0 -W: locked: 0: 0 - -# T1 lock L0, no wait in the wakeup path -C: locknowait: 1: 0 -W: blocked: 1: 0 -T: prioeq: 0: 80 -T: prioeq: 1: 80 - -# T2 lock L0 interruptible, no wait in the wakeup path -C: lockintnowait: 2: 0 -W: blocked: 2: 0 -T: prioeq: 0: 81 -T: prioeq: 1: 80 - -# Interrupt T2 -C: signal: 2: 2 -W: unlocked: 2: 0 -T: prioeq: 1: 80 -T: prioeq: 0: 80 - -T: locked: 0: 0 -T: blocked: 1: 0 - -# T0 unlock L0 -C: unlock: 0: 0 - -# Wait until T1 has locked L0 and exit -W: locked: 1: 0 -W: unlocked: 0: 0 -T: priolt: 0: 1 - -C: unlock: 1: 0 -W: unlocked: 1: 0 - - - diff --git a/scripts/rt-tester/t3-l1-pi-steal.tst b/scripts/rt-tester/t3-l1-pi-steal.tst deleted file mode 100644 index e03db7e010fa24..00000000000000 --- a/scripts/rt-tester/t3-l1-pi-steal.tst +++ /dev/null @@ -1,91 +0,0 @@ -# -# rt-mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal thread to signal (0-7) -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 3 threads 1 lock PI steal pending ownership -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedother: 0: 0 -C: schedfifo: 1: 80 -C: schedfifo: 2: 81 - -# T0 lock L0 -C: lock: 0: 0 -W: locked: 0: 0 - -# T1 lock L0 -C: lock: 1: 0 -W: blocked: 1: 0 -T: prioeq: 0: 80 - -# T0 unlock L0 -C: unlock: 0: 0 - -# Wait until T1 is in the wakeup loop -W: blockedwake: 1: 0 -T: priolt: 0: 1 - -# T2 lock L0 -C: lock: 2: 0 -# T1 leave wakeup loop -C: lockcont: 1: 0 - -# T2 must have the lock and T1 must be blocked -W: locked: 2: 0 -W: blocked: 1: 0 - -# T2 unlock L0 -C: unlock: 2: 0 - -# Wait until T1 is in the wakeup loop and let it run -W: blockedwake: 1: 0 -C: lockcont: 1: 0 -W: locked: 1: 0 -C: unlock: 1: 0 -W: unlocked: 1: 0 diff --git a/scripts/rt-tester/t3-l2-pi.tst b/scripts/rt-tester/t3-l2-pi.tst deleted file mode 100644 index 7b59100d3e48f5..00000000000000 --- a/scripts/rt-tester/t3-l2-pi.tst +++ /dev/null @@ -1,87 +0,0 @@ -# -# rt-mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal thread to signal (0-7) -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 3 threads 2 lock PI -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedother: 0: 0 -C: schedother: 1: 0 -C: schedfifo: 2: 82 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L0 -C: locknowait: 1: 0 -W: blocked: 1: 0 -T: priolt: 0: 1 - -# T2 lock L0 -C: locknowait: 2: 0 -W: blocked: 2: 0 -T: prioeq: 0: 82 - -# T0 unlock L0 -C: unlock: 0: 0 - -# Wait until T2 got the lock -W: locked: 2: 0 -W: unlocked: 0: 0 -T: priolt: 0: 1 - -# T2 unlock L0 -C: unlock: 2: 0 - -W: unlocked: 2: 0 -W: locked: 1: 0 - -C: unlock: 1: 0 -W: unlocked: 1: 0 diff --git a/scripts/rt-tester/t4-l2-pi-deboost.tst b/scripts/rt-tester/t4-l2-pi-deboost.tst deleted file mode 100644 index 2f0e049d64438b..00000000000000 --- a/scripts/rt-tester/t4-l2-pi-deboost.tst +++ /dev/null @@ -1,118 +0,0 @@ -# -# rt-mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal thread to signal (0-7) -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 4 threads 2 lock PI -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedother: 0: 0 -C: schedother: 1: 0 -C: schedfifo: 2: 82 -C: schedfifo: 3: 83 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L1 -C: locknowait: 1: 1 -W: locked: 1: 1 - -# T3 lock L0 -C: lockintnowait: 3: 0 -W: blocked: 3: 0 -T: prioeq: 0: 83 - -# T0 lock L1 -C: lock: 0: 1 -W: blocked: 0: 1 -T: prioeq: 1: 83 - -# T1 unlock L1 -C: unlock: 1: 1 - -# Wait until T0 is in the wakeup code -W: blockedwake: 0: 1 - -# Verify that T1 is unboosted -W: unlocked: 1: 1 -T: priolt: 1: 1 - -# T2 lock L1 (T0 is boosted and pending owner !) -C: locknowait: 2: 1 -W: blocked: 2: 1 -T: prioeq: 0: 83 - -# Interrupt T3 and wait until T3 returned -C: signal: 3: 0 -W: unlocked: 3: 0 - -# Verify prio of T0 (still pending owner, -# but T2 is enqueued due to the previous boost by T3 -T: prioeq: 0: 82 - -# Let T0 continue -C: lockcont: 0: 1 -W: locked: 0: 1 - -# Unlock L1 and let T2 get L1 -C: unlock: 0: 1 -W: locked: 2: 1 - -# Verify that T0 is unboosted -W: unlocked: 0: 1 -T: priolt: 0: 1 - -# Unlock everything and exit -C: unlock: 2: 1 -W: unlocked: 2: 1 - -C: unlock: 0: 0 -W: unlocked: 0: 0 - diff --git a/scripts/rt-tester/t5-l4-pi-boost-deboost-setsched.tst b/scripts/rt-tester/t5-l4-pi-boost-deboost-setsched.tst deleted file mode 100644 index 04f4034ff895a1..00000000000000 --- a/scripts/rt-tester/t5-l4-pi-boost-deboost-setsched.tst +++ /dev/null @@ -1,178 +0,0 @@ -# -# rt-mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal thread to signal (0-7) -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 5 threads 4 lock PI - modify priority of blocked threads -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedother: 0: 0 -C: schedfifo: 1: 81 -C: schedfifo: 2: 82 -C: schedfifo: 3: 83 -C: schedfifo: 4: 84 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L1 -C: locknowait: 1: 1 -W: locked: 1: 1 - -# T1 lock L0 -C: lockintnowait: 1: 0 -W: blocked: 1: 0 -T: prioeq: 0: 81 - -# T2 lock L2 -C: locknowait: 2: 2 -W: locked: 2: 2 - -# T2 lock L1 -C: lockintnowait: 2: 1 -W: blocked: 2: 1 -T: prioeq: 0: 82 -T: prioeq: 1: 82 - -# T3 lock L3 -C: locknowait: 3: 3 -W: locked: 3: 3 - -# T3 lock L2 -C: lockintnowait: 3: 2 -W: blocked: 3: 2 -T: prioeq: 0: 83 -T: prioeq: 1: 83 -T: prioeq: 2: 83 - -# T4 lock L3 -C: lockintnowait: 4: 3 -W: blocked: 4: 3 -T: prioeq: 0: 84 -T: prioeq: 1: 84 -T: prioeq: 2: 84 -T: prioeq: 3: 84 - -# Reduce prio of T4 -C: schedfifo: 4: 80 -T: prioeq: 0: 83 -T: prioeq: 1: 83 -T: prioeq: 2: 83 -T: prioeq: 3: 83 -T: prioeq: 4: 80 - -# Increase prio of T4 -C: schedfifo: 4: 84 -T: prioeq: 0: 84 -T: prioeq: 1: 84 -T: prioeq: 2: 84 -T: prioeq: 3: 84 -T: prioeq: 4: 84 - -# Reduce prio of T3 -C: schedfifo: 3: 80 -T: prioeq: 0: 84 -T: prioeq: 1: 84 -T: prioeq: 2: 84 -T: prioeq: 3: 84 -T: prioeq: 4: 84 - -# Increase prio of T3 -C: schedfifo: 3: 85 -T: prioeq: 0: 85 -T: prioeq: 1: 85 -T: prioeq: 2: 85 -T: prioeq: 3: 85 -T: prioeq: 4: 84 - -# Reduce prio of T3 -C: schedfifo: 3: 83 -T: prioeq: 0: 84 -T: prioeq: 1: 84 -T: prioeq: 2: 84 -T: prioeq: 3: 84 -T: prioeq: 4: 84 - -# Signal T4 -C: signal: 4: 0 -W: unlocked: 4: 3 -T: prioeq: 0: 83 -T: prioeq: 1: 83 -T: prioeq: 2: 83 -T: prioeq: 3: 83 - -# Signal T3 -C: signal: 3: 0 -W: unlocked: 3: 2 -T: prioeq: 0: 82 -T: prioeq: 1: 82 -T: prioeq: 2: 82 - -# Signal T2 -C: signal: 2: 0 -W: unlocked: 2: 1 -T: prioeq: 0: 81 -T: prioeq: 1: 81 - -# Signal T1 -C: signal: 1: 0 -W: unlocked: 1: 0 -T: priolt: 0: 1 - -# Unlock and exit -C: unlock: 3: 3 -C: unlock: 2: 2 -C: unlock: 1: 1 -C: unlock: 0: 0 - -W: unlocked: 3: 3 -W: unlocked: 2: 2 -W: unlocked: 1: 1 -W: unlocked: 0: 0 - diff --git a/scripts/rt-tester/t5-l4-pi-boost-deboost.tst b/scripts/rt-tester/t5-l4-pi-boost-deboost.tst deleted file mode 100644 index a48a6ee29ddcf1..00000000000000 --- a/scripts/rt-tester/t5-l4-pi-boost-deboost.tst +++ /dev/null @@ -1,138 +0,0 @@ -# -# rt-mutex test -# -# Op: C(ommand)/T(est)/W(ait) -# | opcode -# | | threadid: 0-7 -# | | | opcode argument -# | | | | -# C: lock: 0: 0 -# -# Commands -# -# opcode opcode argument -# schedother nice value -# schedfifo priority -# lock lock nr (0-7) -# locknowait lock nr (0-7) -# lockint lock nr (0-7) -# lockintnowait lock nr (0-7) -# lockcont lock nr (0-7) -# unlock lock nr (0-7) -# signal thread to signal (0-7) -# reset 0 -# resetevent 0 -# -# Tests / Wait -# -# opcode opcode argument -# -# prioeq priority -# priolt priority -# priogt priority -# nprioeq normal priority -# npriolt normal priority -# npriogt normal priority -# locked lock nr (0-7) -# blocked lock nr (0-7) -# blockedwake lock nr (0-7) -# unlocked lock nr (0-7) -# opcodeeq command opcode or number -# opcodelt number -# opcodegt number -# eventeq number -# eventgt number -# eventlt number - -# -# 5 threads 4 lock PI -# -C: resetevent: 0: 0 -W: opcodeeq: 0: 0 - -# Set schedulers -C: schedother: 0: 0 -C: schedfifo: 1: 81 -C: schedfifo: 2: 82 -C: schedfifo: 3: 83 -C: schedfifo: 4: 84 - -# T0 lock L0 -C: locknowait: 0: 0 -W: locked: 0: 0 - -# T1 lock L1 -C: locknowait: 1: 1 -W: locked: 1: 1 - -# T1 lock L0 -C: lockintnowait: 1: 0 -W: blocked: 1: 0 -T: prioeq: 0: 81 - -# T2 lock L2 -C: locknowait: 2: 2 -W: locked: 2: 2 - -# T2 lock L1 -C: lockintnowait: 2: 1 -W: blocked: 2: 1 -T: prioeq: 0: 82 -T: prioeq: 1: 82 - -# T3 lock L3 -C: locknowait: 3: 3 -W: locked: 3: 3 - -# T3 lock L2 -C: lockintnowait: 3: 2 -W: blocked: 3: 2 -T: prioeq: 0: 83 -T: prioeq: 1: 83 -T: prioeq: 2: 83 - -# T4 lock L3 -C: lockintnowait: 4: 3 -W: blocked: 4: 3 -T: prioeq: 0: 84 -T: prioeq: 1: 84 -T: prioeq: 2: 84 -T: prioeq: 3: 84 - -# Signal T4 -C: signal: 4: 0 -W: unlocked: 4: 3 -T: prioeq: 0: 83 -T: prioeq: 1: 83 -T: prioeq: 2: 83 -T: prioeq: 3: 83 - -# Signal T3 -C: signal: 3: 0 -W: unlocked: 3: 2 -T: prioeq: 0: 82 -T: prioeq: 1: 82 -T: prioeq: 2: 82 - -# Signal T2 -C: signal: 2: 0 -W: unlocked: 2: 1 -T: prioeq: 0: 81 -T: prioeq: 1: 81 - -# Signal T1 -C: signal: 1: 0 -W: unlocked: 1: 0 -T: priolt: 0: 1 - -# Unlock and exit -C: unlock: 3: 3 -C: unlock: 2: 2 -C: unlock: 1: 1 -C: unlock: 0: 0 - -W: unlocked: 3: 3 -W: unlocked: 2: 2 -W: unlocked: 1: 1 -W: unlocked: 0: 0 - diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 4fa4bc4505f5b5..4252fc22f78f4f 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -880,15 +880,26 @@ struct bpf_object *bpf_object__open(const char *path) } struct bpf_object *bpf_object__open_buffer(void *obj_buf, - size_t obj_buf_sz) + size_t obj_buf_sz, + const char *name) { + char tmp_name[64]; + /* param validation */ if (!obj_buf || obj_buf_sz <= 0) return NULL; - pr_debug("loading object from buffer\n"); + if (!name) { + snprintf(tmp_name, sizeof(tmp_name), "%lx-%lx", + (unsigned long)obj_buf, + (unsigned long)obj_buf_sz); + tmp_name[sizeof(tmp_name) - 1] = '\0'; + name = tmp_name; + } + pr_debug("loading object '%s' from buffer\n", + name); - return __bpf_object__open("[buffer]", obj_buf, obj_buf_sz); + return __bpf_object__open(name, obj_buf, obj_buf_sz); } int bpf_object__unload(struct bpf_object *obj) @@ -975,6 +986,14 @@ bpf_object__next(struct bpf_object *prev) return next; } +const char * +bpf_object__get_name(struct bpf_object *obj) +{ + if (!obj) + return NULL; + return obj->path; +} + struct bpf_program * bpf_program__next(struct bpf_program *prev, struct bpf_object *obj) { diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index ea8adc206b6242..f16170c95ffdc2 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -28,12 +28,14 @@ struct bpf_object; struct bpf_object *bpf_object__open(const char *path); struct bpf_object *bpf_object__open_buffer(void *obj_buf, - size_t obj_buf_sz); + size_t obj_buf_sz, + const char *name); void bpf_object__close(struct bpf_object *object); /* Load/unload object into/from kernel */ int bpf_object__load(struct bpf_object *obj); int bpf_object__unload(struct bpf_object *obj); +const char *bpf_object__get_name(struct bpf_object *obj); struct bpf_object *bpf_object__next(struct bpf_object *prev); #define bpf_object__for_each_safe(pos, tmp) \ diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt index 347a27322ed813..2e9ce77b5e144a 100644 --- a/tools/perf/Documentation/perf-record.txt +++ b/tools/perf/Documentation/perf-record.txt @@ -276,7 +276,11 @@ filter out the startup phase of the program, which is often very different. --intr-regs:: Capture machine state (registers) at interrupt, i.e., on counter overflows for each sample. List of captured registers depends on the architecture. This option -is off by default. +is off by default. It is possible to select the registers to sample using their +symbolic names, e.g. on x86, ax, si. To list the available registers use +--intr-regs=\?. To name registers, pass a comma separated list such as +--intr-regs=ax,bx. The list of register is architecture dependent. + --running-time:: Record running and enabled time for read events (:S) diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt index 614b2c7b0293ef..dc3ec783b7bd50 100644 --- a/tools/perf/Documentation/perf-script.txt +++ b/tools/perf/Documentation/perf-script.txt @@ -116,7 +116,7 @@ OPTIONS --fields:: Comma separated list of fields to print. Options are: comm, tid, pid, time, cpu, event, trace, ip, sym, dso, addr, symoff, - srcline, period, flags. + srcline, period, iregs, flags. Field list can be prepended with the type, trace, sw or hw, to indicate to which event type the field list applies. e.g., -f sw:comm,tid,time,ip,sym and -f trace:time,cpu,trace diff --git a/tools/perf/arch/sh/util/dwarf-regs.c b/tools/perf/arch/sh/util/dwarf-regs.c index 0d0897f57a1027..f8dfa89696f42e 100644 --- a/tools/perf/arch/sh/util/dwarf-regs.c +++ b/tools/perf/arch/sh/util/dwarf-regs.c @@ -51,5 +51,5 @@ const char *sh_regs_table[SH_MAX_REGS] = { /* Return architecture dependent register string (for kprobe-tracer) */ const char *get_arch_regstr(unsigned int n) { - return (n <= SH_MAX_REGS) ? sh_regs_table[n] : NULL; + return (n < SH_MAX_REGS) ? sh_regs_table[n] : NULL; } diff --git a/tools/perf/arch/sparc/util/dwarf-regs.c b/tools/perf/arch/sparc/util/dwarf-regs.c index 92eda412fed343..b704fdb9237a95 100644 --- a/tools/perf/arch/sparc/util/dwarf-regs.c +++ b/tools/perf/arch/sparc/util/dwarf-regs.c @@ -39,5 +39,5 @@ const char *sparc_regs_table[SPARC_MAX_REGS] = { */ const char *get_arch_regstr(unsigned int n) { - return (n <= SPARC_MAX_REGS) ? sparc_regs_table[n] : NULL; + return (n < SPARC_MAX_REGS) ? sparc_regs_table[n] : NULL; } diff --git a/tools/perf/arch/x86/util/Build b/tools/perf/arch/x86/util/Build index 2c55e1b336c53f..ff63649fa9ac60 100644 --- a/tools/perf/arch/x86/util/Build +++ b/tools/perf/arch/x86/util/Build @@ -2,6 +2,7 @@ libperf-y += header.o libperf-y += tsc.o libperf-y += pmu.o libperf-y += kvm-stat.o +libperf-y += perf_regs.o libperf-$(CONFIG_DWARF) += dwarf-regs.o diff --git a/tools/perf/arch/x86/util/dwarf-regs.c b/tools/perf/arch/x86/util/dwarf-regs.c index be22dd463232ed..a08de0a35b8384 100644 --- a/tools/perf/arch/x86/util/dwarf-regs.c +++ b/tools/perf/arch/x86/util/dwarf-regs.c @@ -71,5 +71,5 @@ const char *x86_64_regs_table[X86_64_MAX_REGS] = { /* Return architecture dependent register string (for kprobe-tracer) */ const char *get_arch_regstr(unsigned int n) { - return (n <= ARCH_MAX_REGS) ? arch_regs_table[n] : NULL; + return (n < ARCH_MAX_REGS) ? arch_regs_table[n] : NULL; } diff --git a/tools/perf/arch/x86/util/perf_regs.c b/tools/perf/arch/x86/util/perf_regs.c new file mode 100644 index 00000000000000..c5db14f36cc7a1 --- /dev/null +++ b/tools/perf/arch/x86/util/perf_regs.c @@ -0,0 +1,28 @@ +#include "../../perf.h" +#include "../../util/perf_regs.h" + +const struct sample_reg sample_reg_masks[] = { + SMPL_REG(AX, PERF_REG_X86_AX), + SMPL_REG(BX, PERF_REG_X86_BX), + SMPL_REG(CX, PERF_REG_X86_CX), + SMPL_REG(DX, PERF_REG_X86_DX), + SMPL_REG(SI, PERF_REG_X86_SI), + SMPL_REG(DI, PERF_REG_X86_DI), + SMPL_REG(BP, PERF_REG_X86_BP), + SMPL_REG(SP, PERF_REG_X86_SP), + SMPL_REG(IP, PERF_REG_X86_IP), + SMPL_REG(FLAGS, PERF_REG_X86_FLAGS), + SMPL_REG(CS, PERF_REG_X86_CS), + SMPL_REG(SS, PERF_REG_X86_SS), +#ifdef HAVE_ARCH_X86_64_SUPPORT + SMPL_REG(R8, PERF_REG_X86_R8), + SMPL_REG(R9, PERF_REG_X86_R9), + SMPL_REG(R10, PERF_REG_X86_R10), + SMPL_REG(R11, PERF_REG_X86_R11), + SMPL_REG(R12, PERF_REG_X86_R12), + SMPL_REG(R13, PERF_REG_X86_R13), + SMPL_REG(R14, PERF_REG_X86_R14), + SMPL_REG(R15, PERF_REG_X86_R15), +#endif + SMPL_REG_END +}; diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index a660022f2c929c..142eeb341b295a 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -27,8 +27,10 @@ #include "util/cpumap.h" #include "util/thread_map.h" #include "util/data.h" +#include "util/perf_regs.h" #include "util/auxtrace.h" #include "util/parse-branch-options.h" +#include "util/parse-regs-options.h" #include #include @@ -279,7 +281,7 @@ static int record__open(struct record *rec) evlist__for_each(evlist, pos) { try_again: - if (perf_evsel__open(pos, evlist->cpus, evlist->threads) < 0) { + if (perf_evsel__open(pos, pos->cpus, pos->threads) < 0) { if (perf_evsel__fallback(pos, errno, msg, sizeof(msg))) { if (verbose) ui__warning("%s\n", msg); @@ -1080,8 +1082,9 @@ struct option __record_options[] = { "sample transaction flags (special events only)"), OPT_BOOLEAN(0, "per-thread", &record.opts.target.per_thread, "use per-thread mmaps"), - OPT_BOOLEAN('I', "intr-regs", &record.opts.sample_intr_regs, - "Sample machine registers on interrupt"), + OPT_CALLBACK_OPTARG('I', "intr-regs", &record.opts.sample_intr_regs, NULL, "any register", + "sample selected machine registers on interrupt," + " use -I ? to list register names", parse_regs), OPT_BOOLEAN(0, "running-time", &record.opts.running_time, "Record running/enabled time of read (:S) events"), OPT_CALLBACK('k', "clockid", &record.opts, diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c index 4430340292c052..eb51325e8ad99a 100644 --- a/tools/perf/builtin-script.c +++ b/tools/perf/builtin-script.c @@ -6,6 +6,7 @@ #include "util/exec_cmd.h" #include "util/header.h" #include "util/parse-options.h" +#include "util/perf_regs.h" #include "util/session.h" #include "util/tool.h" #include "util/symbol.h" @@ -46,6 +47,7 @@ enum perf_output_field { PERF_OUTPUT_SYMOFFSET = 1U << 11, PERF_OUTPUT_SRCLINE = 1U << 12, PERF_OUTPUT_PERIOD = 1U << 13, + PERF_OUTPUT_IREGS = 1U << 14, }; struct output_option { @@ -66,6 +68,7 @@ struct output_option { {.str = "symoff", .field = PERF_OUTPUT_SYMOFFSET}, {.str = "srcline", .field = PERF_OUTPUT_SRCLINE}, {.str = "period", .field = PERF_OUTPUT_PERIOD}, + {.str = "iregs", .field = PERF_OUTPUT_IREGS}, }; /* default set to maintain compatibility with current format */ @@ -255,6 +258,11 @@ static int perf_evsel__check_attr(struct perf_evsel *evsel, PERF_OUTPUT_PERIOD)) return -EINVAL; + if (PRINT_FIELD(IREGS) && + perf_evsel__check_stype(evsel, PERF_SAMPLE_REGS_INTR, "IREGS", + PERF_OUTPUT_IREGS)) + return -EINVAL; + return 0; } @@ -352,6 +360,24 @@ static int perf_session__check_output_opt(struct perf_session *session) return 0; } +static void print_sample_iregs(union perf_event *event __maybe_unused, + struct perf_sample *sample, + struct thread *thread __maybe_unused, + struct perf_event_attr *attr) +{ + struct regs_dump *regs = &sample->intr_regs; + uint64_t mask = attr->sample_regs_intr; + unsigned i = 0, r; + + if (!regs) + return; + + for_each_set_bit(r, (unsigned long *) &mask, sizeof(mask) * 8) { + u64 val = regs->regs[i++]; + printf("%5s:0x%"PRIx64" ", perf_reg_name(r), val); + } +} + static void print_sample_start(struct perf_sample *sample, struct thread *thread, struct perf_evsel *evsel) @@ -525,6 +551,9 @@ static void process_event(union perf_event *event, struct perf_sample *sample, PERF_MAX_STACK_DEPTH); } + if (PRINT_FIELD(IREGS)) + print_sample_iregs(event, sample, thread, attr); + printf("\n"); } @@ -1643,7 +1672,7 @@ int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused) "comma separated output fields prepend with 'type:'. " "Valid types: hw,sw,trace,raw. " "Fields: comm,tid,pid,time,cpu,event,trace,ip,sym,dso," - "addr,symoff,period,flags", parse_output_fields), + "addr,symoff,period,iregs,flags", parse_output_fields), OPT_BOOLEAN('a', "all-cpus", &system_wide, "system-wide collection from all CPUs"), OPT_STRING('S', "symbols", &symbol_conf.sym_list_str, "symbol[,symbol...]", diff --git a/tools/perf/perf.h b/tools/perf/perf.h index cccb4cf575d3af..90129accffbe82 100644 --- a/tools/perf/perf.h +++ b/tools/perf/perf.h @@ -54,7 +54,6 @@ struct record_opts { bool sample_time_set; bool callgraph_set; bool period; - bool sample_intr_regs; bool running_time; bool full_auxtrace; bool auxtrace_snapshot_mode; @@ -64,6 +63,7 @@ struct record_opts { unsigned int auxtrace_mmap_pages; unsigned int user_freq; u64 branch_stack; + u64 sample_intr_regs; u64 default_interval; u64 user_interval; size_t auxtrace_snapshot_size; diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c index a337356fd97900..52d55971f66f6f 100644 --- a/tools/perf/tests/llvm.c +++ b/tools/perf/tests/llvm.c @@ -26,7 +26,7 @@ static int test__bpf_parsing(void *obj_buf, size_t obj_buf_sz) { struct bpf_object *obj; - obj = bpf_object__open_buffer(obj_buf, obj_buf_sz); + obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, NULL); if (!obj) return -1; bpf_object__close(obj); diff --git a/tools/perf/util/Build b/tools/perf/util/Build index e912856cc4e5d6..349bc96ca1fedc 100644 --- a/tools/perf/util/Build +++ b/tools/perf/util/Build @@ -75,6 +75,7 @@ libperf-y += record.o libperf-y += srcline.o libperf-y += data.o libperf-$(CONFIG_X86) += tsc.o +libperf-$(CONFIG_AUXTRACE) += tsc.o libperf-y += cloexec.o libperf-y += thread-stack.o libperf-$(CONFIG_AUXTRACE) += auxtrace.o @@ -82,6 +83,7 @@ libperf-$(CONFIG_AUXTRACE) += intel-pt-decoder/ libperf-$(CONFIG_AUXTRACE) += intel-pt.o libperf-$(CONFIG_AUXTRACE) += intel-bts.o libperf-y += parse-branch-options.o +libperf-y += parse-regs-options.o libperf-$(CONFIG_LIBELF) += symbol-elf.o libperf-$(CONFIG_LIBELF) += probe-file.o diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c index 8d00039d6a20ab..d51a5200c8af77 100644 --- a/tools/perf/util/evlist.c +++ b/tools/perf/util/evlist.c @@ -1181,6 +1181,10 @@ int perf_evlist__apply_filters(struct perf_evlist *evlist, struct perf_evsel **e if (evsel->filter == NULL) continue; + /* + * filters only work for tracepoint event, which doesn't have cpu limit. + * So evlist and evsel should always be same. + */ err = perf_evsel__apply_filter(evsel, ncpus, nthreads, evsel->filter); if (err) { *err_evsel = evsel; diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index bac25f41a7518d..c53f79123b37f4 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -787,7 +787,7 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts) perf_evsel__config_callgraph(evsel, opts, &callchain_param); if (opts->sample_intr_regs) { - attr->sample_regs_intr = PERF_REGS_MASK; + attr->sample_regs_intr = opts->sample_intr_regs; perf_evsel__set_sample_bit(evsel, REGS_INTR); } diff --git a/tools/perf/util/intel-pt-decoder/Build b/tools/perf/util/intel-pt-decoder/Build index 240730d682c1b6..2386322ece4fc8 100644 --- a/tools/perf/util/intel-pt-decoder/Build +++ b/tools/perf/util/intel-pt-decoder/Build @@ -4,6 +4,7 @@ inat_tables_script = util/intel-pt-decoder/gen-insn-attr-x86.awk inat_tables_maps = util/intel-pt-decoder/x86-opcode-map.txt $(OUTPUT)util/intel-pt-decoder/inat-tables.c: $(inat_tables_script) $(inat_tables_maps) + $(call rule_mkdir) @$(call echo-cmd,gen)$(AWK) -f $(inat_tables_script) $(inat_tables_maps) > $@ || rm -f $@ $(OUTPUT)util/intel-pt-decoder/intel-pt-insn-decoder.o: util/intel-pt-decoder/inat.c $(OUTPUT)util/intel-pt-decoder/inat-tables.c diff --git a/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c b/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c index 9e4eb8fcd5592c..d23138c06665a8 100644 --- a/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c +++ b/tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c @@ -146,6 +146,9 @@ static void intel_pt_insn_decoder(struct insn *insn, case 4: intel_pt_insn->rel = bswap_32(insn->immediate.value); break; + default: + intel_pt_insn->rel = 0; + break; } #else intel_pt_insn->rel = insn->immediate.value; diff --git a/tools/perf/util/parse-regs-options.c b/tools/perf/util/parse-regs-options.c new file mode 100644 index 00000000000000..4f2c1c255d818b --- /dev/null +++ b/tools/perf/util/parse-regs-options.c @@ -0,0 +1,71 @@ +#include "perf.h" +#include "util/util.h" +#include "util/debug.h" +#include "util/parse-options.h" +#include "util/parse-regs-options.h" + +int +parse_regs(const struct option *opt, const char *str, int unset) +{ + uint64_t *mode = (uint64_t *)opt->value; + const struct sample_reg *r; + char *s, *os = NULL, *p; + int ret = -1; + + if (unset) + return 0; + + /* + * cannot set it twice + */ + if (*mode) + return -1; + + /* str may be NULL in case no arg is passed to -I */ + if (str) { + /* because str is read-only */ + s = os = strdup(str); + if (!s) + return -1; + + for (;;) { + p = strchr(s, ','); + if (p) + *p = '\0'; + + if (!strcmp(s, "?")) { + fprintf(stderr, "available registers: "); + for (r = sample_reg_masks; r->name; r++) { + fprintf(stderr, "%s ", r->name); + } + fputc('\n', stderr); + /* just printing available regs */ + return -1; + } + for (r = sample_reg_masks; r->name; r++) { + if (!strcasecmp(s, r->name)) + break; + } + if (!r->name) { + ui__warning("unknown register %s," + " check man page\n", s); + goto error; + } + + *mode |= r->mask; + + if (!p) + break; + + s = p + 1; + } + } + ret = 0; + + /* default to all possible regs */ + if (*mode == 0) + *mode = PERF_REGS_MASK; +error: + free(os); + return ret; +} diff --git a/tools/perf/util/parse-regs-options.h b/tools/perf/util/parse-regs-options.h new file mode 100644 index 00000000000000..7d762b188007fb --- /dev/null +++ b/tools/perf/util/parse-regs-options.h @@ -0,0 +1,5 @@ +#ifndef _PERF_PARSE_REGS_OPTIONS_H +#define _PERF_PARSE_REGS_OPTIONS_H 1 +struct option; +int parse_regs(const struct option *opt, const char *str, int unset); +#endif /* _PERF_PARSE_REGS_OPTIONS_H */ diff --git a/tools/perf/util/perf_regs.c b/tools/perf/util/perf_regs.c index 43168fb0d9a28a..885e8ac8399790 100644 --- a/tools/perf/util/perf_regs.c +++ b/tools/perf/util/perf_regs.c @@ -2,6 +2,10 @@ #include "perf_regs.h" #include "event.h" +const struct sample_reg __weak sample_reg_masks[] = { + SMPL_REG_END +}; + int perf_reg_value(u64 *valp, struct regs_dump *regs, int id) { int i, idx = 0; diff --git a/tools/perf/util/perf_regs.h b/tools/perf/util/perf_regs.h index 980dbf76bc9841..2984dcc54d67cd 100644 --- a/tools/perf/util/perf_regs.h +++ b/tools/perf/util/perf_regs.h @@ -5,6 +5,15 @@ struct regs_dump; +struct sample_reg { + const char *name; + uint64_t mask; +}; +#define SMPL_REG(n, b) { .name = #n, .mask = 1ULL << (b) } +#define SMPL_REG_END { .name = NULL } + +extern const struct sample_reg sample_reg_masks[]; + #ifdef HAVE_PERF_REGS_SUPPORT #include diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 24ae9e829e9aa2..b8f12e0897e6b1 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -20,6 +20,7 @@ ifneq (1, $(quicktest)) TARGETS += timers endif TARGETS += user +TARGETS += jumplabel TARGETS += vm TARGETS += x86 #Please keep the TARGETS list alphabetically sorted diff --git a/tools/testing/selftests/powerpc/mm/Makefile b/tools/testing/selftests/powerpc/mm/Makefile index 41cc3ed66818bf..ee179e22308c59 100644 --- a/tools/testing/selftests/powerpc/mm/Makefile +++ b/tools/testing/selftests/powerpc/mm/Makefile @@ -2,8 +2,9 @@ noarg: $(MAKE) -C ../ TEST_PROGS := hugetlb_vs_thp_test subpage_prot +TEST_FILES := tempfile -all: $(TEST_PROGS) tempfile +all: $(TEST_PROGS) $(TEST_FILES) $(TEST_PROGS): ../harness.c diff --git a/tools/testing/selftests/seccomp/seccomp_bpf.c b/tools/testing/selftests/seccomp/seccomp_bpf.c index c5abe7fd7590e3..a004b4cce99ef3 100644 --- a/tools/testing/selftests/seccomp/seccomp_bpf.c +++ b/tools/testing/selftests/seccomp/seccomp_bpf.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -82,7 +83,13 @@ struct seccomp_data { }; #endif +#if __BYTE_ORDER == __LITTLE_ENDIAN #define syscall_arg(_n) (offsetof(struct seccomp_data, args[_n])) +#elif __BYTE_ORDER == __BIG_ENDIAN +#define syscall_arg(_n) (offsetof(struct seccomp_data, args[_n]) + sizeof(__u32)) +#else +#error "wut? Unknown __BYTE_ORDER?!" +#endif #define SIBLING_EXIT_UNKILLED 0xbadbeef #define SIBLING_EXIT_FAILURE 0xbadface @@ -1199,6 +1206,10 @@ TEST_F(TRACE_poke, getpid_runs_normally) # define ARCH_REGS struct user_pt_regs # define SYSCALL_NUM regs[8] # define SYSCALL_RET regs[0] +#elif defined(__powerpc__) +# define ARCH_REGS struct pt_regs +# define SYSCALL_NUM gpr[0] +# define SYSCALL_RET gpr[3] #else # error "Do not know how to find your architecture's registers and syscalls" #endif @@ -1232,7 +1243,7 @@ void change_syscall(struct __test_metadata *_metadata, ret = ptrace(PTRACE_GETREGSET, tracee, NT_PRSTATUS, &iov); EXPECT_EQ(0, ret); -#if defined(__x86_64__) || defined(__i386__) || defined(__aarch64__) +#if defined(__x86_64__) || defined(__i386__) || defined(__aarch64__) || defined(__powerpc__) { regs.SYSCALL_NUM = syscall; } @@ -1396,6 +1407,8 @@ TEST_F(TRACE_syscall, syscall_dropped) # define __NR_seccomp 383 # elif defined(__aarch64__) # define __NR_seccomp 277 +# elif defined(__powerpc__) +# define __NR_seccomp 358 # else # warning "seccomp syscall number unknown for this architecture" # define __NR_seccomp 0xffff diff --git a/tools/testing/selftests/static_keys/Makefile b/tools/testing/selftests/static_keys/Makefile new file mode 100644 index 00000000000000..9cdadf37f1144b --- /dev/null +++ b/tools/testing/selftests/static_keys/Makefile @@ -0,0 +1,8 @@ +# Makefile for static keys selftests + +# No binaries, but make sure arg-less "make" doesn't trigger "run_tests" +all: + +TEST_PROGS := test_static_keys.sh + +include ../lib.mk diff --git a/tools/testing/selftests/static_keys/test_static_keys.sh b/tools/testing/selftests/static_keys/test_static_keys.sh new file mode 100644 index 00000000000000..1261e3fa1e3a85 --- /dev/null +++ b/tools/testing/selftests/static_keys/test_static_keys.sh @@ -0,0 +1,16 @@ +#!/bin/sh +# Runs static keys kernel module tests + +if /sbin/modprobe -q test_static_key_base; then + if /sbin/modprobe -q test_static_keys; then + echo "static_key: ok" + /sbin/modprobe -q -r test_static_keys + /sbin/modprobe -q -r test_static_key_base + else + echo "static_keys: [FAIL]" + /sbin/modprobe -q -r test_static_key_base + fi +else + echo "static_key: [FAIL]" + exit 1 +fi