Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

idpf: fix error handling in soft_reset for XDP #26

Open
wants to merge 53 commits into
base: idpf-libie-new
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
ce4e251
netdevice: convert private flags > BIT(31) to bitfields
alobakin Jun 21, 2024
c414a62
netdev_features: remove unused __UNUSED_NETIF_F_1
alobakin Apr 4, 2024
32298a1
netdev_features: convert NETIF_F_LLTX to dev->lltx
alobakin Jun 25, 2024
3309bfa
netdev_features: convert NETIF_F_NETNS_LOCAL to dev->netns_local
alobakin Jun 25, 2024
881404c
netdev_features: convert NETIF_F_FCOE_MTU to dev->fcoe_mtu
alobakin Jun 25, 2024
296fa4d
netdev_features: remove NETIF_F_ALL_FCOE
alobakin Jul 5, 2024
ea2ebdd
firmware/psci: fix missing '%u' format literal in kthread_create_on_c…
alobakin Aug 14, 2024
e468dae
kthread: allow vararg kthread_{create,run}_on_cpu()
alobakin Aug 13, 2024
16f6271
net: napi: add ability to create CPU-pinned threaded NAPI
LorenzoBianconi Aug 14, 2024
4f13a3d
bpf: cpumap: use CPU-pinned threaded NAPI instead of kthread
LorenzoBianconi Aug 14, 2024
562765d
bpf: cpumap: reuse skb array instead of a linked list to chain skbs
alobakin Aug 14, 2024
1e03973
net: skbuff: introduce napi_skb_cache_get_bulk()
alobakin Apr 11, 2022
6123354
bpf: cpumap: switch to napi_skb_cache_get_bulk()
alobakin Apr 11, 2022
05688b0
unroll: add generic loop unroll helpers
alobakin Feb 12, 2024
6871b10
libeth: add common queue stats
alobakin Mar 14, 2023
10db7ad
libie: add Tx buffer completion helpers
alobakin Dec 13, 2023
4a41337
idpf: convert to libie Tx buffer completion
alobakin Dec 13, 2023
48f03f8
netdevice: add netdev_tx_reset_subqueue() shorthand
alobakin Apr 15, 2024
0f202ef
idpf: refactor Tx completion routines
jahay1 Mar 20, 2024
0ed9c5e
idpf: fix netdev Tx queue stop/wake
michalQb Mar 22, 2024
08d9d2f
idpf: enable WB_ON_ITR
jahay1 Dec 15, 2023
9f7618d
idpf: switch to libeth generic statistics
alobakin Apr 16, 2024
b6f12cc
bpf, xdp: constify some bpf_prog * function arguments
alobakin Dec 11, 2023
615fb28
xdp, xsk: constify read-only arguments of some static inline helpers
alobakin Dec 6, 2023
0362ad2
xdp: allow attaching already registered memory model to xdp_rxq_info
alobakin Dec 19, 2023
37b168c
net: Register system page pool as an XDP memory model
tohojo Feb 20, 2024
1e04ade
page_pool: make page_pool_put_page_bulk() actually handle array of pages
alobakin Feb 5, 2024
543144c
page_pool: allow mixing PPs within one bulk
alobakin Dec 7, 2023
eff896b
xdp: get rid of xdp_frame::mem.id
alobakin Dec 7, 2023
da334dc
xdp: add generic xdp_buff_add_frag()
alobakin Dec 6, 2023
d66dbef
xdp: add generic xdp_build_skb_from_buff()
alobakin Dec 6, 2023
929fc4f
xsk: allow attaching XSk pool via xdp_rxq_info_reg_mem_model()
alobakin Jan 31, 2024
4f780d0
xsk: make xsk_buff_add_frag really add a frag via __xdp_buff_add_frag()
alobakin Jan 30, 2024
8b95564
xsk: add generic XSk &xdp_buff -> skb conversion
alobakin Dec 22, 2023
6da01b8
xsk: add helper to get &xdp_desc's DMA and meta pointer in one go
alobakin Mar 25, 2024
be60eb4
skbuff: allow 2-4-argument skb_frag_dma_map()
alobakin Apr 5, 2024
978e7ba
jump_label: export static_key_slow_{inc,dec}_cpuslocked()
alobakin Dec 15, 2023
f5ef048
libeth: support native XDP and register memory model
alobakin Dec 8, 2023
ad65916
libeth: add a couple of XDP helpers (libeth_xdp)
alobakin Dec 11, 2023
ed92bda
idpf: make complq cleaning dependent on scheduling mode
michalQb Oct 20, 2023
ea48a26
idpf: remove SW marker handling from NAPI
michalQb Feb 23, 2024
aefdfc9
idpf: prepare structures to support xdp
michalQb Oct 3, 2023
35d653a
idpf: implement XDP_SETUP_PROG in ndo_bpf for splitq
michalQb Oct 4, 2023
617c3f2
idpf: use generic functions to build xdp_buff and skb
alobakin Dec 6, 2023
7d8d843
idpf: add support for XDP on Rx
alobakin Dec 12, 2023
d901a20
idpf: add support for .ndo_xdp_xmit()
alobakin Oct 30, 2023
acb41d4
idpf: add XDP RSS hash hint
alobakin Mar 8, 2024
d179d25
idpf: add vc functions to manage selected queues
michalQb Nov 17, 2023
45e2844
idpf: add XSk pool initialization
michalQb Nov 27, 2023
de6d9bd
idpf: implement Tx path for AF_XDP
alobakin Dec 7, 2023
fbb9691
idpf: implement Rx path for AF_XDP
alobakin Dec 12, 2023
289b8d1
idpf: enable XSk features and ndo_xsk_wakeup
alobakin Dec 6, 2023
1801366
idpf: fix error handling in soft_reset for XDP
michalQb Aug 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion Documentation/networking/net_cachelines/net_device.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ net_device struct fast path usage breakdown

Type Name fastpath_tx_access fastpath_rx_access Comments
..struct ..net_device
unsigned_long:32 priv_flags read_mostly - __dev_queue_xmit(tx)
unsigned_long:1 lltx read_mostly - HARD_TX_LOCK,HARD_TX_TRYLOCK,HARD_TX_UNLOCK(tx)
char name[16] - -
struct_netdev_name_node* name_node
struct_dev_ifalias* ifalias
Expand All @@ -23,7 +25,6 @@ struct_list_head ptype_specific
struct adj_list
unsigned_int flags read_mostly read_mostly __dev_queue_xmit,__dev_xmit_skb,ip6_output,__ip6_finish_output(tx);ip6_rcv_core(rx)
xdp_features_t xdp_features
unsigned_long_long priv_flags read_mostly - __dev_queue_xmit(tx)
struct_net_device_ops* netdev_ops read_mostly - netdev_core_pick_tx,netdev_start_xmit(tx)
struct_xdp_metadata_ops* xdp_metadata_ops
int ifindex - read_mostly ip6_rcv_core
Expand Down Expand Up @@ -163,6 +164,10 @@ struct_lock_class_key* qdisc_tx_busylock
bool proto_down
unsigned:1 wol_enabled
unsigned:1 threaded - - napi_poll(napi_enable,dev_set_threaded)
unsigned_long:1 see_all_hwtstamp_requests
unsigned_long:1 change_proto_down
unsigned_long:1 netns_local
unsigned_long:1 fcoe_mtu
struct_list_head net_notifier_list
struct_macsec_ops* macsec_ops
struct_udp_tunnel_nic_info* udp_tunnel_nic_info
Expand Down
15 changes: 0 additions & 15 deletions Documentation/networking/netdev-features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -139,21 +139,6 @@ chained skbs (skb->next/prev list).
Features contained in NETIF_F_SOFT_FEATURES are features of networking
stack. Driver should not change behaviour based on them.

* LLTX driver (deprecated for hardware drivers)

NETIF_F_LLTX is meant to be used by drivers that don't need locking at all,
e.g. software tunnels.

This is also used in a few legacy drivers that implement their
own locking, don't use it for new (hardware) drivers.

* netns-local device

NETIF_F_NETNS_LOCAL is set for devices that are not allowed to move between
network namespaces (e.g. loopback).

Don't use it in drivers.

* VLAN challenged

NETIF_F_VLAN_CHALLENGED should be set for devices which can't cope with VLAN
Expand Down
4 changes: 2 additions & 2 deletions Documentation/networking/netdevices.rst
Original file line number Diff line number Diff line change
Expand Up @@ -258,11 +258,11 @@ ndo_get_stats:
ndo_start_xmit:
Synchronization: __netif_tx_lock spinlock.

When the driver sets NETIF_F_LLTX in dev->features this will be
When the driver sets dev->lltx this will be
called without holding netif_tx_lock. In this case the driver
has to lock by itself when needed.
The locking there should also properly protect against
set_rx_mode. WARNING: use of NETIF_F_LLTX is deprecated.
set_rx_mode. WARNING: use of dev->lltx is deprecated.
Don't use it for new drivers.

Context: Process with BHs disabled or BH (timer),
Expand Down
4 changes: 2 additions & 2 deletions Documentation/networking/switchdev.rst
Original file line number Diff line number Diff line change
Expand Up @@ -137,10 +137,10 @@ would be sub-port 0 on port 1 on switch 1.
Port Features
^^^^^^^^^^^^^

NETIF_F_NETNS_LOCAL
dev->netns_local

If the switchdev driver (and device) only supports offloading of the default
network namespace (netns), the driver should set this feature flag to prevent
network namespace (netns), the driver should set this private flag to prevent
the port netdev from being moved out of the default netns. A netns-aware
driver/device would not set this flag and be responsible for partitioning
hardware to preserve netns containment. This means hardware cannot forward
Expand Down
2 changes: 1 addition & 1 deletion drivers/firmware/psci/psci_checker.c
Original file line number Diff line number Diff line change
Expand Up @@ -398,7 +398,7 @@ static int suspend_tests(void)

thread = kthread_create_on_cpu(suspend_test_thread,
(void *)(long)cpu, cpu,
"psci_suspend_test");
"psci_suspend_test-%u");
if (IS_ERR(thread))
pr_err("Failed to create kthread on CPU %d\n", cpu);
else
Expand Down
4 changes: 2 additions & 2 deletions drivers/net/amt.c
Original file line number Diff line number Diff line change
Expand Up @@ -3098,9 +3098,9 @@ static void amt_link_setup(struct net_device *dev)
dev->hard_header_len = 0;
dev->addr_len = 0;
dev->priv_flags |= IFF_NO_QUEUE;
dev->features |= NETIF_F_LLTX;
dev->lltx = true;
dev->netns_local = true;
dev->features |= NETIF_F_GSO_SOFTWARE;
dev->features |= NETIF_F_NETNS_LOCAL;
dev->hw_features |= NETIF_F_SG | NETIF_F_HW_CSUM;
dev->hw_features |= NETIF_F_FRAGLIST | NETIF_F_RXCSUM;
dev->hw_features |= NETIF_F_GSO_SOFTWARE;
Expand Down
2 changes: 1 addition & 1 deletion drivers/net/bareudp.c
Original file line number Diff line number Diff line change
Expand Up @@ -553,7 +553,6 @@ static void bareudp_setup(struct net_device *dev)
SET_NETDEV_DEVTYPE(dev, &bareudp_type);
dev->features |= NETIF_F_SG | NETIF_F_HW_CSUM | NETIF_F_FRAGLIST;
dev->features |= NETIF_F_RXCSUM;
dev->features |= NETIF_F_LLTX;
dev->features |= NETIF_F_GSO_SOFTWARE;
dev->hw_features |= NETIF_F_SG | NETIF_F_HW_CSUM | NETIF_F_FRAGLIST;
dev->hw_features |= NETIF_F_RXCSUM;
Expand All @@ -566,6 +565,7 @@ static void bareudp_setup(struct net_device *dev)
dev->type = ARPHRD_NONE;
netif_keep_dst(dev);
dev->priv_flags |= IFF_NO_QUEUE;
dev->lltx = true;
dev->flags = IFF_POINTOPOINT | IFF_NOARP | IFF_MULTICAST;
dev->pcpu_stat_type = NETDEV_PCPU_STAT_TSTATS;
}
Expand Down
8 changes: 4 additions & 4 deletions drivers/net/bonding/bond_main.c
Original file line number Diff line number Diff line change
Expand Up @@ -5886,7 +5886,10 @@ void bond_setup(struct net_device *bond_dev)
#endif /* CONFIG_XFRM_OFFLOAD */

/* don't acquire bond device's netif_tx_lock when transmitting */
bond_dev->features |= NETIF_F_LLTX;
bond_dev->lltx = true;

/* Don't allow bond devices to change network namespaces. */
bond_dev->netns_local = true;

/* By default, we declare the bond to be fully
* VLAN hardware accelerated capable. Special
Expand All @@ -5895,9 +5898,6 @@ void bond_setup(struct net_device *bond_dev)
* capable
*/

/* Don't allow bond devices to change network namespaces. */
bond_dev->features |= NETIF_F_NETNS_LOCAL;

bond_dev->hw_features = BOND_VLAN_FEATURES |
NETIF_F_HW_VLAN_CTAG_RX |
NETIF_F_HW_VLAN_CTAG_FILTER |
Expand Down
3 changes: 2 additions & 1 deletion drivers/net/dummy.c
Original file line number Diff line number Diff line change
Expand Up @@ -109,9 +109,10 @@ static void dummy_setup(struct net_device *dev)
dev->flags |= IFF_NOARP;
dev->flags &= ~IFF_MULTICAST;
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE;
dev->lltx = true;
dev->features |= NETIF_F_SG | NETIF_F_FRAGLIST;
dev->features |= NETIF_F_GSO_SOFTWARE;
dev->features |= NETIF_F_HW_CSUM | NETIF_F_HIGHDMA | NETIF_F_LLTX;
dev->features |= NETIF_F_HW_CSUM | NETIF_F_HIGHDMA;
dev->features |= NETIF_F_GSO_ENCAP_ALL;
dev->hw_features |= dev->features;
dev->hw_enc_features |= dev->features;
Expand Down
2 changes: 1 addition & 1 deletion drivers/net/ethernet/adi/adin1110.c
Original file line number Diff line number Diff line change
Expand Up @@ -1599,7 +1599,7 @@ static int adin1110_probe_netdevs(struct adin1110_priv *priv)
netdev->netdev_ops = &adin1110_netdev_ops;
netdev->ethtool_ops = &adin1110_ethtool_ops;
netdev->priv_flags |= IFF_UNICAST_FLT;
netdev->features |= NETIF_F_NETNS_LOCAL;
netdev->netns_local = true;

port_priv->phydev = get_phy_device(priv->mii_bus, i + 1, false);
if (IS_ERR(port_priv->phydev)) {
Expand Down
3 changes: 2 additions & 1 deletion drivers/net/ethernet/chelsio/cxgb/cxgb2.c
Original file line number Diff line number Diff line change
Expand Up @@ -1034,7 +1034,8 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
netdev->hw_features |= NETIF_F_SG | NETIF_F_IP_CSUM |
NETIF_F_RXCSUM;
netdev->features |= NETIF_F_SG | NETIF_F_IP_CSUM |
NETIF_F_RXCSUM | NETIF_F_LLTX | NETIF_F_HIGHDMA;
NETIF_F_RXCSUM | NETIF_F_HIGHDMA;
netdev->lltx = true;

if (vlan_tso_capable(adapter)) {
netdev->features |=
Expand Down
6 changes: 2 additions & 4 deletions drivers/net/ethernet/chelsio/cxgb4/cxgb4_fcoe.c
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,7 @@ int cxgb_fcoe_enable(struct net_device *netdev)

netdev->features |= NETIF_F_FCOE_CRC;
netdev->vlan_features |= NETIF_F_FCOE_CRC;
netdev->features |= NETIF_F_FCOE_MTU;
netdev->vlan_features |= NETIF_F_FCOE_MTU;
netdev->fcoe_mtu = true;

netdev_features_change(netdev);

Expand Down Expand Up @@ -112,8 +111,7 @@ int cxgb_fcoe_disable(struct net_device *netdev)

netdev->features &= ~NETIF_F_FCOE_CRC;
netdev->vlan_features &= ~NETIF_F_FCOE_CRC;
netdev->features &= ~NETIF_F_FCOE_MTU;
netdev->vlan_features &= ~NETIF_F_FCOE_MTU;
netdev->fcoe_mtu = false;

netdev_features_change(netdev);

Expand Down
5 changes: 3 additions & 2 deletions drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ static int dpaa_netdev_init(struct net_device *net_dev,
net_dev->max_mtu = dpaa_get_max_mtu();

net_dev->hw_features |= (NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
NETIF_F_LLTX | NETIF_F_RXHASH);
NETIF_F_RXHASH);

net_dev->hw_features |= NETIF_F_SG | NETIF_F_HIGHDMA;
/* The kernels enables GSO automatically, if we declare NETIF_F_SG.
Expand All @@ -239,6 +239,7 @@ static int dpaa_netdev_init(struct net_device *net_dev,
net_dev->features |= NETIF_F_RXCSUM;

net_dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
net_dev->lltx = true;
/* we do not want shared skbs on TX */
net_dev->priv_flags &= ~IFF_TX_SKB_SHARING;

Expand Down Expand Up @@ -2258,7 +2259,7 @@ static int dpaa_a050385_wa_xdpf(struct dpaa_priv *priv,
new_xdpf->len = xdpf->len;
new_xdpf->headroom = priv->tx_headroom;
new_xdpf->frame_sz = DPAA_BP_RAW_SIZE;
new_xdpf->mem.type = MEM_TYPE_PAGE_ORDER0;
new_xdpf->mem_type = MEM_TYPE_PAGE_ORDER0;

/* Release the initial buffer */
xdp_return_frame_rx_napi(xdpf);
Expand Down
3 changes: 2 additions & 1 deletion drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
Original file line number Diff line number Diff line change
Expand Up @@ -4594,12 +4594,13 @@ static int dpaa2_eth_netdev_init(struct net_device *net_dev)

net_dev->priv_flags |= supported;
net_dev->priv_flags &= ~not_supported;
net_dev->lltx = true;

/* Features */
net_dev->features = NETIF_F_RXCSUM |
NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
NETIF_F_SG | NETIF_F_HIGHDMA |
NETIF_F_LLTX | NETIF_F_HW_TC | NETIF_F_TSO;
NETIF_F_HW_TC | NETIF_F_TSO;
net_dev->gso_max_segs = DPAA2_ETH_ENQUEUE_MAX_FDS;
net_dev->hw_features = net_dev->features;
net_dev->xdp_features = NETDEV_XDP_ACT_BASIC |
Expand Down
30 changes: 3 additions & 27 deletions drivers/net/ethernet/intel/i40e/i40e_xsk.c
Original file line number Diff line number Diff line change
Expand Up @@ -395,32 +395,6 @@ static void i40e_handle_xdp_result_zc(struct i40e_ring *rx_ring,
WARN_ON_ONCE(1);
}

static int
i40e_add_xsk_frag(struct i40e_ring *rx_ring, struct xdp_buff *first,
struct xdp_buff *xdp, const unsigned int size)
{
struct skb_shared_info *sinfo = xdp_get_shared_info_from_buff(first);

if (!xdp_buff_has_frags(first)) {
sinfo->nr_frags = 0;
sinfo->xdp_frags_size = 0;
xdp_buff_set_frags_flag(first);
}

if (unlikely(sinfo->nr_frags == MAX_SKB_FRAGS)) {
xsk_buff_free(first);
return -ENOMEM;
}

__skb_fill_page_desc_noacc(sinfo, sinfo->nr_frags++,
virt_to_page(xdp->data_hard_start),
XDP_PACKET_HEADROOM, size);
sinfo->xdp_frags_size += size;
xsk_buff_add_frag(xdp);

return 0;
}

/**
* i40e_clean_rx_irq_zc - Consumes Rx packets from the hardware ring
* @rx_ring: Rx ring
Expand Down Expand Up @@ -486,8 +460,10 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget)

if (!first)
first = bi;
else if (i40e_add_xsk_frag(rx_ring, first, bi, size))
else if (!xsk_buff_add_frag(first, bi)) {
xsk_buff_free(first);
break;
}

if (++next_to_process == count)
next_to_process = 0;
Expand Down
32 changes: 2 additions & 30 deletions drivers/net/ethernet/intel/ice/ice_xsk.c
Original file line number Diff line number Diff line change
Expand Up @@ -809,35 +809,6 @@ ice_run_xdp_zc(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp,
return result;
}

static int
ice_add_xsk_frag(struct ice_rx_ring *rx_ring, struct xdp_buff *first,
struct xdp_buff *xdp, const unsigned int size)
{
struct skb_shared_info *sinfo = xdp_get_shared_info_from_buff(first);

if (!size)
return 0;

if (!xdp_buff_has_frags(first)) {
sinfo->nr_frags = 0;
sinfo->xdp_frags_size = 0;
xdp_buff_set_frags_flag(first);
}

if (unlikely(sinfo->nr_frags == MAX_SKB_FRAGS)) {
xsk_buff_free(first);
return -ENOMEM;
}

__skb_fill_page_desc_noacc(sinfo, sinfo->nr_frags++,
virt_to_page(xdp->data_hard_start),
XDP_PACKET_HEADROOM, size);
sinfo->xdp_frags_size += size;
xsk_buff_add_frag(xdp);

return 0;
}

/**
* ice_clean_rx_irq_zc - consumes packets from the hardware ring
* @rx_ring: AF_XDP Rx ring
Expand Down Expand Up @@ -903,7 +874,8 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring,

if (!first) {
first = xdp;
} else if (ice_add_xsk_frag(rx_ring, first, xdp, size)) {
} else if (likely(size) && !xsk_buff_add_frag(first, xdp)) {
xsk_buff_free(first);
break;
}

Expand Down
2 changes: 1 addition & 1 deletion drivers/net/ethernet/intel/idpf/Kconfig
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ config IDPF
tristate "Intel(R) Infrastructure Data Path Function Support"
depends on PCI_MSI
select DIMLIB
select LIBETH
select LIBETH_XDP
help
This driver supports Intel(R) Infrastructure Data Path Function
devices.
Expand Down
3 changes: 3 additions & 0 deletions drivers/net/ethernet/intel/idpf/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,6 @@ idpf-y := \
idpf_vf_dev.o

idpf-$(CONFIG_IDPF_SINGLEQ) += idpf_singleq_txrx.o

idpf-y += xdp.o
idpf-y += xsk.o
Loading