Skip to content

Commit 15b760c

Browse files
andraprsgregkh
andraprs
authored and
gregkh
committed
nitro_enclaves: Add ioctl interface definition
The Nitro Enclaves driver handles the enclave lifetime management. This includes enclave creation, termination and setting up its resources such as memory and CPU. An enclave runs alongside the VM that spawned it. It is abstracted as a process running in the VM that launched it. The process interacts with the NE driver, that exposes an ioctl interface for creating an enclave and setting up its resources. Changelog v9 -> v10 * Update commit message to include the changelog before the SoB tag(s). v8 -> v9 * No changes. v7 -> v8 * Add NE custom error codes for user space memory regions not backed by pages multiple of 2 MiB, invalid flags and enclave CID. * Add max flag value for enclave image load info. v6 -> v7 * Clarify in the ioctls documentation that the return value is -1 and errno is set on failure. * Update the error code value for NE_ERR_INVALID_MEM_REGION_SIZE as it gets in user space as value 25 (ENOTTY) instead of 515. Update the NE custom error codes values range to not be the same as the ones defined in include/linux/errno.h, although these are not propagated to user space. v5 -> v6 * Fix typo in the description about the NE CPU pool. * Update documentation to kernel-doc format. * Remove the ioctl to query API version. v4 -> v5 * Add more details about the ioctl calls usage e.g. error codes, file descriptors used. * Update the ioctl to set an enclave vCPU to not return a file descriptor. * Add specific NE error codes. v3 -> v4 * Decouple NE ioctl interface from KVM API. * Add NE API version and the corresponding ioctl call. * Add enclave / image load flags options. v2 -> v3 * Remove the GPL additional wording as SPDX-License-Identifier is already in place. v1 -> v2 * Add ioctl for getting enclave image load metadata. * Update NE_ENCLAVE_START ioctl name to NE_START_ENCLAVE. * Add entry in Documentation/userspace-api/ioctl/ioctl-number.rst for NE ioctls. * Update NE ioctls definition based on the updated ioctl range for major and minor. Reviewed-by: Alexander Graf <graf@amazon.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Alexandru Vasile <lexnv@amazon.com> Signed-off-by: Andra Paraschiv <andraprs@amazon.com> Link: https://lore.kernel.org/r/20200921121732.44291-2-andraprs@amazon.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
1 parent 8fd0e2a commit 15b760c

File tree

3 files changed

+374
-1
lines changed

3 files changed

+374
-1
lines changed

‎Documentation/userspace-api/ioctl/ioctl-number.rst‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -328,8 +328,11 @@ Code Seq# Include File Comments
328328
0xAC 00-1F linux/raw.h
329329
0xAD 00 Netfilter device in development:
330330
<mailto:rusty@rustcorp.com.au>
331-
0xAE all linux/kvm.h Kernel-based Virtual Machine
331+
0xAE 00-1F linux/kvm.h Kernel-based Virtual Machine
332332
<mailto:kvm@vger.kernel.org>
333+
0xAE 40-FF linux/kvm.h Kernel-based Virtual Machine
334+
<mailto:kvm@vger.kernel.org>
335+
0xAE 20-3F linux/nitro_enclaves.h Nitro Enclaves
333336
0xAF 00-1F linux/fsl_hypervisor.h Freescale hypervisor
334337
0xB0 all RATIO devices in development:
335338
<mailto:vgo@ratio.de>

‎include/linux/nitro_enclaves.h‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
/* SPDX-License-Identifier: GPL-2.0 */
2+
/*
3+
* Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
4+
*/
5+
6+
#ifndef _LINUX_NITRO_ENCLAVES_H_
7+
#define _LINUX_NITRO_ENCLAVES_H_
8+
9+
#include <uapi/linux/nitro_enclaves.h>
10+
11+
#endif /* _LINUX_NITRO_ENCLAVES_H_ */

‎include/uapi/linux/nitro_enclaves.h‎

Lines changed: 359 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,359 @@
1+
/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
2+
/*
3+
* Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
4+
*/
5+
6+
#ifndef _UAPI_LINUX_NITRO_ENCLAVES_H_
7+
#define _UAPI_LINUX_NITRO_ENCLAVES_H_
8+
9+
#include <linux/types.h>
10+
11+
/**
12+
* DOC: Nitro Enclaves (NE) Kernel Driver Interface
13+
*/
14+
15+
/**
16+
* NE_CREATE_VM - The command is used to create a slot that is associated with
17+
* an enclave VM.
18+
* The generated unique slot id is an output parameter.
19+
* The ioctl can be invoked on the /dev/nitro_enclaves fd, before
20+
* setting any resources, such as memory and vCPUs, for an
21+
* enclave. Memory and vCPUs are set for the slot mapped to an enclave.
22+
* A NE CPU pool has to be set before calling this function. The
23+
* pool can be set after the NE driver load, using
24+
* /sys/module/nitro_enclaves/parameters/ne_cpus.
25+
* Its format is the detailed in the cpu-lists section:
26+
* https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html
27+
* CPU 0 and its siblings have to remain available for the
28+
* primary / parent VM, so they cannot be set for enclaves. Full
29+
* CPU core(s), from the same NUMA node, need(s) to be included
30+
* in the CPU pool.
31+
*
32+
* Context: Process context.
33+
* Return:
34+
* * Enclave file descriptor - Enclave file descriptor used with
35+
* ioctl calls to set vCPUs and memory
36+
* regions, then start the enclave.
37+
* * -1 - There was a failure in the ioctl logic.
38+
* On failure, errno is set to:
39+
* * EFAULT - copy_to_user() failure.
40+
* * ENOMEM - Memory allocation failure for internal
41+
* bookkeeping variables.
42+
* * NE_ERR_NO_CPUS_AVAIL_IN_POOL - No NE CPU pool set / no CPUs available
43+
* in the pool.
44+
* * Error codes from get_unused_fd_flags() and anon_inode_getfile().
45+
* * Error codes from the NE PCI device request.
46+
*/
47+
#define NE_CREATE_VM _IOR(0xAE, 0x20, __u64)
48+
49+
/**
50+
* NE_ADD_VCPU - The command is used to set a vCPU for an enclave. The vCPU can
51+
* be auto-chosen from the NE CPU pool or it can be set by the
52+
* caller, with the note that it needs to be available in the NE
53+
* CPU pool. Full CPU core(s), from the same NUMA node, need(s) to
54+
* be associated with an enclave.
55+
* The vCPU id is an input / output parameter. If its value is 0,
56+
* then a CPU is chosen from the enclave CPU pool and returned via
57+
* this parameter.
58+
* The ioctl can be invoked on the enclave fd, before an enclave
59+
* is started.
60+
*
61+
* Context: Process context.
62+
* Return:
63+
* * 0 - Logic succesfully completed.
64+
* * -1 - There was a failure in the ioctl logic.
65+
* On failure, errno is set to:
66+
* * EFAULT - copy_from_user() / copy_to_user() failure.
67+
* * ENOMEM - Memory allocation failure for internal
68+
* bookkeeping variables.
69+
* * EIO - Current task mm is not the same as the one
70+
* that created the enclave.
71+
* * NE_ERR_NO_CPUS_AVAIL_IN_POOL - No CPUs available in the NE CPU pool.
72+
* * NE_ERR_VCPU_ALREADY_USED - The provided vCPU is already used.
73+
* * NE_ERR_VCPU_NOT_IN_CPU_POOL - The provided vCPU is not available in the
74+
* NE CPU pool.
75+
* * NE_ERR_VCPU_INVALID_CPU_CORE - The core id of the provided vCPU is invalid
76+
* or out of range.
77+
* * NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state
78+
* (init = before being started).
79+
* * NE_ERR_INVALID_VCPU - The provided vCPU is not in the available
80+
* CPUs range.
81+
* * Error codes from the NE PCI device request.
82+
*/
83+
#define NE_ADD_VCPU _IOWR(0xAE, 0x21, __u32)
84+
85+
/**
86+
* NE_GET_IMAGE_LOAD_INFO - The command is used to get information needed for
87+
* in-memory enclave image loading e.g. offset in
88+
* enclave memory to start placing the enclave image.
89+
* The image load info is an input / output parameter.
90+
* It includes info provided by the caller - flags -
91+
* and returns the offset in enclave memory where to
92+
* start placing the enclave image.
93+
* The ioctl can be invoked on the enclave fd, before
94+
* an enclave is started.
95+
*
96+
* Context: Process context.
97+
* Return:
98+
* * 0 - Logic succesfully completed.
99+
* * -1 - There was a failure in the ioctl logic.
100+
* On failure, errno is set to:
101+
* * EFAULT - copy_from_user() / copy_to_user() failure.
102+
* * NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state (init =
103+
* before being started).
104+
* * NE_ERR_INVALID_FLAG_VALUE - The value of the provided flag is invalid.
105+
*/
106+
#define NE_GET_IMAGE_LOAD_INFO _IOWR(0xAE, 0x22, struct ne_image_load_info)
107+
108+
/**
109+
* NE_SET_USER_MEMORY_REGION - The command is used to set a memory region for an
110+
* enclave, given the allocated memory from the
111+
* userspace. Enclave memory needs to be from the
112+
* same NUMA node as the enclave CPUs.
113+
* The user memory region is an input parameter. It
114+
* includes info provided by the caller - flags,
115+
* memory size and userspace address.
116+
* The ioctl can be invoked on the enclave fd,
117+
* before an enclave is started.
118+
*
119+
* Context: Process context.
120+
* Return:
121+
* * 0 - Logic succesfully completed.
122+
* * -1 - There was a failure in the ioctl logic.
123+
* On failure, errno is set to:
124+
* * EFAULT - copy_from_user() failure.
125+
* * EINVAL - Invalid physical memory region(s) e.g.
126+
* unaligned address.
127+
* * EIO - Current task mm is not the same as
128+
* the one that created the enclave.
129+
* * ENOMEM - Memory allocation failure for internal
130+
* bookkeeping variables.
131+
* * NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state
132+
* (init = before being started).
133+
* * NE_ERR_INVALID_MEM_REGION_SIZE - The memory size of the region is not
134+
* multiple of 2 MiB.
135+
* * NE_ERR_INVALID_MEM_REGION_ADDR - Invalid user space address given.
136+
* * NE_ERR_UNALIGNED_MEM_REGION_ADDR - Unaligned user space address given.
137+
* * NE_ERR_MEM_REGION_ALREADY_USED - The memory region is already used.
138+
* * NE_ERR_MEM_NOT_HUGE_PAGE - The memory region is not backed by
139+
* huge pages.
140+
* * NE_ERR_MEM_DIFFERENT_NUMA_NODE - The memory region is not from the same
141+
* NUMA node as the CPUs.
142+
* * NE_ERR_MEM_MAX_REGIONS - The number of memory regions set for
143+
* the enclave reached maximum.
144+
* * NE_ERR_INVALID_PAGE_SIZE - The memory region is not backed by
145+
* pages multiple of 2 MiB.
146+
* * NE_ERR_INVALID_FLAG_VALUE - The value of the provided flag is invalid.
147+
* * Error codes from get_user_pages().
148+
* * Error codes from the NE PCI device request.
149+
*/
150+
#define NE_SET_USER_MEMORY_REGION _IOW(0xAE, 0x23, struct ne_user_memory_region)
151+
152+
/**
153+
* NE_START_ENCLAVE - The command is used to trigger enclave start after the
154+
* enclave resources, such as memory and CPU, have been set.
155+
* The enclave start info is an input / output parameter. It
156+
* includes info provided by the caller - enclave cid and
157+
* flags - and returns the cid (if input cid is 0).
158+
* The ioctl can be invoked on the enclave fd, after an
159+
* enclave slot is created and resources, such as memory and
160+
* vCPUs are set for an enclave.
161+
*
162+
* Context: Process context.
163+
* Return:
164+
* * 0 - Logic succesfully completed.
165+
* * -1 - There was a failure in the ioctl logic.
166+
* On failure, errno is set to:
167+
* * EFAULT - copy_from_user() / copy_to_user() failure.
168+
* * NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state
169+
* (init = before being started).
170+
* * NE_ERR_NO_MEM_REGIONS_ADDED - No memory regions are set.
171+
* * NE_ERR_NO_VCPUS_ADDED - No vCPUs are set.
172+
* * NE_ERR_FULL_CORES_NOT_USED - Full core(s) not set for the enclave.
173+
* * NE_ERR_ENCLAVE_MEM_MIN_SIZE - Enclave memory is less than minimum
174+
* memory size (64 MiB).
175+
* * NE_ERR_INVALID_FLAG_VALUE - The value of the provided flag is invalid.
176+
* * NE_ERR_INVALID_ENCLAVE_CID - The provided enclave CID is invalid.
177+
* * Error codes from the NE PCI device request.
178+
*/
179+
#define NE_START_ENCLAVE _IOWR(0xAE, 0x24, struct ne_enclave_start_info)
180+
181+
/**
182+
* DOC: NE specific error codes
183+
*/
184+
185+
/**
186+
* NE_ERR_VCPU_ALREADY_USED - The provided vCPU is already used.
187+
*/
188+
#define NE_ERR_VCPU_ALREADY_USED (256)
189+
/**
190+
* NE_ERR_VCPU_NOT_IN_CPU_POOL - The provided vCPU is not available in the
191+
* NE CPU pool.
192+
*/
193+
#define NE_ERR_VCPU_NOT_IN_CPU_POOL (257)
194+
/**
195+
* NE_ERR_VCPU_INVALID_CPU_CORE - The core id of the provided vCPU is invalid
196+
* or out of range of the NE CPU pool.
197+
*/
198+
#define NE_ERR_VCPU_INVALID_CPU_CORE (258)
199+
/**
200+
* NE_ERR_INVALID_MEM_REGION_SIZE - The user space memory region size is not
201+
* multiple of 2 MiB.
202+
*/
203+
#define NE_ERR_INVALID_MEM_REGION_SIZE (259)
204+
/**
205+
* NE_ERR_INVALID_MEM_REGION_ADDR - The user space memory region address range
206+
* is invalid.
207+
*/
208+
#define NE_ERR_INVALID_MEM_REGION_ADDR (260)
209+
/**
210+
* NE_ERR_UNALIGNED_MEM_REGION_ADDR - The user space memory region address is
211+
* not aligned.
212+
*/
213+
#define NE_ERR_UNALIGNED_MEM_REGION_ADDR (261)
214+
/**
215+
* NE_ERR_MEM_REGION_ALREADY_USED - The user space memory region is already used.
216+
*/
217+
#define NE_ERR_MEM_REGION_ALREADY_USED (262)
218+
/**
219+
* NE_ERR_MEM_NOT_HUGE_PAGE - The user space memory region is not backed by
220+
* contiguous physical huge page(s).
221+
*/
222+
#define NE_ERR_MEM_NOT_HUGE_PAGE (263)
223+
/**
224+
* NE_ERR_MEM_DIFFERENT_NUMA_NODE - The user space memory region is backed by
225+
* pages from different NUMA nodes than the CPUs.
226+
*/
227+
#define NE_ERR_MEM_DIFFERENT_NUMA_NODE (264)
228+
/**
229+
* NE_ERR_MEM_MAX_REGIONS - The supported max memory regions per enclaves has
230+
* been reached.
231+
*/
232+
#define NE_ERR_MEM_MAX_REGIONS (265)
233+
/**
234+
* NE_ERR_NO_MEM_REGIONS_ADDED - The command to start an enclave is triggered
235+
* and no memory regions are added.
236+
*/
237+
#define NE_ERR_NO_MEM_REGIONS_ADDED (266)
238+
/**
239+
* NE_ERR_NO_VCPUS_ADDED - The command to start an enclave is triggered and no
240+
* vCPUs are added.
241+
*/
242+
#define NE_ERR_NO_VCPUS_ADDED (267)
243+
/**
244+
* NE_ERR_ENCLAVE_MEM_MIN_SIZE - The enclave memory size is lower than the
245+
* minimum supported.
246+
*/
247+
#define NE_ERR_ENCLAVE_MEM_MIN_SIZE (268)
248+
/**
249+
* NE_ERR_FULL_CORES_NOT_USED - The command to start an enclave is triggered and
250+
* full CPU cores are not set.
251+
*/
252+
#define NE_ERR_FULL_CORES_NOT_USED (269)
253+
/**
254+
* NE_ERR_NOT_IN_INIT_STATE - The enclave is not in init state when setting
255+
* resources or triggering start.
256+
*/
257+
#define NE_ERR_NOT_IN_INIT_STATE (270)
258+
/**
259+
* NE_ERR_INVALID_VCPU - The provided vCPU is out of range of the available CPUs.
260+
*/
261+
#define NE_ERR_INVALID_VCPU (271)
262+
/**
263+
* NE_ERR_NO_CPUS_AVAIL_IN_POOL - The command to create an enclave is triggered
264+
* and no CPUs are available in the pool.
265+
*/
266+
#define NE_ERR_NO_CPUS_AVAIL_IN_POOL (272)
267+
/**
268+
* NE_ERR_INVALID_PAGE_SIZE - The user space memory region is not backed by pages
269+
* multiple of 2 MiB.
270+
*/
271+
#define NE_ERR_INVALID_PAGE_SIZE (273)
272+
/**
273+
* NE_ERR_INVALID_FLAG_VALUE - The provided flag value is invalid.
274+
*/
275+
#define NE_ERR_INVALID_FLAG_VALUE (274)
276+
/**
277+
* NE_ERR_INVALID_ENCLAVE_CID - The provided enclave CID is invalid, either
278+
* being a well-known value or the CID of the
279+
* parent / primary VM.
280+
*/
281+
#define NE_ERR_INVALID_ENCLAVE_CID (275)
282+
283+
/**
284+
* DOC: Image load info flags
285+
*/
286+
287+
/**
288+
* NE_EIF_IMAGE - Enclave Image Format (EIF)
289+
*/
290+
#define NE_EIF_IMAGE (0x01)
291+
292+
#define NE_IMAGE_LOAD_MAX_FLAG_VAL (0x02)
293+
294+
/**
295+
* struct ne_image_load_info - Info necessary for in-memory enclave image
296+
* loading (in / out).
297+
* @flags: Flags to determine the enclave image type
298+
* (e.g. Enclave Image Format - EIF) (in).
299+
* @memory_offset: Offset in enclave memory where to start placing the
300+
* enclave image (out).
301+
*/
302+
struct ne_image_load_info {
303+
__u64 flags;
304+
__u64 memory_offset;
305+
};
306+
307+
/**
308+
* DOC: User memory region flags
309+
*/
310+
311+
/**
312+
* NE_DEFAULT_MEMORY_REGION - Memory region for enclave general usage.
313+
*/
314+
#define NE_DEFAULT_MEMORY_REGION (0x00)
315+
316+
#define NE_MEMORY_REGION_MAX_FLAG_VAL (0x01)
317+
318+
/**
319+
* struct ne_user_memory_region - Memory region to be set for an enclave (in).
320+
* @flags: Flags to determine the usage for the memory region (in).
321+
* @memory_size: The size, in bytes, of the memory region to be set for
322+
* an enclave (in).
323+
* @userspace_addr: The start address of the userspace allocated memory of
324+
* the memory region to set for an enclave (in).
325+
*/
326+
struct ne_user_memory_region {
327+
__u64 flags;
328+
__u64 memory_size;
329+
__u64 userspace_addr;
330+
};
331+
332+
/**
333+
* DOC: Enclave start info flags
334+
*/
335+
336+
/**
337+
* NE_ENCLAVE_PRODUCTION_MODE - Start enclave in production mode.
338+
*/
339+
#define NE_ENCLAVE_PRODUCTION_MODE (0x00)
340+
/**
341+
* NE_ENCLAVE_DEBUG_MODE - Start enclave in debug mode.
342+
*/
343+
#define NE_ENCLAVE_DEBUG_MODE (0x01)
344+
345+
#define NE_ENCLAVE_START_MAX_FLAG_VAL (0x02)
346+
347+
/**
348+
* struct ne_enclave_start_info - Setup info necessary for enclave start (in / out).
349+
* @flags: Flags for the enclave to start with (e.g. debug mode) (in).
350+
* @enclave_cid: Context ID (CID) for the enclave vsock device. If 0 as
351+
* input, the CID is autogenerated by the hypervisor and
352+
* returned back as output by the driver (in / out).
353+
*/
354+
struct ne_enclave_start_info {
355+
__u64 flags;
356+
__u64 enclave_cid;
357+
};
358+
359+
#endif /* _UAPI_LINUX_NITRO_ENCLAVES_H_ */

0 commit comments

Comments
 (0)