Commit 7fa95f9a authored by Nicholas Piggin's avatar Nicholas Piggin Committed by Michael Ellerman
Browse files

powerpc/64s: system call support for scv/rfscv instructions 



Add support for the scv instruction on POWER9 and later CPUs.

For now this implements the zeroth scv vector 'scv 0', as identical to
'sc' system calls, with the exception that LR is not preserved, nor
are volatile CR registers, and error is not indicated with CR0[SO],
but by returning a negative errno.

rfscv is implemented to return from scv type system calls. It can not
be used to return from sc system calls because those are defined to
preserve LR.

getpid syscall throughput on POWER9 is improved by 26% (428 to 318
cycles), largely due to reducing mtmsr and mtspr.

Signed-off-by: default avatarNicholas Piggin <npiggin@gmail.com>
[mpe: Fix ppc64e build]
Signed-off-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20200611081203.995112-3-npiggin@gmail.com
parent b2dc2977

Documentation/powerpc/syscall64-abi.rst

+30 −12
Original line number Diff line number Diff line
@@ -5,6 +5,15 @@ Power Architecture 64-bit Linux system call ABI 
syscall

=======



Invocation

----------

The syscall is made with the sc instruction, and returns with execution

continuing at the instruction following the sc instruction.



If PPC_FEATURE2_SCV appears in the AT_HWCAP2 ELF auxiliary vector, the

scv 0 instruction is an alternative that may provide better performance,

with some differences to calling sequence.



syscall calling sequence\ [1]_ matches the Power Architecture 64-bit ELF ABI

specification C function calling sequence, including register preservation

rules, with the following differences.
@@ -12,16 +21,23 @@ rules, with the following differences. 
.. [1] Some syscalls (typically low-level management functions) may have

       different calling sequences (e.g., rt_sigreturn).



Parameters and return value

---------------------------

Parameters

----------

The system call number is specified in r0.



There is a maximum of 6 integer parameters to a syscall, passed in r3-r8.



Both a return value and a return error code are returned. cr0.SO is the return

error code, and r3 is the return value or error code. When cr0.SO is clear,

the syscall succeeded and r3 is the return value. When cr0.SO is set, the

syscall failed and r3 is the error code that generally corresponds to errno.

Return value

------------

- For the sc instruction, both a value and an error condition are returned.

  cr0.SO is the error condition, and r3 is the return value. When cr0.SO is

  clear, the syscall succeeded and r3 is the return value. When cr0.SO is set,

  the syscall failed and r3 is the error value (that normally corresponds to

  errno).



- For the scv 0 instruction, the return value indicates failure if it is

  -4095..-1 (i.e., it is >= -MAX_ERRNO (-4095) as an unsigned comparison),

  in which case the error value is the negated return value.



Stack

-----
@@ -34,22 +50,23 @@ Register preservation rules match the ELF ABI calling sequence with the 
following differences:



=========== ============= ========================================

--- For the sc instruction, differences with the ELF ABI ---

r0          Volatile      (System call number.)

r3          Volatile      (Parameter 1, and return value.)

r4-r8       Volatile      (Parameters 2-6.)

cr0         Volatile      (cr0.SO is the return error condition)

cr0         Volatile      (cr0.SO is the return error condition.)

cr1, cr5-7  Nonvolatile

lr          Nonvolatile



--- For the scv 0 instruction, differences with the ELF ABI ---

r0          Volatile      (System call number.)

r3          Volatile      (Parameter 1, and return value.)

r4-r8       Volatile      (Parameters 2-6.)

=========== ============= ========================================



All floating point and vector data registers as well as control and status

registers are nonvolatile.



Invocation

----------

The syscall is performed with the sc instruction, and returns with execution

continuing at the instruction following the sc instruction.



Transactional Memory

--------------------

Syscall behavior can change if the processor is in transactional or suspended
@@ -75,6 +92,7 @@ auxiliary vector. 
  returning to the caller. This case is not well defined or supported, so this

  behavior should not be relied upon.



scv 0 syscalls will always behave as PPC_FEATURE2_HTM_NOSC.



vsyscall

========

arch/powerpc/include/asm/asm-prototypes.h

+1 −1
Original line number Diff line number Diff line
@@ -98,7 +98,7 @@ unsigned long __init early_init(unsigned long dt_ptr); 
void __init machine_init(u64 dt_ptr);

#endif

long system_call_exception(long r3, long r4, long r5, long r6, long r7, long r8, unsigned long r0, struct pt_regs *regs);

notrace unsigned long syscall_exit_prepare(unsigned long r3, struct pt_regs *regs);

notrace unsigned long syscall_exit_prepare(unsigned long r3, struct pt_regs *regs, long scv);

notrace unsigned long interrupt_exit_user_prepare(struct pt_regs *regs, unsigned long msr);

notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs, unsigned long msr);

arch/powerpc/include/asm/exception-64s.h

+6 −0
Original line number Diff line number Diff line
@@ -123,6 +123,12 @@ 
	hrfid;								\

	b	hrfi_flush_fallback



#define RFSCV_TO_USER							\

	STF_EXIT_BARRIER_SLOT;						\

	RFI_FLUSH_SLOT;							\

	RFSCV;								\

	b	rfscv_flush_fallback



#endif /* __ASSEMBLY__ */



#endif	/* _ASM_POWERPC_EXCEPTION_H */

arch/powerpc/include/asm/head-64.h

+1 −1
Original line number Diff line number Diff line
@@ -128,7 +128,7 @@ name: 
	.if ((start) % (size) != 0);				\

	.error "Fixed section exception vector misalignment";	\

	.endif;							\

	.if ((size) != 0x20) && ((size) != 0x80) && ((size) != 0x100); \

	.if ((size) != 0x20) && ((size) != 0x80) && ((size) != 0x100) && ((size) != 0x1000); \

	.error "Fixed section exception vector bad size";	\

	.endif;							\

	.if (start) < sname##_start;				\

arch/powerpc/include/asm/ppc_asm.h

+2 −0
Original line number Diff line number Diff line
@@ -755,6 +755,8 @@ END_FTR_SECTION_NESTED(CPU_FTR_CELL_TB_BUG, CPU_FTR_CELL_TB_BUG, 96) 
#define N_SLINE	68

#define N_SO	100



#define RFSCV	.long 0x4c0000a4



/*

 * Create an endian fixup trampoline

 *

CRA Git | Maintained and supported by SUSTech CRA and CCSE