Commit 5ffdfaed authored by Vladimir Murzin's avatar Vladimir Murzin Committed by Catalin Marinas
Browse files

arm64: mm: Support Common Not Private translations 



Common Not Private (CNP) is a feature of ARMv8.2 extension which
allows translation table entries to be shared between different PEs in
the same inner shareable domain, so the hardware can use this fact to
optimise the caching of such entries in the TLB.

CNP occupies one bit in TTBRx_ELy and VTTBR_EL2, which advertises to
the hardware that the translation table entries pointed to by this
TTBR are the same as every PE in the same inner shareable domain for
which the equivalent TTBR also has CNP bit set. In case CNP bit is set
but TTBR does not point at the same translation table entries for a
given ASID and VMID, then the system is mis-configured, so the results
of translations are UNPREDICTABLE.

For kernel we postpone setting CNP till all cpus are up and rely on
cpufeature framework to 1) patch the code which is sensitive to CNP
and 2) update TTBR1_EL1 with CNP bit set. TTBR1_EL1 can be
reprogrammed as result of hibernation or cpuidle (via __enable_mmu).
For these two cases we restore CnP bit via __cpu_suspend_exit().

There are a few cases we need to care of changes in TTBR0_EL1:
  - a switch to idmap
  - software emulated PAN

we rule out latter via Kconfig options and for the former we make
sure that CNP is set for non-zero ASIDs only.

Reviewed-by: default avatarJames Morse <james.morse@arm.com>
Reviewed-by: default avatarSuzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: default avatarVladimir Murzin <vladimir.murzin@arm.com>
[catalin.marinas@arm.com: default y for CONFIG_ARM64_CNP]
Signed-off-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
parent 74e24828

arch/arm64/Kconfig

+14 −0
Original line number Diff line number Diff line
@@ -1134,6 +1134,20 @@ config ARM64_RAS_EXTN 
	  and access the new registers if the system supports the extension.

	  Platform RAS features may additionally depend on firmware support.



config ARM64_CNP

	bool "Enable support for Common Not Private (CNP) translations"

	default y

	depends on ARM64_PAN || !ARM64_SW_TTBR0_PAN

	help

	  Common Not Private (CNP) allows translation table entries to

	  be shared between different PEs in the same inner shareable

	  domain, so the hardware can use this fact to optimise the

	  caching of such entries in the TLB.



	  Selecting this option allows the CNP feature to be detected

	  at runtime, and does not affect PEs that do not implement

	  this feature.



endmenu



config ARM64_SVE

arch/arm64/include/asm/cpucaps.h

+2 −1
Original line number Diff line number Diff line
@@ -53,7 +53,8 @@ 
#define ARM64_HAS_STAGE2_FWB			32

#define ARM64_HAS_CRC32				33

#define ARM64_SSBS				34

#define ARM64_HAS_CNP				35



#define ARM64_NCAPS				35

#define ARM64_NCAPS				36



#endif /* __ASM_CPUCAPS_H */

arch/arm64/include/asm/cpufeature.h

+6 −0
Original line number Diff line number Diff line
@@ -508,6 +508,12 @@ static inline bool system_supports_sve(void) 
		cpus_have_const_cap(ARM64_SVE);

}



static inline bool system_supports_cnp(void)

{

	return IS_ENABLED(CONFIG_ARM64_CNP) &&

		cpus_have_const_cap(ARM64_HAS_CNP);

}



#define ARM64_SSBD_UNKNOWN		-1

#define ARM64_SSBD_FORCE_DISABLE	0

#define ARM64_SSBD_KERNEL		1

arch/arm64/include/asm/mmu_context.h

+15 −2
Original line number Diff line number Diff line
@@ -147,12 +147,25 @@ static inline void cpu_replace_ttbr1(pgd_t *pgdp) 
	extern ttbr_replace_func idmap_cpu_replace_ttbr1;

	ttbr_replace_func *replace_phys;



	phys_addr_t pgd_phys = virt_to_phys(pgdp);

	/* phys_to_ttbr() zeros lower 2 bits of ttbr with 52-bit PA */

	phys_addr_t ttbr1 = phys_to_ttbr(virt_to_phys(pgdp));



	if (system_supports_cnp() && !WARN_ON(pgdp != lm_alias(swapper_pg_dir))) {

		/*

		 * cpu_replace_ttbr1() is used when there's a boot CPU

		 * up (i.e. cpufeature framework is not up yet) and

		 * latter only when we enable CNP via cpufeature's

		 * enable() callback.

		 * Also we rely on the cpu_hwcap bit being set before

		 * calling the enable() function.

		 */

		ttbr1 |= TTBR_CNP_BIT;

	}



	replace_phys = (void *)__pa_symbol(idmap_cpu_replace_ttbr1);



	cpu_install_idmap();

	replace_phys(pgd_phys);

	replace_phys(ttbr1);

	cpu_uninstall_idmap();

}

arch/arm64/include/asm/pgtable-hwdef.h

+2 −0
Original line number Diff line number Diff line
@@ -211,6 +211,8 @@ 
#define PHYS_MASK_SHIFT		(CONFIG_ARM64_PA_BITS)

#define PHYS_MASK		((UL(1) << PHYS_MASK_SHIFT) - 1)



#define TTBR_CNP_BIT		(UL(1) << 0)



/*

 * TCR flags.

 */

CRA Git | Maintained and supported by SUSTech CRA and CCSE