Move IOC initialization from cache_init() to a separate function.

This is the preparation for the next patch where we'll switch
to is_isa_arcv2() function usage instead of "CONFIG_ISA_ARCV2"
ifdef.

Also it makes cache_init function a bit cleaner.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

We don't implement separate flush_dcache_all() intentionally as
entire data cache invalidation is dangerous operation even if we flush
data cache right before invalidation.

There is the real example:
We may get stuck in the following code if we store any context (like
BLINK register) on stack in invalidate_dcache_all() function.

BLINK register is the register where return address is automatically saved
when we do function call with instructions like 'bl'.

void flush_dcache_all() {
	__dc_entire_op(OP_FLUSH);
	// Other code //
}

void invalidate_dcache_all() {
	__dc_entire_op(OP_INV);
	// Other code //
}

void foo(void) {
	flush_dcache_all();
	invalidate_dcache_all();
}

Now let's see what really happens during that code execution:

foo()
  |->> call flush_dcache_all
  	[return address is saved to BLINK register]
  	[push BLINK] (save to stack)              ![point 1]
  	|->> call __dc_entire_op(OP_FLUSH)
  		[return address is saved to BLINK register]
  		[flush L1 D$]
  		return [jump to BLINK]
  	<<------
  	[other flush_dcache_all code]
  	[pop BLINK] (get from stack)
  	return [jump to BLINK]
  <<------
  |->> call invalidate_dcache_all
  	[return address is saved to BLINK register]
  	[push BLINK] (save to stack)               ![point 2]
  	|->> call __dc_entire_op(OP_FLUSH)
  		[return address is saved to BLINK register]
  		[invalidate L1 D$]                 ![point 3]
  		// Oops!!!
  		// We lose return address from invalidate_dcache_all function:
  		// we save it to stack and invalidate L1 D$ after that!
  		return [jump to BLINK]
  	<<------
  	[other invalidate_dcache_all code]
  	[pop BLINK] (get from stack)
  	// we don't have this data in L1 dcache as we invalidated it in [point 3]
  	// so we get it from next memory level (for example DDR memory)
  	// but in the memory we have value which we save in [point 1], which
  	// is return address from flush_dcache_all function (instead of
  	// address from current invalidate_dcache_all function which we
  	// saved in [point 2] !)
  	return [jump to BLINK]
  <<------
  // As BLINK points to invalidate_dcache_all, we call it again and
  // loop forever.

Fortunately we may do flush and invalidation of D$ with a single one
instruction which automatically mitigates a situation described above.

And because invalidate_dcache_all() isn't used in common U-Boot code we
implement "flush and invalidate dcache all" instead.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

As of today __dc_line_op() and __dc_entire_op() support
only separate flush (OP_FLUSH) and invalidate (OP_INV) operations.

Add support of combined flush and invalidate (OP_FLUSH_N_INV)
operation which we planing to use in subsequent patches.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

__cache_line_loop() function was copied from Linux kernel
where per-line instruction cache operations are really used.

In U-Boot we use only entire I$ ops, so we can drop support of
per-line I$ ops from __cache_line_loop() because __cache_line_loop()
is never called with OP_INV_IC parameter.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

Move instruction cache entire operation to a separate function
because we are planing to use it in other places like
sync_icache_dcache_all().

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

We'd like to keep IOC HW at the same state as t is right after reset when we
start Linux kernel so there will be no re-configuration of IOC on the go.

The point is U-Boot doesn't benefit a lot from IOC as it doesn't do a
lot of DMA operations especially on multiple cores simultaneously.

At the same time re-configuration of IOC in run-time might become quite
a tricky experience because we need to make sure there're no DMA
trannsactions in flight otherwise unexpected consequencses might affect
us much later and debugging those kinds of issues will be a real
nightmare.

That said let's make our life easier a little bit.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

Previous SLC management implementation is broken. Seems like it was
never sufficiently tested probably because most of the time IOC was used
instead (i.e. no manual cache operations were done).

Now if we disable IOC in U-boot we'll get a lot of errors while using
DMA-enabled peripherals.

This time we fix it by substitution of broken per-line SLC operations
region operations as it is done in the Linux kernel (we took it from
v4.14 which is the latest stable as of today).

Among other things this implementation might be a bit faster because
instead of iteration over each and every cache line we're taking care
about entire region in one go.

Main changes:
 * Replaced __slc_line_op (per line operations) by __slc_rgn_op
   (region operations).

 * Reworked __slc_entire_op to get rid of __after_slc_op and
   __before_slc_op functions.
   Note flush fix (flush only instead of flush-n-inv when OP_FLUSH is
   used, see [1] for more details) is already incorporated here.

 * Added SLC invalidation to invalidate_icache_all().

 * Added (start >= end) check to invalidate_dcache_range() and
   flush_dcache_range() as some buggy drivers pass region start == end.

 * Added read-out of MMU BCR so we may know if PAE40 exists in HW and then
   act on a particular AUX regs accordingly.

[1] http://lists.infradead.org/pipermail/linux-snps-arc/2018-January/003357.html



Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

dcache_exists, icache_exists, slc_exists and ioc_exists global
variables in "arch/arc/lib/cache.c" remain uninitialized if
SoC doesn't have corresponding HW.

This happens because we use the next constructions for their
definition and initialization:
-------------------------->>---------------------
int ioc_exists __section(".data");

if (/* condition */)
		ioc_exists = 1;
-------------------------->>---------------------

That's quite a non-trivial issue as one may think of it.
The point is we intentionally put those variables in ".data" section
so they might survive relocation (remember we initilaize them very early
before relocation and continue to use after reloaction). While being
non-initialized and not explicitly put in .data section they would end-up
in ".bss" section which by definition is filled with zeroes.
But since we place those variables in .data section we need to care
about their proper initialization ourselves.

Also while at it we change their type to "bool" as more appropriate.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

As per ARC HS databook (see chapter 5.3.3.2) it is required to add
3 NOPs after each write to IC_IVIC which we do from now on.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Eugeniy Paltsev <paltsev@synopsys.com>

We used to use the same memory layout and size for a couple of
boards and thus we just hardcoding IOC aperture start and size.

Now when we're getting more boards with more memory on board we
need to have an ability to set IOC so it matches real DDR layout
and size.

Even though it is not really a must but for simplicity we assume
IOC covers all the DDR we have, that gives us a chance to not
bother where DMA buffers are allocated - any part of DDR is OK.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

As reported in STAR 9001165532, an SLC control reg read (for checking
busy state) right after SLC invalidate command may incorrectly return
NOT busy causing software to NOT spin-wait while operation is underway.
(and for some reason this only happens if L1 cache is also disabled - as
required by IOC programming model)

Suggested workaround is to do an additional Control Reg read, which
ensures the 2nd read gets the right status.

Same fix made in Linux kernel:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c70c473396cbdec1168a6eff60e13029c0916854



Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

According to ARC HS databook it is required to flush and disable
caches prior programming IOC registers. Otherwise ongoing coherent
memory operations may not observe the coherency protocols as
expected.

But since in ARC HS v2.1 there's no way to disable SLC (AKA L2 cache)
we're doing our best flushing and invalidating it.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

invalidate_dcache_all() could be used in different use-cases
and what is especially important most of those cases won't be
related to DMAed data to or from peripherals, i.e. we'll be doing
invalidation of data used purely by CPU cores.

Given that IOC engine only snoops data that goes through DMA
we need to care ourselves about data used only by CPU cores
and so remove dependency on IOC from invalidate_dcache_all()
and always do real invalidation.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

flush_dcache_all() is used in the very end of U-Boot self relocation
to write back all copied and then patched code and data to their
new location in the very end of available memory space.

Since that has nothing to do with IO (i.e. no external DMA happens
here) IOC won't help here and we need to write back data cache contents
manually.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

With release of ARC HS38 v2.1 new IO coherency engine could be built-in
ARC core. This hardware module ensures coherency between DMA-ed data
from peripherals and L2 cache.

With L2 and IOC enabled there's no overhead for L2 cache manual
maintenance which results in significantly improved IO bandwidth.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

ARC core could be configured with different L1 and L2 (AKA SLC) cache
line lengths. At least these values are possible and were really used:
32, 64 or 128 bytes.

Current implementation requires cache line to be selected upon U-Boot
configuration and then it will only work on matching hardware. Indeed
this is quite efficient because cache line length gets hardcoded during
code compilation. But OTOH it makes binary less portable.

With this commit we allow U-Boot to determine real L1 cache line length
early in runtime and use this value later on. This extends portability
of U-Boot binary a lot.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

[1] Align cache management functions to those in Linux kernel. I.e.:
    a) Use the same functions for all cache ops (D$ Inv/Flush)
    b) Split cache ops in 3 sub-functions: "before", "lineloop" and
"after". That way we may re-use "before" and "after" functions for
region and full cache ops.

 [2] Implement full-functional L2 (SLC) management. Before SLC was
simply disabled early on boot. It's also possible to enable or disable
L2 cache from config utility.

 [3] Disable/enable corresponding caches early on boot. So if U-Boot is
configured to use caches they will be used at all times (this is useful
in partucular for speed-up of relocation).

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

ARCv2 cores may have built-in SLC (System Level Cache, AKA L2-cache).
This change adds functions required for controlling SLC:
 * slc_enable/disable
 * slc_flush/invalidate

For now we just disable SLC to escape DMA coherency issues until either:
 * SLC flush/invalidate is supported in DMA APIin U-Boot
 * hardware DMA coherency is implemented (that might be board specific
   so probably we'll need to have a separate Kconfig option for
   controlling SLC explicitly)

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

always

Make both invalidate_icache_all() and invalidate_dcache_all() available
even if U-Boot is configured with CONFIG_SYS_DCACHE_OFF and/or
CONFIG_SYS_ICACHE_OFF.

This is useful because configuration of U-Boot may not match actual
hardware features. Real board may have cache(s) but for some reason we
may want to run U-Boot with cache(s) disabled (for example if some
peripherals work improperly with existing drivers if data cache is
enabled). So board may start with cache(s) enabled (that's the case for
ARC cores with built-in caches) but early in U-Boot we disable cache(s)
and make sure all contents of data cache gets flushed in RAM.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

This change allows to keep board description clean and minimalistic.
This is especially helpful if one board may house different CPUs with
different features.

It is applicable to both FPGA-based boards or those that have CPUs
mounted on interchnagable daughter-boards.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

DC_PTAG and IC_PTAG registers only exist in MMUv3.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

"reset.c" and "cpu.c" have no architecture-specific code at all.
Others are applicable to either ARC CPU.

This change is a preparation to submission of ARCv2 architecture port.

Even though ARCv1 and ARCv2 ISAs are not binary compatible most of
built-in modules still have the same programming model - AUX registers
are mapped in the same addresses and hold the same data (new featues
extend existing ones).

So only low-level assembly code (start-up, interrupt handlers) is left
as CPU(actually ISA)-specific. This significantyl simplifies maintenance
of multiple CPUs/ISAs.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Igor Guryanov <guryanov@synopsys.com>

Some cache operations ({i|d}cache_{enable|disable|status} or
flush_dcache_all) are built and used even if CONFIG_SYS_{I|D}CACHE_OFF
is set.

This is required for force disable of caches on early boot.
What if something was executed before U-boot and enabled caches
(low-level bootloaders, previously run kernel etc.)?

But if CPU doesn't really have caches any attempt to access
cache-related AUX registers triggers instruction error exception.

So for convenience we'll try to avoid exceptions by checking if CPU
actually has caches (we check separately data and instruction cache
existence) at all.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Igor Guryanov <guryanov@synopsys.com>

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>

Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Francois Bedard <fbedard@synopsys.com>
Cc: Wolfgang Denk <wd@denx.de>
Cc: Heiko Schocher <hs@denx.de>