Add extra constraints to tell the compiler that the memory be modified
in the arm __swp() and sparc64 casa() and casax() functions is actually
being used as an input and output and not just the value

Add extra constraints to tell the compiler that the memory be modified
in the arm __swp() and sparc64 casa() and casax() functions is actually
being used as an input and output and not just the value of the register
that points to the memory location. This was the underlying source of
the mbuf refcount problems on sparc64 a while back. For arm this should be
a nop because __swp() has a constraint to clobber all memory which can
probably be removed now.

Reviewed by: alc, cognet
MFC after: 1 week

show more ...

Convert the atomic_ptr() operations over to operating on uintptr_t
variables rather than void * variables. This makes it easier and simpler
to get asm constraints and volatile keywords correct.

MFC

Convert the atomic_ptr() operations over to operating on uintptr_t
variables rather than void * variables. This makes it easier and simpler
to get asm constraints and volatile keywords correct.

MFC after: 3 days
Tested on: i386, alpha, sparc64
Compiled on: ia64, powerpc, amd64
Kernel toolchain busted on: arm

show more ...

Fix a typo.

Approved by: re (scottl)

Make sure we clean the RAS start address once we're done.
This fixes the random segfaults which occurs at high interrupts rate.

Import a basic implementation of the restartable atomic sequences to provide
atomic operations to userland (this is OK for UP only, but SMP is still so
far away).

Start all license statements with /*-

Make sure gcc doesn't generate something such as swp r3, r4, [r3] for __swp,
as it has unpredictable results.

Disable interrupts for atomic_cmpset_32, this one is just not atomic.
Don't export it to userland.

Try to implement atomic operations using swp, instead of disabling interrupts.

Import FreeBSD/arm kernel bits.
It only supports sa1110 (on simics) right now, but xscale support should come
soon.
Some of the initial work has been provided by :
Stephane Potvin <sepotvin at videot

Import FreeBSD/arm kernel bits.
It only supports sa1110 (on simics) right now, but xscale support should come
soon.
Some of the initial work has been provided by :
Stephane Potvin <sepotvin at videotron.ca>
Most of this comes from NetBSD.

show more ...

- Merge from HEAD

Fix the userland, RAS, version of atomic_fetchadd_32 :
return the correct value, and do not store the wrong one in the supplied
pointer.

Submitted by: Mark Tinguely <tinguely casselton net>

Update the projects tree to a newer FreeBSD current.

force atomic_cmpset_ptr types to match atomic_cmpset_32;
this matches what powerpc does

Submitted by: stass
MFC after: 2 weeks

- bump __FreeBSD version to reflect added buf_ring, memory barriers,
and ifnet functions

- add memory barriers to <machine/atomic.h>
- update drivers to only conditionally define their own

- add

- bump __FreeBSD version to reflect added buf_ring, memory barriers,
and ifnet functions

- add memory barriers to <machine/atomic.h>
- update drivers to only conditionally define their own

- add lockless producer / consumer ring buffer
- remove ring buffer implementation from cxgb and update its callers

- add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to
allow drivers to efficiently manage multiple hardware queues
(i.e. not serialize all packets through one ifq)
- expose if_qflush to allow drivers to flush any driver managed queues

This work was supported by Bitgravity Inc. and Chelsio Inc.

show more ...

Improve ARM_TP_ADDRESS and RAS area.

De-hardcode usage of ARM_TP_ADDRESS and RAS local storage, and move this
special purpose page to a more convenient place i.e. after the vectors high
page, more t

Improve ARM_TP_ADDRESS and RAS area.

De-hardcode usage of ARM_TP_ADDRESS and RAS local storage, and move this
special purpose page to a more convenient place i.e. after the vectors high
page, more towards the end of address space. Previous location (0xe000_0000)
caused grief if KVA was to go beyond the default limit.

Note that ARM world rebuilding is required after this change since the
location of ARM_TP_ADDRESS is shared between kernel and userland.

Submitted by: Grzegorz Bernacki (gjb AT semihalf dot com)
Reviewed by: imp
Approved by: cognet (mentor)

show more ...

Close a race.

The RAS implementation would set the end address, then the start
address. These were used by the kernel to restart a RAS sequence if
it was interrupted. When the thread switching cod

Close a race.

The RAS implementation would set the end address, then the start
address. These were used by the kernel to restart a RAS sequence if
it was interrupted. When the thread switching code ran, it would
check these values and adjust the PC and clear them if it did.

However, there's a small flaw in this scheme. Thread T1, sets the end
address and gets preempted. Thread T2 runs and also does a RAS
operation. This resets end to zero. Thread T1 now runs again and
sets start and then begins the RAS sequence, but is preempted before
the RAS sequence executes its last instruction. The kernel code that
would ordinarily restart the RAS sequence doesn't because the PC isn't
between start and 0, so the PC isn't set to the start of the sequence.
So when T1 is resumed again, it is at the wrong location for RAS to
produce the correct results. This causes the wrong results for the
atomic sequence.

The window for the first race is 3 instructions. The window for the
second race is 5-10 instructions depending on the atomic operation.
This makes this failure fairly rare and hard to reproduce.

Mutexs are implemented in libthr using atomic operations. When the
above race would occur, a lock could get stuck locked, causing many
downstream problems, as you might expect.

Also, make sure to reset the start and end address when doing a syscall, or
a malicious process could set them before doing a syscall.

Reviewed by: imp, ups (thanks guys)
Pointy hat to: cognet
MFC After: 3 days

show more ...

In atomic_fetchadd_32(), do not blindly increase the value of %3.
It should just contain the value we want to add, as if we're interrupted
between the add and the str, we will restart from the beginn

In atomic_fetchadd_32(), do not blindly increase the value of %3.
It should just contain the value we want to add, as if we're interrupted
between the add and the str, we will restart from the beginning. Just use
a register we can scratch instead.

MFC After: 1 week

show more ...

MFp4: Add missing atomic functions
Based on a patch by: des

Add atomic_cmpset_acq_32.

Add definitions for atomic_subtract_rel_32, atomic_add_rel_32 and
atomic_load_acq_32, needed for hwpmc.

Disable/enable fiqs as well as irqs.

Use memory clobbers, to be on the safe side.
Suggested by: jhb

Backout rev 1.12. It would have been a good thing, if gcc was smart enough
not to generate bad code.

A #define is not enough, we need to cast from u_long * to uint32_t *.