clock: Add a long ticks variable, ticksl

For compatibility with Linux, it's useful to have a tick counter of
width sizeof(long), but our tick counter is an int. Currently the
linuxkpi tries paper o

clock: Add a long ticks variable, ticksl

For compatibility with Linux, it's useful to have a tick counter of
width sizeof(long), but our tick counter is an int. Currently the
linuxkpi tries paper over this difference, but this cannot really be
done reliably, so it's desirable to have a wider tick counter. This
change introduces ticksl, keeping the existing ticks variable.

Follow a suggestion from kib to avoid having to maintain two separate
counters and to avoid converting existing code to use ticksl: change
hardclock() to update ticksl instead of ticks, and then use assembler
directives to make ticks and ticksl overlap such that loading ticks
gives the bottom 32 bits. This makes it possible to use ticksl in the
linuxkpi without having to convert any native code, and without making
hardclock() more complicated or expensive. Then, the linuxkpi can be
modified to use ticksl instead of ticks.

Reviewed by: olce, kib, emaste
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D48383

show more ...

timerfd: Move implementation from linux compat to sys/kern

Move the timerfd impelemntation from linux compat code to sys/kern. Use
it to implement the new system calls for timerfd. Add a hook to ker

timerfd: Move implementation from linux compat to sys/kern

Move the timerfd impelemntation from linux compat code to sys/kern. Use
it to implement the new system calls for timerfd. Add a hook to kern_tc
to allow timerfd to know when the system time has stepped. Add kqueue
support to timerfd. Adjust a few names to be less Linux centric.

RelNotes: YES
Reviewed by: markj (on irc), imp, kib (with reservations), jhb (slack)
Differential Revision: https://reviews.freebsd.org/D38459

show more ...

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

pps: Round to closest integer in pps_event()

The comment above bintime2timespec() says:

When converting between timestamps on parallel timescales of differing
resolutions it is historical and s

pps: Round to closest integer in pps_event()

The comment above bintime2timespec() says:

When converting between timestamps on parallel timescales of differing
resolutions it is historical and scientific practice to round down.

However, the delta_nsec value is a time difference and not a timestamp. Also
the rounding errors accumulate in the frequency accumulator, see hardpps().
So, rounding to the closest integer is probably slightly better.

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/604

show more ...

pps: Simplify the nsec calculation in pps_event()

Let A be the current calculation of the frequency accumulator (pps_fcount)
update in pps_event()

scale = (uint64_t)1 << 63;
scale /= captc->tc_

pps: Simplify the nsec calculation in pps_event()

Let A be the current calculation of the frequency accumulator (pps_fcount)
update in pps_event()

scale = (uint64_t)1 << 63;
scale /= captc->tc_frequency;
scale *= 2;
bt.sec = 0;
bt.frac = 0;
bintime_addx(&bt, scale * tcount);
bintime2timespec(&bt, &ts);
hardpps(tsp, ts.tv_nsec + 1000000000 * ts.tv_sec);

and hardpps(..., delta_nsec):

u_nsec = delta_nsec;
if (u_nsec > (NANOSECOND >> 1))
u_nsec -= NANOSECOND;
else if (u_nsec < -(NANOSECOND >> 1))
u_nsec += NANOSECOND;
pps_fcount += u_nsec;

This change introduces a new calculation which is slightly simpler and more
straight forward. Name it B.

Consider the following sample values with a tcount of 2000000100 and a
tc_frequency of 2000000000 (2GHz).

For A, the scale is 9223372036. Then scale * tcount is 18446744994337203600
which is larger than UINT64_MAX (= 18446744073709551615). The result is
920627651984 == 18446744994337203600 % UINT64_MAX. Since all operands are
unsigned the result is well defined through modulo arithmetic. The result of
bintime2timespec(&bt, &ts) is 49. This is equal to the correct result
1000000049 % NANOSECOND.

In hardpps(), both conditional statements are not executed and pps_fcount is
incremented by 49.

For the new calculation B, we have 1000000000 * tcount is 2000000100000000000
which is less than UINT64_MAX. This yields after the division with tc_frequency
the correct result of 1000000050 for delta_nsec.

In hardpps(), the first conditional statement is executed and pps_fcount is
incremented by 50.

This shows that both methods yield roughly the same results. However, method B
is easier to understand and requires fewer conditional statements.

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/604

show more ...

pps: Directly assign the timestamps in pps_event()

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/604

pps: Move pcount assignment in pps_event()

Move the pseq increment. This makes it possible to reuse registers earlier.

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/604

pps: Simplify capture and event processing

Use local variables for the captured timehand and timecounter in pps_event().
This fixes a potential issue in the nsec preparation for hardpps(). Here the

pps: Simplify capture and event processing

Use local variables for the captured timehand and timecounter in pps_event().
This fixes a potential issue in the nsec preparation for hardpps(). Here the
timecounter was accessed through the captured timehand after the generation was
checked.

Make a snapshot of the relevent timehand values early in pps_event(). Check
the timehand generation only once during the capture and event processing. Use
atomic_thread_fence_acq() similar to the other readers.

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/604

show more ...

pps: Load timecounter once in pps_capture()

This ensures that the timecounter and the tc_get_timecount handler belong
together.

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src

pps: Load timecounter once in pps_capture()

This ensures that the timecounter and the tc_get_timecount handler belong
together.

Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/604

show more ...

set_cputicker: use a bool

The third argument to this function indicates whether the supplied
ticker is fixed or variable, i.e. requiring calibration. Give this
argument a type and name that better c

set_cputicker: use a bool

The third argument to this function indicates whether the supplied
ticker is fixed or variable, i.e. requiring calibration. Give this
argument a type and name that better conveys this purpose.

Reviewed by: kib, markj
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35459

show more ...

kern_tc.c/cputick2usec() (which is used to calculate cputime from
cpu ticks) has some imprecision and, worse, huge timestep (about
20 minutes on 4GHz CPU) near 53.4 days of elapsed time.

kern_time.c

kern_tc.c/cputick2usec() (which is used to calculate cputime from
cpu ticks) has some imprecision and, worse, huge timestep (about
20 minutes on 4GHz CPU) near 53.4 days of elapsed time.

kern_time.c/cputick2timespec() (it is used for clock_gettime() for
querying process or thread consumed cpu time) Uses cputick2usec()
and then needlessly converting usec to nsec, obviously losing
precision even with fixed cputick2usec().

kern_time.c/kern_clock_getres() uses some weird (anyway wrong)
formula for getting cputick resolution.

PR: 262215
Reviewed by: gnn
Differential Revision: https://reviews.freebsd.org/D34558

show more ...

kern_tc: unify timecounter to bintime delta conversion

There are two places where we convert from a timecounter delta to
a bintime delta: tc_windup and bintime_off.
Both functions use the same calcu

kern_tc: unify timecounter to bintime delta conversion

There are two places where we convert from a timecounter delta to
a bintime delta: tc_windup and bintime_off.
Both functions use the same calculations when the timecounter delta is
small. But for a large delta (greater than approximately an equivalent
of 1 second) the calculations were different. Both functions use
approximate calculations based on th_scale that avoid division. Both
produce values slightly greater than a true value, calculated with
division by tc_frequency, would be. tc_windup is slightly more
accurate, so its result is closer to the true value and, thus, smaller
than bintime_off result.

As a consequence there can be a jump back in time when time hands are
switched after a long period of time (a large delta). Just before the
switch the time would be calculated with a large delta from
th_offset_count in bintime_off. tc_windup does the switch using its own
calculations of a new th_offset using the large delta. As explained
earlier, the new th_offset may end up being less than the previously
produced binuptime. So, for a period of time new binuptime values may
be "back in time" comparing to values just before the switch.

Such a jump must never happen. All the code assumes that the uptime is
monotonically nondecreasing and some code works incorrectly when that
assumption is broken. For example, we have observed sleepq_timeout()
ignoring a timeout when the sbinuptime value obtained by the callout
code was greater than the expiration value, but the sbinuptime obtained
in sleepq_timeout() was less than it. In that case the target thread
would never get woken up.

The unified calculations should ensure the monotonic property of the
uptime.

The problem is quite rare as normally tc_windup should be called HZ
times per second (typically 1000 or 100). But it may happen in VMs on
very busy hypervisors where a VM's virtual CPU may not get an execution
time slot for a second or more.

Reviewed by: kib
MFC after: 2 weeks
Sponsored by: Panzura LLC

show more ...

timecounter: Initialize tc_lock earlier

Hyper-V wants to register its MSR-based timecounter during
SI_SUB_HYPERVISOR, before SI_SUB_LOCK, since an emulated 8254 may not be
available for DELAY(). So

timecounter: Initialize tc_lock earlier

Hyper-V wants to register its MSR-based timecounter during
SI_SUB_HYPERVISOR, before SI_SUB_LOCK, since an emulated 8254 may not be
available for DELAY(). So we cannot use MTX_SYSINIT to initialize the
timecounter lock.

PR: 259878
Reviewed by: kib
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33014

show more ...

timecounter: Load the currently selected tc once in tc_windup()

Reported by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Reviewed by: kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundat

timecounter: Load the currently selected tc once in tc_windup()

Reported by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Reviewed by: kib
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32729

show more ...

kern_tc.c: Scaling/large delta recalculation

This change is a slight performance optimization for systems with a slow
64-bit division.

The th->th_scale and th->th_large_delta values only depend on

kern_tc.c: Scaling/large delta recalculation

This change is a slight performance optimization for systems with a slow
64-bit division.

The th->th_scale and th->th_large_delta values only depend on the
timecounter frequency and the th->th_adjustment. The timecounter
frequency of a timehand only changes when a new timecounter is activated
for the timehand. The th->th_adjustment is only changed by the NTP
second update. The NTP second update is not done for every call of
tc_windup().

Move the code block to recalculate the scaling factor and
the large delta of a timehand to the new helper function
recalculate_scaling_factor_and_large_delta().

Call recalculate_scaling_factor_and_large_delta() when a new timecounter
is activated and a NTP second update occurred.

MFC after: 1 week

show more ...

timecounter: Lock the timecounter list

Timecounter registration is dynamic, i.e., there is no requirement that
timecounters must be registered during single-threaded boot. Loadable
drivers may in p

timecounter: Lock the timecounter list

Timecounter registration is dynamic, i.e., there is no requirement that
timecounters must be registered during single-threaded boot. Loadable
drivers may in principle register timecounters (which can be switched to
automatically). Timecounters cannot be unregistered, though this could
be implemented.

Registered timecounters belong to a global linked list. Add a mutex to
synchronize insertions and the traversals done by (mpsafe) sysctl
handlers. No functional change intended.

Reviewed by: imp, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32511

show more ...

timecounter: Let kern.timecounter.stepwarnings be set as a tunable

MFC after: 1 week

Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights

These ones were unambiguous cases where the Foundation was the only
listed copyright holder (in the associated license block).

S

Remove "All Rights Reserved" from FreeBSD Foundation sys/ copyrights

These ones were unambiguous cases where the Foundation was the only
listed copyright holder (in the associated license block).

Sponsored by: The FreeBSD Foundation

show more ...

kern: clarify boot time

In FreeBSD, the current time is computed from uptime + boottime. Uptime
is a continuous, smooth function that's monotonically increasing. To
effect changes to the current tim

kern: clarify boot time

In FreeBSD, the current time is computed from uptime + boottime. Uptime
is a continuous, smooth function that's monotonically increasing. To
effect changes to the current time, boottime is adjusted. boottime is
mutable and shouldn't be cached against future need. Document the
current implementation, with the caveat that we may stop stepping
boottime on resume in the future and will step uptime instead (noted in
the commit message, but not in the code).

Sponsored by: Netflix
Reviewed by: phk, rpokala
Differential Revision: https://reviews.freebsd.org/D30116

show more ...

Make kern.timecounter.hardware tunable

Noted and reviewed by: kevans
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D29122

Add ddb 'show timecounter' command.

MFC after: 1 week
Sponsored by: The FreeBSD Foundation

Changes that improve DTrace FBT reliability on freebsd/arm64:

- Implement a dtrace_getnanouptime(), matching the existing
dtrace_getnanotime(), to avoid DTrace calling out to a potentially
instr

Changes that improve DTrace FBT reliability on freebsd/arm64:

- Implement a dtrace_getnanouptime(), matching the existing
dtrace_getnanotime(), to avoid DTrace calling out to a potentially
instrumentable function.

(These should probably both be under KDTRACE_HOOKS. Also, it's not clear
to me that they are correct implementations for the DTrace thread time
functions they are used in .. fixes for another commit.)

- Don't allow FBT to instrument functions involved in EL1 exception handling
that are involved in FBT trap processing: handle_el1h_sync() and
do_el1h_sync().

- Don't allow FBT to instrument DDB and KDB functions, as that makes it
rather harder to debug FBT problems.

Prior to these changes, use of FBT on FreeBSD/arm64 rapidly led to kernel
panics due to recursion in DTrace.

Reliable FBT on FreeBSD/arm64 is reliant on another change from @andrew to
have the aarch64 instrumentor more carefully check that instructions it
replaces are against the stack pointer, which can otherwise lead to memory
corruption. That change remains under review.

MFC after: 2 weeks
Reviewed by: andrew, kp, markj (earlier version), jrtc27 (earlier version)
Differential revision: https://reviews.freebsd.org/D27766

show more ...

Remove double-calls to tc_get_timecount() to warm timecounters.

It seems that second call does not add any useful state change for all
implemented timecounters.

Discussed with: bde
Sponsored by: Th

Remove double-calls to tc_get_timecount() to warm timecounters.

It seems that second call does not add any useful state change for all
implemented timecounters.

Discussed with: bde
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks

show more ...

Merge ^/head r358269 through r358399.

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly mark

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

show more ...