tcp: Fix wrap around comparison bug

The variables p_curtick and p_lasttick are not in usecs.

Reviewed by: tuexen
MFC after: 1 week
Sponsored by: Netflix, Inc.

tcp hpts: cleanup header file

Cleanup tcp_hpts.h by
* move definition used only in tcp_hpts.c to that file
* fix a typo
* remove duplicate declarion of tcp_min_hptsi_time
* rearange declarations for

tcp hpts: cleanup header file

Cleanup tcp_hpts.h by
* move definition used only in tcp_hpts.c to that file
* fix a typo
* remove duplicate declarion of tcp_min_hptsi_time
* rearange declarations for simpler reading

Approved by: tuexen
MFC after: 1 week
Sponsored by: Netflix, Inc.

show more ...

tcp: improve variable and constant names

Don't use ticks in variable names or constant when they don't have
a relation to ticks. Use slots or usecs.
No functional change intended.

Reviewed by: tuex

tcp: improve variable and constant names

Don't use ticks in variable names or constant when they don't have
a relation to ticks. Use slots or usecs.
No functional change intended.

Reviewed by: tuexen
MFC after: 1 week
Sponsored by: Netflix, Inc.

show more ...

tcp: improve function names

tcp_tv_to_usectick(), tcp_tv_to_mssectick(), and tcp_tv_to_lusectick()
are not related to ticks. Therefore remove the trailing 'tick'.
No functional change intended.

Rev

tcp: improve function names

tcp_tv_to_usectick(), tcp_tv_to_mssectick(), and tcp_tv_to_lusectick()
are not related to ticks. Therefore remove the trailing 'tick'.
No functional change intended.

Reviewed by: tuexen
MFC after: 1 week
Sponsored by: Netflix, Inc.

show more ...

tcp hpts: remove unused line argument from tcp_set_hpts

Reviewed by: tuexen
MFC after: 1 week
Sponsored by: Netflix, Inc.

tcp: remove duplicate tcp_bblogging_on checks

Reviewed by: tuexen
MFC after: 1 week
Sponsored by: Netflix, Inc.

tcp: clear blackbox logging union

Clear the black box logging containing union rather than the u_bbr
structure for clarity and consistency. Currently u_bbr, u_raw, and
u64_raw are the same size.
No

tcp: clear blackbox logging union

Clear the black box logging containing union rather than the u_bbr
structure for clarity and consistency. Currently u_bbr, u_raw, and
u64_raw are the same size.
No functional change intended.

Reviewed by: tuexen
MFC after: 3 days
Sponsored by: Netflix, Inc.

show more ...

tcp_hpts: refactor the per tcpcb call to either input/output method

Either input or output return unlocked on failure. Should be no
functional change.

Reviewed by: rrs
Differential Revision: http

tcp_hpts: refactor the per tcpcb call to either input/output method

Either input or output return unlocked on failure. Should be no
functional change.

Reviewed by: rrs
Differential Revision: https://reviews.freebsd.org/D47925

show more ...

tcp_hpts: consistenly use macros to lock & unlock

The macros version of lock/unlock has already been used 23 times in this
file and the bare version was used 6 times only, so prefer the former.
No f

tcp_hpts: consistenly use macros to lock & unlock

The macros version of lock/unlock has already been used 23 times in this
file and the bare version was used 6 times only, so prefer the former.
No functional change.

show more ...

tcp_hpts: use booleans for tcp_hptsi() local variables

No functional change.

tcp_hpts: use boolean to tell is it callout or userret context

No functional change.

tcp hpts: initialize variable

Ensure that tv.tv_sec is zero in all code paths.

Reported by: Coverity Scan
CID: 1527724
Reviewed by: rscheff
MFC after: 3 days
Sponsored by: Netflix, Inc.
Diff

tcp hpts: initialize variable

Ensure that tv.tv_sec is zero in all code paths.

Reported by: Coverity Scan
CID: 1527724
Reviewed by: rscheff
MFC after: 3 days
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D44584

show more ...

tcp hpts: improve consistency

The target_slot argument of max_slots_available() can be NULL.
Therefore, check for this in all places.
Right now, all callers provide non-NULL pointer.

Reported by:

tcp hpts: improve consistency

The target_slot argument of max_slots_available() can be NULL.
Therefore, check for this in all places.
Right now, all callers provide non-NULL pointer.

Reported by: Coverity Scan
CID: 1527732
Reviewed by: rrs
MFC after: 3 days
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D44527

show more ...

Optimize HPTS so that little work is done until we have a hpts thread that is over the connection threshold

HPTS inserts a softclock for system call return that optimizes performance. However when
n

Optimize HPTS so that little work is done until we have a hpts thread that is over the connection threshold

HPTS inserts a softclock for system call return that optimizes performance. However when
no HPTS threads need the help (i.e. when they have less than 100 or so connections) then
there should be little work done i.e. check the counter and return instead of running through
all the threads getting locks etc.ptimize HPTS so that little work is done until we have a hpts
thread that is over the connection threshold.

Reported by: eduardo
Reviewed by: gallatin, glebius, tuexen
Tested by: gallatin
Differential Revision: https://reviews.freebsd.org/D44420

show more ...

HTPS has actually three states not two so the macro needs to account for that.

Ok lets fix up the tcp_in_hpts() so that it also says yes if you
are in the race state moving and you are scheduled to

HTPS has actually three states not two so the macro needs to account for that.

Ok lets fix up the tcp_in_hpts() so that it also says yes if you
are in the race state moving and you are scheduled to be put in.
This also requires changing the MPASS to be the old version non
inline function of tcp_in_hpts().

This change also adds a new inline macro so that a uint64_t timestamp can be
obtained by a transport (aka Rack will use this).

Reviewed by: glebius, tuexen
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D44157

show more ...

tcp_hpts: Fix a typo of a function name in a comment

- s/tcp_ouput/tcp_output/

MFC after: 3 days

hpts: avoid duplicate call to tcp_output()

Obtained from: rrs

tcp_hpts: make the module unloadable

Although the HPTS subsytem wasn't initially designed as a loadable
module, now it is so. Make it possible to also unload it, but for
safety reasons hide that un

tcp_hpts: make the module unloadable

Although the HPTS subsytem wasn't initially designed as a loadable
module, now it is so. Make it possible to also unload it, but for
safety reasons hide that under 'kldunload -f'.

Reviewed by: tuexen
Differential Revision: https://reviews.freebsd.org/D43092

show more ...

tcp_hpts: use tcp_pace.cts_last_ran for last ran table

Remove the global cts_last_ran and use already existing unused field of
struct tcp_hptsi, which seems originally planned to hold this table. T

tcp_hpts: use tcp_pace.cts_last_ran for last ran table

Remove the global cts_last_ran and use already existing unused field of
struct tcp_hptsi, which seems originally planned to hold this table. This
makes it consistent with other malloc-ed tables, like main array of HPTS
entities and CPU groups.

Reviewed by: tuexen
Differential Revision: https://reviews.freebsd.org/D43091

show more ...

tcp_hpts: let tcp_hpts_init() set a random CPU only once

After d2ef52ef3dee the tcp_hpts_init() function can be called multiple
times on a tcpcb if it is switched there and back between two TCP stac

tcp_hpts: let tcp_hpts_init() set a random CPU only once

After d2ef52ef3dee the tcp_hpts_init() function can be called multiple
times on a tcpcb if it is switched there and back between two TCP stacks.
First, this makes existing assertion in tcp_hpts_init() incorrect. Second,
it creates possibility to change a randomly set t_hpts_cpu to a different
random value, while a tcpcb is already in the HPTS wheel, triggering other
assertions later in tcp_hptsi().

The best approach here would be to work on the stacks to really clear a
tcpcb out of HPTS wheel in tfb_tcp_fb_fini, draining the IHPTS_MOVING
state. But that's pretty intrusive change, so let's just get back to the
old logic (pre d2ef52ef3dee) where t_hpts_cpu was set to a random value
only once in a CPU lifetime and a newly switched stack inherits t_hpts_cpu
from the previous stack.

Reviewed by: rrs, tuexen
Differential Revision: https://reviews.freebsd.org/D42946
Reported-by: syzbot+fab29fe1ab089c52998d@syzkaller.appspotmail.com
Reported-by: syzbot+ca5f2aa0fda15dcfe6d7@syzkaller.appspotmail.com
Fixes: 2b3a77467dd3d74a7170f279fb25f9736b46ef8a

show more ...

hpts/lro: make tcp_lro_flush_tcphpts() and tcp_run_hpts() pointers

Rename tcp_run_hpts() to tcp_hpts_softlock() to better describe its
function. This makes loadable hpts.ko working correctly with L

hpts/lro: make tcp_lro_flush_tcphpts() and tcp_run_hpts() pointers

Rename tcp_run_hpts() to tcp_hpts_softlock() to better describe its
function. This makes loadable hpts.ko working correctly with LRO.

Reviewed by: tuexen, rrs
Differential Revision: https://reviews.freebsd.org/D42858

show more ...

Fix two latent bugs in hpts. One where a static is put on
a local variable, the other an initialization bug where
we should be setting tv.tv_sec to 0.

PR: 275482

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

tcp: move HPTS/LRO flags out of inpcb to tcpcb

These flags are TCP specific. While here, make also several LRO
internal functions to pass tcpcb pointer instead of inpcb one.

Reviewed by: rrs
Diff

tcp: move HPTS/LRO flags out of inpcb to tcpcb

These flags are TCP specific. While here, make also several LRO
internal functions to pass tcpcb pointer instead of inpcb one.

Reviewed by: rrs
Differential Revision: https://reviews.freebsd.org/D39698

show more ...

tcp_hpts: move HPTS related fields from inpcb to tcpcb

This makes inpcb lighter and allows future cache line optimizations
of tcpcb. The reason why HPTS originally used inpcb is the compressed
TIME

tcp_hpts: move HPTS related fields from inpcb to tcpcb

This makes inpcb lighter and allows future cache line optimizations
of tcpcb. The reason why HPTS originally used inpcb is the compressed
TIME-WAIT state (see 0d7445193ab), that used to free a tcpcb, while the
associated connection is still on the HPTS ring.

Reviewed by: rrs
Differential Revision: https://reviews.freebsd.org/D39697

show more ...