Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull scheduler updates from Ingo Molnar:
"The biggest change in this cycle is the rewrite of the main SM

Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull scheduler updates from Ingo Molnar:
"The biggest change in this cycle is the rewrite of the main SMP load
balancing metric: the CPU load/utilization. The main goal was to make
the metric more precise and more representative - see the changelog of
this commit for the gory details:

9d89c257dfb9 ("sched/fair: Rewrite runnable load and utilization average tracking")

It is done in a way that significantly reduces complexity of the code:

5 files changed, 249 insertions(+), 494 deletions(-)

and the performance testing results are encouraging. Nevertheless we
need to keep an eye on potential regressions, since this potentially
affects every SMP workload in existence.

This work comes from Yuyang Du.

Other changes:

- SCHED_DL updates. (Andrea Parri)

- Simplify architecture callbacks by removing finish_arch_switch().
(Peter Zijlstra et al)

- cputime accounting: guarantee stime + utime == rtime. (Peter
Zijlstra)

- optimize idle CPU wakeups some more - inspired by Facebook server
loads. (Mike Galbraith)

- stop_machine fixes and updates. (Oleg Nesterov)

- Introduce the 'trace_sched_waking' tracepoint. (Peter Zijlstra)

- sched/numa tweaks. (Srikar Dronamraju)

- misc fixes and small cleanups"

* 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (44 commits)
sched/deadline: Fix comment in enqueue_task_dl()
sched/deadline: Fix comment in push_dl_tasks()
sched: Change the sched_class::set_cpus_allowed() calling context
sched: Make sched_class::set_cpus_allowed() unconditional
sched: Fix a race between __kthread_bind() and sched_setaffinity()
sched: Ensure a task has a non-normalized vruntime when returning back to CFS
sched/numa: Fix NUMA_DIRECT topology identification
tile: Reorganize _switch_to()
sched, sparc32: Update scheduler comments in copy_thread()
sched: Remove finish_arch_switch()
sched, tile: Remove finish_arch_switch
sched, sh: Fold finish_arch_switch() into switch_to()
sched, score: Remove finish_arch_switch()
sched, avr32: Remove finish_arch_switch()
sched, MIPS: Get rid of finish_arch_switch()
sched, arm: Remove finish_arch_switch()
sched/fair: Clean up load average references
sched/fair: Provide runnable_load_avg back to cfs_rq
sched/fair: Remove task and group entity load when they are dead
sched/fair: Init cfs_rq's sched_entity load average
...

show more ...

Merge tag 'v4.2-rc3' into next

Sync up with Linux 4.2-rc3 to bring in infrastructure (OF) pieces.

sched/numa: Prefer NUMA hotness over cache hotness

The current load balancer may not try to prevent a task from moving
out of a preferred node to a less preferred node. The reason for this
being:

sched/numa: Prefer NUMA hotness over cache hotness

The current load balancer may not try to prevent a task from moving
out of a preferred node to a less preferred node. The reason for this
being:

- Since sched features NUMA and NUMA_RESIST_LOWER are disabled by
default, migrate_degrades_locality() always returns false.

- Even if NUMA_RESIST_LOWER were to be enabled, if its cache hot,
migrate_degrades_locality() never gets called.

The above behaviour can mean that tasks can move out of their
preferred node but they may be eventually be brought back to their
preferred node by numa balancer (due to higher numa faults).

To avoid the above, this commit merges migrate_degrades_locality() and
migrate_improves_locality(). It also replaces 3 sched features NUMA,
NUMA_FAVOUR_HIGHER and NUMA_RESIST_LOWER by a single sched feature
NUMA.

Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Rik van Riel <riel@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mike Galbraith <efault@gmx.de>
Link: http://lkml.kernel.org/r/1434455762-30857-2-git-send-email-srikar@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>

show more ...

Merge branch 'fix/fsl-dspi' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/spi into spi-fsl-dspi

Merge tag 'v4.1-rc1' into patchwork

Linux 4.1-rc1

* tag 'v4.1-rc1': (11651 commits)
Linux 4.1-rc1
x86_64, asm: Work around AMD SYSRET SS descriptor attribute issue
v4l: xilinx: fix for includ

Merge tag 'v4.1-rc1' into patchwork

Linux 4.1-rc1

* tag 'v4.1-rc1': (11651 commits)
Linux 4.1-rc1
x86_64, asm: Work around AMD SYSRET SS descriptor attribute issue
v4l: xilinx: fix for include file movement
platform/chrome: chromeos_laptop - instantiate Atmel at primary address
RCU pathwalk breakage when running into a symlink overmounting something
fix I_DIO_WAKEUP definition
direct-io: only inc/dec inode->i_dio_count for file systems
fs/9p: fix readdir()
Btrfs: prevent list corruption during free space cache processing
toshiba_acpi: Do not register vendor backlight when acpi_video bl is available
x86: fix special __probe_kernel_write() tail zeroing case
crypto: img-hash - CRYPTO_DEV_IMGTEC_HASH should depend on HAS_DMA
crypto: x86/sha512_ssse3 - fixup for asm function prototype change
nios2: rework cache
nios2: Add types.h header required for __u32 type
ALSA: hda - fix headset mic detection problem for one more machine
eth: bf609 eth clock: add pclk clock for stmmac driver probe
blackfin: Wire up missing syscalls
Btrfs: fix inode cache writeout
ACPI / scan: Add a scan handler for PRP0001
...

show more ...

Merge branch 'drm-next-merged' of git://people.freedesktop.org/~airlied/linux into v4l_for_linus

* 'drm-next-merged' of git://people.freedesktop.org/~airlied/linux: (9717 commits)
media-bus: Fixup

Merge branch 'drm-next-merged' of git://people.freedesktop.org/~airlied/linux into v4l_for_linus

* 'drm-next-merged' of git://people.freedesktop.org/~airlied/linux: (9717 commits)
media-bus: Fixup RGB444_1X12, RGB565_1X16, and YUV8_1X24 media bus format
hexdump: avoid warning in test function
fs: take i_mutex during prepare_binprm for set[ug]id executables
smp: Fix error case handling in smp_call_function_*()
iommu-common: Fix PARISC compile-time warnings
sparc: Make LDC use common iommu poll management functions
sparc: Make sparc64 use scalable lib/iommu-common.c functions
Break up monolithic iommu table/lock into finer graularity pools and lock
sparc: Revert generic IOMMU allocator.
tools/power turbostat: correct dumped pkg-cstate-limit value
tools/power turbostat: calculate TSC frequency from CPUID(0x15) on SKL
tools/power turbostat: correct DRAM RAPL units on recent Xeon processors
tools/power turbostat: Initial Skylake support
tools/power turbostat: Use $(CURDIR) instead of $(PWD) and add support for O= option in Makefile
tools/power turbostat: modprobe msr, if needed
tools/power turbostat: dump MSR_TURBO_RATIO_LIMIT2
tools/power turbostat: use new MSR_TURBO_RATIO_LIMIT names
Bluetooth: hidp: Fix regression with older userspace and flags validation
config: Enable NEED_DMA_MAP_STATE by default when SWIOTLB is selected
perf/x86/intel/pt: Fix and clean up error handling in pt_event_add()
...

That solves several merge conflicts:
Documentation/DocBook/media/v4l/subdev-formats.xml
Documentation/devicetree/bindings/vendor-prefixes.txt
drivers/staging/media/mn88473/mn88473.c
include/linux/kconfig.h
include/uapi/linux/media-bus-format.h

The ones at subdev-formats.xml and media-bus-format.h are not trivial.
That's why we opted to merge from DRM.

show more ...

Merge Linus master into drm-next

The merge is clean, but the arm build fails afterwards,
due to API changes in the regulator tree.

I've included the patch into the merge to fix the build.

Signed-o

Merge Linus master into drm-next

The merge is clean, but the arm build fails afterwards,
due to API changes in the regulator tree.

I've included the patch into the merge to fix the build.

Signed-off-by: Dave Airlie <airlied@redhat.com>

show more ...

Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull scheduler changes from Ingo Molnar:
"Major changes:

- Reworked CPU capacity code, for better SM

Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull scheduler changes from Ingo Molnar:
"Major changes:

- Reworked CPU capacity code, for better SMP load balancing on
systems with assymetric CPUs. (Vincent Guittot, Morten Rasmussen)

- Reworked RT task SMP balancing to be push based instead of pull
based, to reduce latencies on large CPU count systems. (Steven
Rostedt)

- SCHED_DEADLINE support updates and fixes. (Juri Lelli)

- SCHED_DEADLINE task migration support during CPU hotplug. (Wanpeng Li)

- x86 mwait-idle optimizations and fixes. (Mike Galbraith, Len Brown)

- sched/numa improvements. (Rik van Riel)

- various cleanups"

* 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (28 commits)
sched/core: Drop debugging leftover trace_printk call
sched/deadline: Support DL task migration during CPU hotplug
sched/core: Check for available DL bandwidth in cpuset_cpu_inactive()
sched/deadline: Always enqueue on previous rq when dl_task_timer() fires
sched/core: Remove unused argument from init_[rt|dl]_rq()
sched/deadline: Fix rt runtime corruption when dl fails its global constraints
sched/deadline: Avoid a superfluous check
sched: Improve load balancing in the presence of idle CPUs
sched: Optimize freq invariant accounting
sched: Move CFS tasks to CPUs with higher capacity
sched: Add SD_PREFER_SIBLING for SMT level
sched: Remove unused struct sched_group_capacity::capacity_orig
sched: Replace capacity_factor by usage
sched: Calculate CPU's usage statistic and put it into struct sg_lb_stats::group_usage
sched: Add struct rq::cpu_capacity_orig
sched: Make scale_rt invariant with frequency
sched: Make sched entity usage tracking scale-invariant
sched: Remove frequency scaling from cpu_capacity
sched: Track group sched_entity usage contributions
sched: Add sched_avg::utilization_avg_contrib
...

show more ...

sched/rt: Use IPI to trigger RT task push migration instead of pulling

When debugging the latencies on a 40 core box, where we hit 300 to
500 microsecond latencies, I found there was a huge contenti

sched/rt: Use IPI to trigger RT task push migration instead of pulling

When debugging the latencies on a 40 core box, where we hit 300 to
500 microsecond latencies, I found there was a huge contention on the
runqueue locks.

Investigating it further, running ftrace, I found that it was due to
the pulling of RT tasks.

The test that was run was the following:

cyclictest --numa -p95 -m -d0 -i100

This created a thread on each CPU, that would set its wakeup in iterations
of 100 microseconds. The -d0 means that all the threads had the same
interval (100us). Each thread sleeps for 100us and wakes up and measures
its latencies.

cyclictest is maintained at:
git://git.kernel.org/pub/scm/linux/kernel/git/clrkwllms/rt-tests.git

What happened was another RT task would be scheduled on one of the CPUs
that was running our test, when the other CPU tests went to sleep and
scheduled idle. This caused the "pull" operation to execute on all
these CPUs. Each one of these saw the RT task that was overloaded on
the CPU of the test that was still running, and each one tried
to grab that task in a thundering herd way.

To grab the task, each thread would do a double rq lock grab, grabbing
its own lock as well as the rq of the overloaded CPU. As the sched
domains on this box was rather flat for its size, I saw up to 12 CPUs
block on this lock at once. This caused a ripple affect with the
rq locks especially since the taking was done via a double rq lock, which
means that several of the CPUs had their own rq locks held while trying
to take this rq lock. As these locks were blocked, any wakeups or load
balanceing on these CPUs would also block on these locks, and the wait
time escalated.

I've tried various methods to lessen the load, but things like an
atomic counter to only let one CPU grab the task wont work, because
the task may have a limited affinity, and we may pick the wrong
CPU to take that lock and do the pull, to only find out that the
CPU we picked isn't in the task's affinity.

Instead of doing the PULL, I now have the CPUs that want the pull to
send over an IPI to the overloaded CPU, and let that CPU pick what
CPU to push the task to. No more need to grab the rq lock, and the
push/pull algorithm still works fine.

With this patch, the latency dropped to just 150us over a 20 hour run.
Without the patch, the huge latencies would trigger in seconds.

I've created a new sched feature called RT_PUSH_IPI, which is enabled
by default.

When RT_PUSH_IPI is not enabled, the old method of grabbing the rq locks
and having the pulling CPU do the work is implemented. When RT_PUSH_IPI
is enabled, the IPI is sent to the overloaded CPU to do a push.

To enabled or disable this at run time:

# mount -t debugfs nodev /sys/kernel/debug
# echo RT_PUSH_IPI > /sys/kernel/debug/sched_features
or
# echo NO_RT_PUSH_IPI > /sys/kernel/debug/sched_features

Update: This original patch would send an IPI to all CPUs in the RT overload
list. But that could theoretically cause the reverse issue. That is, there
could be lots of overloaded RT queues and one CPU lowers its priority. It would
then send an IPI to all the overloaded RT queues and they could then all try
to grab the rq lock of the CPU lowering its priority, and then we have the
same problem.

The latest design sends out only one IPI to the first overloaded CPU. It tries to
push any tasks that it can, and then looks for the next overloaded CPU that can
push to the source CPU. The IPIs stop when all overloaded CPUs that have pushable
tasks that have priorities greater than the source CPU are covered. In case the
source CPU lowers its priority again, a flag is set to tell the IPI traversal to
restart with the first RT overloaded CPU after the source CPU.

Parts-suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Joern Engel <joern@purestorage.com>
Cc: Clark Williams <williams@redhat.com>
Cc: Mike Galbraith <umgwanakikbuti@gmail.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20150318144946.2f3cc982@gandalf.local.home
Signed-off-by: Ingo Molnar <mingo@kernel.org>

show more ...

Merge branch 'next' into for-linus

Prepare first round of input updates for 3.17.

Merge branches 'for-3.17/upstream', 'for-3.17/cp2112', 'for-3.17/huion', 'for-3.17/hyperv', 'for-3.17/i2c', 'for-3.17/lenovo', 'for-3.17/rmi' and 'for-3.17/sony' into for-linus

Merge tag 'v3.16' into next

Linux 3.16

Merge tag 'v3.16-rc6' into next

Merge with mainline to bring in changes to MFD to allow merging
ipaq-micro-ts driver.

Merge tag 'v3.16-rc5' into next/fixes-non-critical

Linux 3.16-rc5

Merge tag 'v3.16-rc5' into timers/core

Reason: Bring in upstream modifications, so the pending changes which
depend on them can be queued.

Sync with the changes pushed by Serge in the last merge window.

Merge tag 'v3.16-rc4' into drm-intel-next-queued

Due to Dave's vacation drm-next hasn't opened yet for 3.17 so I
couldn't move my drm-intel-next queue forward yet like I usually do.
Just pull in the

Merge tag 'v3.16-rc4' into drm-intel-next-queued

Due to Dave's vacation drm-next hasn't opened yet for 3.17 so I
couldn't move my drm-intel-next queue forward yet like I usually do.
Just pull in the latest upstream -rc to unblock patch merging - I
don't want to needlessly rebase my current patch pile really and void
all the testing we've done already.

Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

show more ...

Merge tag 'sti-fixes-for-v3.16-rc1' of git://git.stlinux.com/devel/kernel/linux-sti into fixes

Merge "STi: DT fixes for v3.16" from Maxime Coquelin:

Couple of DT fixes for STi platform issues disco

Merge tag 'sti-fixes-for-v3.16-rc1' of git://git.stlinux.com/devel/kernel/linux-sti into fixes

Merge "STi: DT fixes for v3.16" from Maxime Coquelin:

Couple of DT fixes for STi platform issues discovered on V3.16-rc1.

The fixes included are:
- Ethernet clocks were wrongly defined for STiH415/416 platforms
- STiH416 B2020 revision E DTS file name contained uppercase, change to
lowercase.

* tag 'sti-fixes-for-v3.16-rc1' of git://git.stlinux.com/devel/kernel/linux-sti: (2963 commits)
ARM: stih41x: Rename stih416-b2020-revE.dts to stih416-b2020e.dts
ARM: STi: DT: Properly define sti-ethclk & stmmaceth for stih415/6

Signed-off-by: Arnd Bergmann <arnd@arndb.de>

show more ...

Merge tag 'v3.16-rc1' into x86/cpufeature

Linux 3.16-rc1

Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>

Merge tag 'v3.16-rc1' into patchwork

Linux 3.16-rc1

* tag 'v3.16-rc1': (11443 commits)
Linux 3.16-rc1
net: sctp: fix permissions for rto_alpha and rto_beta knobs
vxlan: Checksum fixes
net:

Merge tag 'v3.16-rc1' into patchwork

Linux 3.16-rc1

* tag 'v3.16-rc1': (11443 commits)
Linux 3.16-rc1
net: sctp: fix permissions for rto_alpha and rto_beta knobs
vxlan: Checksum fixes
net: add skb_pop_rcv_encapsulation
udp: call __skb_checksum_complete when doing full checksum
net: Fix save software checksum complete
net: Fix GSO constants to match NETIF flags
fix __swap_writepage() compile failure on old gcc versions
udp: ipv4: do not waste time in __udp4_lib_mcast_demux_lookup
vxlan: use dev->needed_headroom instead of dev->hard_header_len
MAINTAINERS: update cxgb4 maintainer
x86/vdso: Fix vdso_install
NVMe: Fix START_STOP_UNIT Scsi->NVMe translation.
btrfs: fix error handling in create_pending_snapshot
btrfs: fix use of uninit "ret" in end_extent_writepage()
btrfs: free ulist in qgroup_shared_accounting() error path
Btrfs: fix qgroups sanity test crash or hang
btrfs: prevent RCU warning when dereferencing radix tree slot
Btrfs: fix unfinished readahead thread for raid5/6 degraded mounting
btrfs: new ioctl TREE_SEARCH_V2
...

show more ...

Merge tag 'v3.16-rc1' into i2c/for-next

Merge a stable base (Linux 3.16-rc1)

Signed-off-by: Wolfram Sang <wsa@the-dreams.de>

Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull more scheduler updates from Ingo Molnar:
"Second round of scheduler changes:
- try-to-wakeup and

Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull more scheduler updates from Ingo Molnar:
"Second round of scheduler changes:
- try-to-wakeup and IPI reduction speedups, from Andy Lutomirski
- continued power scheduling cleanups and refactorings, from Nicolas
Pitre
- misc fixes and enhancements"

* 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/deadline: Delete extraneous extern for to_ratio()
sched/idle: Optimize try-to-wake-up IPI
sched/idle: Simplify wake_up_idle_cpu()
sched/idle: Clear polling before descheduling the idle thread
sched, trace: Add a tracepoint for IPI-less remote wakeups
cpuidle: Set polling in poll_idle
sched: Remove redundant assignment to "rt_rq" in update_curr_rt(...)
sched: Rename capacity related flags
sched: Final power vs. capacity cleanups
sched: Remove remaining dubious usage of "power"
sched: Let 'struct sched_group_power' care about CPU capacity
sched/fair: Disambiguate existing/remaining "capacity" usage
sched/fair: Change "has_capacity" to "has_free_capacity"
sched/fair: Remove "power" from 'struct numa_stats'
sched: Fix signedness bug in yield_to()
sched/fair: Use time_after() in record_wakee()
sched/balancing: Reduce the rate of needless idle load balancing
sched/fair: Fix unlocked reads of some cfs_b->quota/period

show more ...

sched: Rename capacity related flags

It is better not to think about compute capacity as being equivalent
to "CPU power". The upcoming "power aware" scheduler work may create
confusion with the not

sched: Rename capacity related flags

It is better not to think about compute capacity as being equivalent
to "CPU power". The upcoming "power aware" scheduler work may create
confusion with the notion of energy consumption if "power" is used too
liberally.

Let's rename the following feature flags since they do relate to capacity:

SD_SHARE_CPUPOWER -> SD_SHARE_CPUCAPACITY
ARCH_POWER -> ARCH_CAPACITY
NONTASK_POWER -> NONTASK_CAPACITY

Signed-off-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Cc: linaro-kernel@lists.linaro.org
Cc: Andy Fleming <afleming@freescale.com>
Cc: Anton Blanchard <anton@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Preeti U Murthy <preeti@linux.vnet.ibm.com>
Cc: Rob Herring <robh+dt@kernel.org>
Cc: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Cc: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: devicetree@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lkml.kernel.org/n/tip-e93lpnxb87owfievqatey6b5@git.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

show more ...

Merge tag 'v3.13' into for-3.15

Linux 3.13

Conflicts:
include/net/xfrm.h

Simple merge where v3.13 removed 'extern' from definitions and the audit
tree did s/u32/unsigned int/ to the same definiti

Merge tag 'v3.13' into for-3.15

Linux 3.13

Conflicts:
include/net/xfrm.h

Simple merge where v3.13 removed 'extern' from definitions and the audit
tree did s/u32/unsigned int/ to the same definitions.

show more ...

Merge tag 'v3.13' into next

Linux 3.13

Minor fixup needed in selinux_inet_conn_request()

Conflicts:
security/selinux/hooks.c