1FPGA Region Device Tree Binding 2 3Alan Tull 2016 4 5 CONTENTS 6 - Introduction 7 - Terminology 8 - Sequence 9 - FPGA Region 10 - Supported Use Models 11 - Device Tree Examples 12 - Constraints 13 14 15Introduction 16============ 17 18FPGA Regions represent FPGA's and partial reconfiguration regions of FPGA's in 19the Device Tree. FPGA Regions provide a way to program FPGAs under device tree 20control. 21 22This device tree binding document hits some of the high points of FPGA usage and 23attempts to include terminology used by both major FPGA manufacturers. This 24document isn't a replacement for any manufacturers specifications for FPGA 25usage. 26 27 28Terminology 29=========== 30 31Full Reconfiguration 32 * The entire FPGA is programmed. 33 34Partial Reconfiguration (PR) 35 * A section of an FPGA is reprogrammed while the rest of the FPGA is not 36 affected. 37 * Not all FPGA's support PR. 38 39Partial Reconfiguration Region (PRR) 40 * Also called a "reconfigurable partition" 41 * A PRR is a specific section of a FPGA reserved for reconfiguration. 42 * A base (or static) FPGA image may create a set of PRR's that later may 43 be independently reprogrammed many times. 44 * The size and specific location of each PRR is fixed. 45 * The connections at the edge of each PRR are fixed. The image that is loaded 46 into a PRR must fit and must use a subset of the region's connections. 47 * The busses within the FPGA are split such that each region gets its own 48 branch that may be gated independently. 49 50Persona 51 * Also called a "partial bit stream" 52 * An FPGA image that is designed to be loaded into a PRR. There may be 53 any number of personas designed to fit into a PRR, but only one at at time 54 may be loaded. 55 * A persona may create more regions. 56 57FPGA Bridge 58 * FPGA Bridges gate bus signals between a host and FPGA. 59 * FPGA Bridges should be disabled while the FPGA is being programmed to 60 prevent spurious signals on the cpu bus and to the soft logic. 61 * FPGA bridges may be actual hardware or soft logic on an FPGA. 62 * During Full Reconfiguration, hardware bridges between the host and FPGA 63 will be disabled. 64 * During Partial Reconfiguration of a specific region, that region's bridge 65 will be used to gate the busses. Traffic to other regions is not affected. 66 * In some implementations, the FPGA Manager transparantly handles gating the 67 buses, eliminating the need to show the hardware FPGA bridges in the 68 device tree. 69 * An FPGA image may create a set of reprogrammable regions, each having its 70 own bridge and its own split of the busses in the FPGA. 71 72FPGA Manager 73 * An FPGA Manager is a hardware block that programs an FPGA under the control 74 of a host processor. 75 76Base Image 77 * Also called the "static image" 78 * An FPGA image that is designed to do full reconfiguration of the FPGA. 79 * A base image may set up a set of partial reconfiguration regions that may 80 later be reprogrammed. 81 82 ---------------- ---------------------------------- 83 | Host CPU | | FPGA | 84 | | | | 85 | ----| | ----------- -------- | 86 | | H | | |==>| Bridge0 |<==>| PRR0 | | 87 | | W | | | ----------- -------- | 88 | | | | | | 89 | | B |<=====>|<==| ----------- -------- | 90 | | R | | |==>| Bridge1 |<==>| PRR1 | | 91 | | I | | | ----------- -------- | 92 | | D | | | | 93 | | G | | | ----------- -------- | 94 | | E | | |==>| Bridge2 |<==>| PRR2 | | 95 | ----| | ----------- -------- | 96 | | | | 97 ---------------- ---------------------------------- 98 99Figure 1: An FPGA set up with a base image that created three regions. Each 100region (PRR0-2) gets its own split of the busses that is independently gated by 101a soft logic bridge (Bridge0-2) in the FPGA. The contents of each PRR can be 102reprogrammed independently while the rest of the system continues to function. 103 104 105Sequence 106======== 107 108When a DT overlay that targets a FPGA Region is applied, the FPGA Region will 109do the following: 110 111 1. Disable appropriate FPGA bridges. 112 2. Program the FPGA using the FPGA manager. 113 3. Enable the FPGA bridges. 114 4. The Device Tree overlay is accepted into the live tree. 115 5. Child devices are populated. 116 117When the overlay is removed, the child nodes will be removed and the FPGA Region 118will disable the bridges. 119 120 121FPGA Region 122=========== 123 124FPGA Regions represent FPGA's and FPGA PR regions in the device tree. An FPGA 125Region brings together the elements needed to program on a running system and 126add the child devices: 127 128 * FPGA Manager 129 * FPGA Bridges 130 * image-specific information needed to to the programming. 131 * child nodes 132 133The intended use is that a Device Tree overlay (DTO) can be used to reprogram an 134FPGA while an operating system is running. 135 136An FPGA Region that exists in the live Device Tree reflects the current state. 137If the live tree shows a "firmware-name" property or child nodes under a FPGA 138Region, the FPGA already has been programmed. A DTO that targets a FPGA Region 139and adds the "firmware-name" property is taken as a request to reprogram the 140FPGA. After reprogramming is successful, the overlay is accepted into the live 141tree. 142 143The base FPGA Region in the device tree represents the FPGA and supports full 144reconfiguration. It must include a phandle to an FPGA Manager. The base 145FPGA region will be the child of one of the hardware bridges (the bridge that 146allows register access) between the cpu and the FPGA. If there are more than 147one bridge to control during FPGA programming, the region will also contain a 148list of phandles to the additional hardware FPGA Bridges. 149 150For partial reconfiguration (PR), each PR region will have an FPGA Region. 151These FPGA regions are children of FPGA bridges which are then children of the 152base FPGA region. The "Full Reconfiguration to add PRR's" example below shows 153this. 154 155If an FPGA Region does not specify a FPGA Manager, it will inherit the FPGA 156Manager specified by its ancestor FPGA Region. This supports both the case 157where the same FPGA Manager is used for all of a FPGA as well the case where 158a different FPGA Manager is used for each region. 159 160FPGA Regions do not inherit their ancestor FPGA regions' bridges. This prevents 161shutting down bridges that are upstream from the other active regions while one 162region is getting reconfigured (see Figure 1 above). During PR, the FPGA's 163hardware bridges remain enabled. The PR regions' bridges will be FPGA bridges 164within the static image of the FPGA. 165 166Required properties: 167- compatible : should contain "fpga-region" 168- fpga-mgr : should contain a phandle to an FPGA Manager. Child FPGA Regions 169 inherit this property from their ancestor regions. A fpga-mgr property 170 in a region will override any inherited FPGA manager. 171- #address-cells, #size-cells, ranges : must be present to handle address space 172 mapping for child nodes. 173 174Optional properties: 175- firmware-name : should contain the name of an FPGA image file located on the 176 firmware search path. If this property shows up in a live device tree 177 it indicates that the FPGA has already been programmed with this image. 178 If this property is in an overlay targeting a FPGA region, it is a 179 request to program the FPGA with that image. 180- fpga-bridges : should contain a list of phandles to FPGA Bridges that must be 181 controlled during FPGA programming along with the parent FPGA bridge. 182 This property is optional if the FPGA Manager handles the bridges. 183 If the fpga-region is the child of a fpga-bridge, the list should not 184 contain the parent bridge. 185- partial-fpga-config : boolean, set if partial reconfiguration is to be done, 186 otherwise full reconfiguration is done. 187- external-fpga-config : boolean, set if the FPGA has already been configured 188 prior to OS boot up. 189- encrypted-fpga-config : boolean, set if the bitstream is encrypted 190- region-unfreeze-timeout-us : The maximum time in microseconds to wait for 191 bridges to successfully become enabled after the region has been 192 programmed. 193- region-freeze-timeout-us : The maximum time in microseconds to wait for 194 bridges to successfully become disabled before the region has been 195 programmed. 196- config-complete-timeout-us : The maximum time in microseconds time for the 197 FPGA to go to operating mode after the region has been programmed. 198- child nodes : devices in the FPGA after programming. 199 200In the example below, when an overlay is applied targeting fpga-region0, 201fpga_mgr is used to program the FPGA. Two bridges are controlled during 202programming: the parent fpga_bridge0 and fpga_bridge1. Because the region is 203the child of fpga_bridge0, only fpga_bridge1 needs to be specified in the 204fpga-bridges property. During programming, these bridges are disabled, the 205firmware specified in the overlay is loaded to the FPGA using the FPGA manager 206specified in the region. If FPGA programming succeeds, the bridges are 207reenabled and the overlay makes it into the live device tree. The child devices 208are then populated. If FPGA programming fails, the bridges are left disabled 209and the overlay is rejected. The overlay's ranges property maps the lwhps 210bridge's region (0xff200000) and the hps bridge's region (0xc0000000) for use by 211the two child devices. 212 213Example: 214Base tree contains: 215 216 fpga_mgr: fpga-mgr@ff706000 { 217 compatible = "altr,socfpga-fpga-mgr"; 218 reg = <0xff706000 0x1000 219 0xffb90000 0x20>; 220 interrupts = <0 175 4>; 221 }; 222 223 fpga_bridge0: fpga-bridge@ff400000 { 224 compatible = "altr,socfpga-lwhps2fpga-bridge"; 225 reg = <0xff400000 0x100000>; 226 resets = <&rst LWHPS2FPGA_RESET>; 227 clocks = <&l4_main_clk>; 228 229 #address-cells = <1>; 230 #size-cells = <1>; 231 ranges; 232 233 fpga_region0: fpga-region0 { 234 compatible = "fpga-region"; 235 fpga-mgr = <&fpga_mgr>; 236 }; 237 }; 238 239 fpga_bridge1: fpga-bridge@ff500000 { 240 compatible = "altr,socfpga-hps2fpga-bridge"; 241 reg = <0xff500000 0x10000>; 242 resets = <&rst HPS2FPGA_RESET>; 243 clocks = <&l4_main_clk>; 244 }; 245 246Overlay contains: 247 248/dts-v1/ /plugin/; 249/ { 250 fragment@0 { 251 target = <&fpga_region0>; 252 #address-cells = <1>; 253 #size-cells = <1>; 254 __overlay__ { 255 #address-cells = <1>; 256 #size-cells = <1>; 257 258 firmware-name = "soc_system.rbf"; 259 fpga-bridges = <&fpga_bridge1>; 260 ranges = <0x20000 0xff200000 0x100000>, 261 <0x0 0xc0000000 0x20000000>; 262 263 gpio@10040 { 264 compatible = "altr,pio-1.0"; 265 reg = <0x10040 0x20>; 266 altr,ngpio = <4>; 267 #gpio-cells = <2>; 268 clocks = <2>; 269 gpio-controller; 270 }; 271 272 onchip-memory { 273 device_type = "memory"; 274 compatible = "altr,onchipmem-15.1"; 275 reg = <0x0 0x10000>; 276 }; 277 }; 278 }; 279}; 280 281 282Supported Use Models 283==================== 284 285In all cases the live DT must have the FPGA Manager, FPGA Bridges (if any), and 286a FPGA Region. The target of the Device Tree Overlay is the FPGA Region. Some 287uses are specific to a FPGA device. 288 289 * No FPGA Bridges 290 In this case, the FPGA Manager which programs the FPGA also handles the 291 bridges behind the scenes. No FPGA Bridge devices are needed for full 292 reconfiguration. 293 294 * Full reconfiguration with hardware bridges 295 In this case, there are hardware bridges between the processor and FPGA that 296 need to be controlled during full reconfiguration. Before the overlay is 297 applied, the live DT must include the FPGA Manager, FPGA Bridges, and a 298 FPGA Region. The FPGA Region is the child of the bridge that allows 299 register access to the FPGA. Additional bridges may be listed in a 300 fpga-bridges property in the FPGA region or in the device tree overlay. 301 302 * Partial reconfiguration with bridges in the FPGA 303 In this case, the FPGA will have one or more PRR's that may be programmed 304 separately while the rest of the FPGA can remain active. To manage this, 305 bridges need to exist in the FPGA that can gate the buses going to each FPGA 306 region while the buses are enabled for other sections. Before any partial 307 reconfiguration can be done, a base FPGA image must be loaded which includes 308 PRR's with FPGA bridges. The device tree should have a FPGA region for each 309 PRR. 310 311Device Tree Examples 312==================== 313 314The intention of this section is to give some simple examples, focusing on 315the placement of the elements detailed above, especially: 316 * FPGA Manager 317 * FPGA Bridges 318 * FPGA Region 319 * ranges 320 * target-path or target 321 322For the purposes of this section, I'm dividing the Device Tree into two parts, 323each with its own requirements. The two parts are: 324 * The live DT prior to the overlay being added 325 * The DT overlay 326 327The live Device Tree must contain an FPGA Region, an FPGA Manager, and any FPGA 328Bridges. The FPGA Region's "fpga-mgr" property specifies the manager by phandle 329to handle programming the FPGA. If the FPGA Region is the child of another FPGA 330Region, the parent's FPGA Manager is used. If FPGA Bridges need to be involved, 331they are specified in the FPGA Region by the "fpga-bridges" property. During 332FPGA programming, the FPGA Region will disable the bridges that are in its 333"fpga-bridges" list and will re-enable them after FPGA programming has 334succeeded. 335 336The Device Tree Overlay will contain: 337 * "target-path" or "target" 338 The insertion point where the the contents of the overlay will go into the 339 live tree. target-path is a full path, while target is a phandle. 340 * "ranges" 341 The address space mapping from processor to FPGA bus(ses). 342 * "firmware-name" 343 Specifies the name of the FPGA image file on the firmware search 344 path. The search path is described in the firmware class documentation. 345 * "partial-fpga-config" 346 This binding is a boolean and should be present if partial reconfiguration 347 is to be done. 348 * child nodes corresponding to hardware that will be loaded in this region of 349 the FPGA. 350 351Device Tree Example: Full Reconfiguration without Bridges 352========================================================= 353 354Live Device Tree contains: 355 fpga_mgr0: fpga-mgr@f8007000 { 356 compatible = "xlnx,zynq-devcfg-1.0"; 357 reg = <0xf8007000 0x100>; 358 interrupt-parent = <&intc>; 359 interrupts = <0 8 4>; 360 clocks = <&clkc 12>; 361 clock-names = "ref_clk"; 362 syscon = <&slcr>; 363 }; 364 365 fpga_region0: fpga-region0 { 366 compatible = "fpga-region"; 367 fpga-mgr = <&fpga_mgr0>; 368 #address-cells = <0x1>; 369 #size-cells = <0x1>; 370 ranges; 371 }; 372 373DT Overlay contains: 374/dts-v1/ /plugin/; 375/ { 376fragment@0 { 377 target = <&fpga_region0>; 378 #address-cells = <1>; 379 #size-cells = <1>; 380 __overlay__ { 381 #address-cells = <1>; 382 #size-cells = <1>; 383 384 firmware-name = "zynq-gpio.bin"; 385 386 gpio1: gpio@40000000 { 387 compatible = "xlnx,xps-gpio-1.00.a"; 388 reg = <0x40000000 0x10000>; 389 gpio-controller; 390 #gpio-cells = <0x2>; 391 xlnx,gpio-width= <0x6>; 392 }; 393 }; 394}; 395 396Device Tree Example: Full Reconfiguration to add PRR's 397====================================================== 398 399The base FPGA Region is specified similar to the first example above. 400 401This example programs the FPGA to have two regions that can later be partially 402configured. Each region has its own bridge in the FPGA fabric. 403 404DT Overlay contains: 405/dts-v1/ /plugin/; 406/ { 407 fragment@0 { 408 target = <&fpga_region0>; 409 #address-cells = <1>; 410 #size-cells = <1>; 411 __overlay__ { 412 #address-cells = <1>; 413 #size-cells = <1>; 414 415 firmware-name = "base.rbf"; 416 417 fpga-bridge@4400 { 418 compatible = "altr,freeze-bridge-controller"; 419 reg = <0x4400 0x10>; 420 421 fpga_region1: fpga-region1 { 422 compatible = "fpga-region"; 423 #address-cells = <0x1>; 424 #size-cells = <0x1>; 425 ranges; 426 }; 427 }; 428 429 fpga-bridge@4420 { 430 compatible = "altr,freeze-bridge-controller"; 431 reg = <0x4420 0x10>; 432 433 fpga_region2: fpga-region2 { 434 compatible = "fpga-region"; 435 #address-cells = <0x1>; 436 #size-cells = <0x1>; 437 ranges; 438 }; 439 }; 440 }; 441 }; 442}; 443 444Device Tree Example: Partial Reconfiguration 445============================================ 446 447This example reprograms one of the PRR's set up in the previous example. 448 449The sequence that occurs when this overlay is similar to the above, the only 450differences are that the FPGA is partially reconfigured due to the 451"partial-fpga-config" boolean and the only bridge that is controlled during 452programming is the FPGA based bridge of fpga_region1. 453 454/dts-v1/ /plugin/; 455/ { 456 fragment@0 { 457 target = <&fpga_region1>; 458 #address-cells = <1>; 459 #size-cells = <1>; 460 __overlay__ { 461 #address-cells = <1>; 462 #size-cells = <1>; 463 464 firmware-name = "soc_image2.rbf"; 465 partial-fpga-config; 466 467 gpio@10040 { 468 compatible = "altr,pio-1.0"; 469 reg = <0x10040 0x20>; 470 clocks = <0x2>; 471 altr,ngpio = <0x4>; 472 #gpio-cells = <0x2>; 473 gpio-controller; 474 }; 475 }; 476 }; 477}; 478 479Constraints 480=========== 481 482It is beyond the scope of this document to fully describe all the FPGA design 483constraints required to make partial reconfiguration work[1] [2] [3], but a few 484deserve quick mention. 485 486A persona must have boundary connections that line up with those of the partion 487or region it is designed to go into. 488 489During programming, transactions through those connections must be stopped and 490the connections must be held at a fixed logic level. This can be achieved by 491FPGA Bridges that exist on the FPGA fabric prior to the partial reconfiguration. 492 493-- 494[1] www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/ug/ug_partrecon.pdf 495[2] tspace.library.utoronto.ca/bitstream/1807/67932/1/Byma_Stuart_A_201411_MAS_thesis.pdf 496[3] https://www.xilinx.com/support/documentation/sw_manuals/xilinx14_1/ug702.pdf 497