1FPGA Region Device Tree Binding 2 3Alan Tull 2016 4 5 CONTENTS 6 - Introduction 7 - Terminology 8 - Sequence 9 - FPGA Region 10 - Supported Use Models 11 - Device Tree Examples 12 - Constraints 13 14 15Introduction 16============ 17 18FPGA Regions represent FPGA's and partial reconfiguration regions of FPGA's in 19the Device Tree. FPGA Regions provide a way to program FPGAs under device tree 20control. 21 22This device tree binding document hits some of the high points of FPGA usage and 23attempts to include terminology used by both major FPGA manufacturers. This 24document isn't a replacement for any manufacturers specifications for FPGA 25usage. 26 27 28Terminology 29=========== 30 31Full Reconfiguration 32 * The entire FPGA is programmed. 33 34Partial Reconfiguration (PR) 35 * A section of an FPGA is reprogrammed while the rest of the FPGA is not 36 affected. 37 * Not all FPGA's support PR. 38 39Partial Reconfiguration Region (PRR) 40 * Also called a "reconfigurable partition" 41 * A PRR is a specific section of an FPGA reserved for reconfiguration. 42 * A base (or static) FPGA image may create a set of PRR's that later may 43 be independently reprogrammed many times. 44 * The size and specific location of each PRR is fixed. 45 * The connections at the edge of each PRR are fixed. The image that is loaded 46 into a PRR must fit and must use a subset of the region's connections. 47 * The busses within the FPGA are split such that each region gets its own 48 branch that may be gated independently. 49 50Persona 51 * Also called a "partial bit stream" 52 * An FPGA image that is designed to be loaded into a PRR. There may be 53 any number of personas designed to fit into a PRR, but only one at at time 54 may be loaded. 55 * A persona may create more regions. 56 57FPGA Bridge 58 * FPGA Bridges gate bus signals between a host and FPGA. 59 * FPGA Bridges should be disabled while the FPGA is being programmed to 60 prevent spurious signals on the cpu bus and to the soft logic. 61 * FPGA bridges may be actual hardware or soft logic on an FPGA. 62 * During Full Reconfiguration, hardware bridges between the host and FPGA 63 will be disabled. 64 * During Partial Reconfiguration of a specific region, that region's bridge 65 will be used to gate the busses. Traffic to other regions is not affected. 66 * In some implementations, the FPGA Manager transparantly handles gating the 67 buses, eliminating the need to show the hardware FPGA bridges in the 68 device tree. 69 * An FPGA image may create a set of reprogrammable regions, each having its 70 own bridge and its own split of the busses in the FPGA. 71 72FPGA Manager 73 * An FPGA Manager is a hardware block that programs an FPGA under the control 74 of a host processor. 75 76Base Image 77 * Also called the "static image" 78 * An FPGA image that is designed to do full reconfiguration of the FPGA. 79 * A base image may set up a set of partial reconfiguration regions that may 80 later be reprogrammed. 81 82 ---------------- ---------------------------------- 83 | Host CPU | | FPGA | 84 | | | | 85 | ----| | ----------- -------- | 86 | | H | | |==>| Bridge0 |<==>| PRR0 | | 87 | | W | | | ----------- -------- | 88 | | | | | | 89 | | B |<=====>|<==| ----------- -------- | 90 | | R | | |==>| Bridge1 |<==>| PRR1 | | 91 | | I | | | ----------- -------- | 92 | | D | | | | 93 | | G | | | ----------- -------- | 94 | | E | | |==>| Bridge2 |<==>| PRR2 | | 95 | ----| | ----------- -------- | 96 | | | | 97 ---------------- ---------------------------------- 98 99Figure 1: An FPGA set up with a base image that created three regions. Each 100region (PRR0-2) gets its own split of the busses that is independently gated by 101a soft logic bridge (Bridge0-2) in the FPGA. The contents of each PRR can be 102reprogrammed independently while the rest of the system continues to function. 103 104 105Sequence 106======== 107 108When a DT overlay that targets an FPGA Region is applied, the FPGA Region will 109do the following: 110 111 1. Disable appropriate FPGA bridges. 112 2. Program the FPGA using the FPGA manager. 113 3. Enable the FPGA bridges. 114 4. The Device Tree overlay is accepted into the live tree. 115 5. Child devices are populated. 116 117When the overlay is removed, the child nodes will be removed and the FPGA Region 118will disable the bridges. 119 120 121FPGA Region 122=========== 123 124FPGA Regions represent FPGA's and FPGA PR regions in the device tree. An FPGA 125Region brings together the elements needed to program on a running system and 126add the child devices: 127 128 * FPGA Manager 129 * FPGA Bridges 130 * image-specific information needed to to the programming. 131 * child nodes 132 133The intended use is that a Device Tree overlay (DTO) can be used to reprogram an 134FPGA while an operating system is running. 135 136An FPGA Region that exists in the live Device Tree reflects the current state. 137If the live tree shows a "firmware-name" property or child nodes under an FPGA 138Region, the FPGA already has been programmed. A DTO that targets an FPGA Region 139and adds the "firmware-name" property is taken as a request to reprogram the 140FPGA. After reprogramming is successful, the overlay is accepted into the live 141tree. 142 143The base FPGA Region in the device tree represents the FPGA and supports full 144reconfiguration. It must include a phandle to an FPGA Manager. The base 145FPGA region will be the child of one of the hardware bridges (the bridge that 146allows register access) between the cpu and the FPGA. If there are more than 147one bridge to control during FPGA programming, the region will also contain a 148list of phandles to the additional hardware FPGA Bridges. 149 150For partial reconfiguration (PR), each PR region will have an FPGA Region. 151These FPGA regions are children of FPGA bridges which are then children of the 152base FPGA region. The "Full Reconfiguration to add PRR's" example below shows 153this. 154 155If an FPGA Region does not specify an FPGA Manager, it will inherit the FPGA 156Manager specified by its ancestor FPGA Region. This supports both the case 157where the same FPGA Manager is used for all of an FPGA as well the case where 158a different FPGA Manager is used for each region. 159 160FPGA Regions do not inherit their ancestor FPGA regions' bridges. This prevents 161shutting down bridges that are upstream from the other active regions while one 162region is getting reconfigured (see Figure 1 above). During PR, the FPGA's 163hardware bridges remain enabled. The PR regions' bridges will be FPGA bridges 164within the static image of the FPGA. 165 166Required properties: 167- compatible : should contain "fpga-region" 168- fpga-mgr : should contain a phandle to an FPGA Manager. Child FPGA Regions 169 inherit this property from their ancestor regions. An fpga-mgr property 170 in a region will override any inherited FPGA manager. 171- #address-cells, #size-cells, ranges : must be present to handle address space 172 mapping for child nodes. 173 174Optional properties: 175- firmware-name : should contain the name of an FPGA image file located on the 176 firmware search path. If this property shows up in a live device tree 177 it indicates that the FPGA has already been programmed with this image. 178 If this property is in an overlay targeting an FPGA region, it is a 179 request to program the FPGA with that image. 180- fpga-bridges : should contain a list of phandles to FPGA Bridges that must be 181 controlled during FPGA programming along with the parent FPGA bridge. 182 This property is optional if the FPGA Manager handles the bridges. 183 If the fpga-region is the child of an fpga-bridge, the list should not 184 contain the parent bridge. 185- partial-fpga-config : boolean, set if partial reconfiguration is to be done, 186 otherwise full reconfiguration is done. 187- external-fpga-config : boolean, set if the FPGA has already been configured 188 prior to OS boot up. 189- encrypted-fpga-config : boolean, set if the bitstream is encrypted 190- region-unfreeze-timeout-us : The maximum time in microseconds to wait for 191 bridges to successfully become enabled after the region has been 192 programmed. 193- region-freeze-timeout-us : The maximum time in microseconds to wait for 194 bridges to successfully become disabled before the region has been 195 programmed. 196- config-complete-timeout-us : The maximum time in microseconds time for the 197 FPGA to go to operating mode after the region has been programmed. 198- child nodes : devices in the FPGA after programming. 199 200In the example below, when an overlay is applied targeting fpga-region0, 201fpga_mgr is used to program the FPGA. Two bridges are controlled during 202programming: the parent fpga_bridge0 and fpga_bridge1. Because the region is 203the child of fpga_bridge0, only fpga_bridge1 needs to be specified in the 204fpga-bridges property. During programming, these bridges are disabled, the 205firmware specified in the overlay is loaded to the FPGA using the FPGA manager 206specified in the region. If FPGA programming succeeds, the bridges are 207reenabled and the overlay makes it into the live device tree. The child devices 208are then populated. If FPGA programming fails, the bridges are left disabled 209and the overlay is rejected. The overlay's ranges property maps the lwhps 210bridge's region (0xff200000) and the hps bridge's region (0xc0000000) for use by 211the two child devices. 212 213Example: 214Base tree contains: 215 216 fpga_mgr: fpga-mgr@ff706000 { 217 compatible = "altr,socfpga-fpga-mgr"; 218 reg = <0xff706000 0x1000 219 0xffb90000 0x20>; 220 interrupts = <0 175 4>; 221 }; 222 223 fpga_bridge0: fpga-bridge@ff400000 { 224 compatible = "altr,socfpga-lwhps2fpga-bridge"; 225 reg = <0xff400000 0x100000>; 226 resets = <&rst LWHPS2FPGA_RESET>; 227 clocks = <&l4_main_clk>; 228 229 #address-cells = <1>; 230 #size-cells = <1>; 231 ranges; 232 233 fpga_region0: fpga-region0 { 234 compatible = "fpga-region"; 235 fpga-mgr = <&fpga_mgr>; 236 }; 237 }; 238 239 fpga_bridge1: fpga-bridge@ff500000 { 240 compatible = "altr,socfpga-hps2fpga-bridge"; 241 reg = <0xff500000 0x10000>; 242 resets = <&rst HPS2FPGA_RESET>; 243 clocks = <&l4_main_clk>; 244 }; 245 246Overlay contains: 247 248/dts-v1/; 249/plugin/; 250 251&fpga_region0 { 252 #address-cells = <1>; 253 #size-cells = <1>; 254 255 firmware-name = "soc_system.rbf"; 256 fpga-bridges = <&fpga_bridge1>; 257 ranges = <0x20000 0xff200000 0x100000>, 258 <0x0 0xc0000000 0x20000000>; 259 260 gpio@10040 { 261 compatible = "altr,pio-1.0"; 262 reg = <0x10040 0x20>; 263 altr,ngpio = <4>; 264 #gpio-cells = <2>; 265 clocks = <2>; 266 gpio-controller; 267 }; 268 269 onchip-memory { 270 device_type = "memory"; 271 compatible = "altr,onchipmem-15.1"; 272 reg = <0x0 0x10000>; 273 }; 274}; 275 276 277Supported Use Models 278==================== 279 280In all cases the live DT must have the FPGA Manager, FPGA Bridges (if any), and 281a FPGA Region. The target of the Device Tree Overlay is the FPGA Region. Some 282uses are specific to an FPGA device. 283 284 * No FPGA Bridges 285 In this case, the FPGA Manager which programs the FPGA also handles the 286 bridges behind the scenes. No FPGA Bridge devices are needed for full 287 reconfiguration. 288 289 * Full reconfiguration with hardware bridges 290 In this case, there are hardware bridges between the processor and FPGA that 291 need to be controlled during full reconfiguration. Before the overlay is 292 applied, the live DT must include the FPGA Manager, FPGA Bridges, and a 293 FPGA Region. The FPGA Region is the child of the bridge that allows 294 register access to the FPGA. Additional bridges may be listed in a 295 fpga-bridges property in the FPGA region or in the device tree overlay. 296 297 * Partial reconfiguration with bridges in the FPGA 298 In this case, the FPGA will have one or more PRR's that may be programmed 299 separately while the rest of the FPGA can remain active. To manage this, 300 bridges need to exist in the FPGA that can gate the buses going to each FPGA 301 region while the buses are enabled for other sections. Before any partial 302 reconfiguration can be done, a base FPGA image must be loaded which includes 303 PRR's with FPGA bridges. The device tree should have an FPGA region for each 304 PRR. 305 306Device Tree Examples 307==================== 308 309The intention of this section is to give some simple examples, focusing on 310the placement of the elements detailed above, especially: 311 * FPGA Manager 312 * FPGA Bridges 313 * FPGA Region 314 * ranges 315 * target-path or target 316 317For the purposes of this section, I'm dividing the Device Tree into two parts, 318each with its own requirements. The two parts are: 319 * The live DT prior to the overlay being added 320 * The DT overlay 321 322The live Device Tree must contain an FPGA Region, an FPGA Manager, and any FPGA 323Bridges. The FPGA Region's "fpga-mgr" property specifies the manager by phandle 324to handle programming the FPGA. If the FPGA Region is the child of another FPGA 325Region, the parent's FPGA Manager is used. If FPGA Bridges need to be involved, 326they are specified in the FPGA Region by the "fpga-bridges" property. During 327FPGA programming, the FPGA Region will disable the bridges that are in its 328"fpga-bridges" list and will re-enable them after FPGA programming has 329succeeded. 330 331The Device Tree Overlay will contain: 332 * "target-path" or "target" 333 The insertion point where the contents of the overlay will go into the 334 live tree. target-path is a full path, while target is a phandle. 335 * "ranges" 336 The address space mapping from processor to FPGA bus(ses). 337 * "firmware-name" 338 Specifies the name of the FPGA image file on the firmware search 339 path. The search path is described in the firmware class documentation. 340 * "partial-fpga-config" 341 This binding is a boolean and should be present if partial reconfiguration 342 is to be done. 343 * child nodes corresponding to hardware that will be loaded in this region of 344 the FPGA. 345 346Device Tree Example: Full Reconfiguration without Bridges 347========================================================= 348 349Live Device Tree contains: 350 fpga_mgr0: fpga-mgr@f8007000 { 351 compatible = "xlnx,zynq-devcfg-1.0"; 352 reg = <0xf8007000 0x100>; 353 interrupt-parent = <&intc>; 354 interrupts = <0 8 4>; 355 clocks = <&clkc 12>; 356 clock-names = "ref_clk"; 357 syscon = <&slcr>; 358 }; 359 360 fpga_region0: fpga-region0 { 361 compatible = "fpga-region"; 362 fpga-mgr = <&fpga_mgr0>; 363 #address-cells = <0x1>; 364 #size-cells = <0x1>; 365 ranges; 366 }; 367 368DT Overlay contains: 369 370/dts-v1/; 371/plugin/; 372 373&fpga_region0 { 374 #address-cells = <1>; 375 #size-cells = <1>; 376 377 firmware-name = "zynq-gpio.bin"; 378 379 gpio1: gpio@40000000 { 380 compatible = "xlnx,xps-gpio-1.00.a"; 381 reg = <0x40000000 0x10000>; 382 gpio-controller; 383 #gpio-cells = <0x2>; 384 xlnx,gpio-width= <0x6>; 385 }; 386}; 387 388Device Tree Example: Full Reconfiguration to add PRR's 389====================================================== 390 391The base FPGA Region is specified similar to the first example above. 392 393This example programs the FPGA to have two regions that can later be partially 394configured. Each region has its own bridge in the FPGA fabric. 395 396DT Overlay contains: 397 398/dts-v1/; 399/plugin/; 400 401&fpga_region0 { 402 #address-cells = <1>; 403 #size-cells = <1>; 404 405 firmware-name = "base.rbf"; 406 407 fpga-bridge@4400 { 408 compatible = "altr,freeze-bridge-controller"; 409 reg = <0x4400 0x10>; 410 411 fpga_region1: fpga-region1 { 412 compatible = "fpga-region"; 413 #address-cells = <0x1>; 414 #size-cells = <0x1>; 415 ranges; 416 }; 417 }; 418 419 fpga-bridge@4420 { 420 compatible = "altr,freeze-bridge-controller"; 421 reg = <0x4420 0x10>; 422 423 fpga_region2: fpga-region2 { 424 compatible = "fpga-region"; 425 #address-cells = <0x1>; 426 #size-cells = <0x1>; 427 ranges; 428 }; 429 }; 430}; 431 432Device Tree Example: Partial Reconfiguration 433============================================ 434 435This example reprograms one of the PRR's set up in the previous example. 436 437The sequence that occurs when this overlay is similar to the above, the only 438differences are that the FPGA is partially reconfigured due to the 439"partial-fpga-config" boolean and the only bridge that is controlled during 440programming is the FPGA based bridge of fpga_region1. 441 442/dts-v1/; 443/plugin/; 444 445&fpga_region1 { 446 #address-cells = <1>; 447 #size-cells = <1>; 448 449 firmware-name = "soc_image2.rbf"; 450 partial-fpga-config; 451 452 gpio@10040 { 453 compatible = "altr,pio-1.0"; 454 reg = <0x10040 0x20>; 455 clocks = <0x2>; 456 altr,ngpio = <0x4>; 457 #gpio-cells = <0x2>; 458 gpio-controller; 459 }; 460}; 461 462Constraints 463=========== 464 465It is beyond the scope of this document to fully describe all the FPGA design 466constraints required to make partial reconfiguration work[1] [2] [3], but a few 467deserve quick mention. 468 469A persona must have boundary connections that line up with those of the partion 470or region it is designed to go into. 471 472During programming, transactions through those connections must be stopped and 473the connections must be held at a fixed logic level. This can be achieved by 474FPGA Bridges that exist on the FPGA fabric prior to the partial reconfiguration. 475 476-- 477[1] www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/ug/ug_partrecon.pdf 478[2] tspace.library.utoronto.ca/bitstream/1807/67932/1/Byma_Stuart_A_201411_MAS_thesis.pdf 479[3] https://www.xilinx.com/support/documentation/sw_manuals/xilinx14_1/ug702.pdf 480