Microkernel Dedicated for Dynamic Partial Reconfiguration on ARM-FPGA Platform Tian Xia, Jean-Christophe Prévotet and Fabienne Nouvel Université Europe de Bretagne, France INSA, IETR, UMR 6164, F-35708 RENNES {tian.xia; jean-christophe.prevotet; fabienne.nouvel}@insa-rennes.fr ABSTRACT [1]. With DPR feature, hardware accelerators can be dy- This paper describes the first implementation of a custom namically dispatched and managed, becoming as flexible as micro-kernel on a ARM-FPGA platform capable of manag- software functions. ing reconfigurable hardware parts dynamically. After de- On the other hand, with the widespread applications of scribing the structure of the proposed micro-kernel, we will handheld devices, reliability and security of embedded sys- focus on a custom specific system task dealing with the re- tems have become a serious concern. Dealing with microker- configuration management, which is associated to a dedi- nels constitutes a promising idea because it allows the user cated scheduling mechanism. We will describe the hardware to execute various applications (commodity APIs, real-time platform on which the microkernel has been ported and pro- tasks, etc.) in their own isolated container to ensure isola- vide a use case application in order to demonstrate the fea- tion and thus security. Consequently, it has been a popular sibility of the approach. At the end of this paper, we will research trend in the embedded systems domain for many provide quantitative results in terms of reconfiguration over- years[2]. head and microkernel timing performances. In this paper, we describe and study a custom embedded microkernel on a hybrid ARM-FPGA Zynq-7000 platform [3]. This microkernel is a revised version of the NOVA mi- Keywords crohypervisor [4], and is integrated with the management Microkernel, Real-Time Systems, FPGA, Embedded Sys- and scheduling of reconfigurable hardware resources. This tem, Reconfigurable Architectures proposed architecture allows for dynamic management of SW/HW tasks, secure task isolation and efficient SW/HW 1. INTRODUCTION communication. The remainder of the paper is organized as follows: Sec- During the last decades, with the development of com- tion 2 presents current researches in management of DPR ar- modity field-programmable gate array (FPGA), the tech- chitectures. In Section 3, an overall architecture of the pro- nique of reconfigurable computing has gained increasing at- posed platform is introduced. Section 4 focuses on the design tention for its potential in exploiting hardware resources. and implementation of the microkernel, with detailed intro- Through time-multiplexed sharing of the FPGA fabric, a duction to the hardware tasks management and scheduling higher integration of functionalities can be achieved. The mechanisms. In Section 5, we present a case study to demon- main drawback of traditional FPGA reconfiguration com- strate the capabilities of the proposed microkernel. Finally puting is the lack of flexibility, because the whole fabric is section 6 concludes the paper. required to be reconfigured even when modification is only required for a part of the FPGA. As a consequence, enor- mous time overhead and power consumption are produced, 2. RELATED WORK which severely limits reconfiguration in embedded systems. Compared with the traditional full reconfiguration mech- As a solution, a more advanced technique enabling to re- anism, the DPR technique benefits from the following major configure particular areas of an FPGA while the rest contin- advantages [3]: ues executing has been proposed and is known as Dynamic Partial Reconfiguration (DPR). This technique has proved • Reduced hardware resource utilization to be quite prospective in the embedded domain because • Improved design efficiency of its runtime adaptivity for hardware algorithms and lower power consumption compared to large-scale static circuits • Reduced reconfiguration latency and better robustness Despite of the enhanced flexibility provided by DPR tech- niques, the reconfiguration overhead remains a crucial issue in practice. In modern high-end FPGAs which may have tens of millions of configuration points, one reconfiguration of a complex module will be very time-consuming. Numer- ous studies have been led to propose efficient hardware re- configuration management with dedicated architecture and EWiLi’14, November 2014, Lisbon, Portugal. OS support. A custom DPR controller was introduced in Copyright retained by the authors. [5] to realize high-speed on-chip reconfiguration. In [6], a specific operating system CAP-OS was proposed to provide FPGA fabric is engaged in several hardware acceleration op- clients with hardware task management and priority-based erations, which are executing concurrently with SW tasks. scheduling. Other researches were made in the OveRSoC A specific hardware task management routine is proposed to project, which provided a model at high-level abstraction control and reconfigure the hardware accelerators dynami- and allowed to efficiently simulate and validate embedded cally. Such a routine runs as a guest to the microkernel and RTOS for reconfigurable platforms [7]. Most researches on is scheduled whenever the management of HW tasks is re- traditional DPR devices (i.e. Virtex FPGA family) employ quired. The mechanism related to this part will be described embedded processors such as MicroBlaze or PowerPC, whose in detail in Section 4. In this way, the FPGA resource is computing ability is relatively limited. seen as a standard user application by the microkernel and Compared to classical devices, the Zynq-7000 platform thereby hardware and software tasks can be managed con- integrates the ARM Cortex-A9 processor with various on- currently in our framework. board resources and brings up enormous possibilities for em- bedded techniques. In this platform, the programmable fab- 3.1 Hardware Tasks ric is considered as a unique auxiliary computing resource In a reconfigurable embedded system, hardware tasks are to this fully capable processing system, and the reconfigu- implemented using functional fabric structures in the FPGA, ration management is expected to be one of many tasks in which can be user-defined computing blocks or commercial the system. Hence, a specific kernel is the ideal solution to IP cores. As shown in Fig. 1, the FPGA fabric is divided in- rationally dispatch both hardware and software resources. to multiple partial reconfigurable black boxes or containers, While considerable efforts have been made to port micro- which are capable of housing hardware tasks independent- kernel techniques to traditional embedded systems, such as ly. These containers are defined as partial reconfigurable the OKL4 from Open Kernel Labs [2], most existing micro- regions (PRR). The hardware task which is running in each kernels on ARM do not consider reconfigurable hardware. container is run-time switchable under the control of the Instead, most of the works only use a micro-kernel to man- hardware task manager. Different sizes of blocks are allo- age heterogeneous platforms i.e. software and static hard- cated to different PRRs for different task purposes. ware parts. For example, in [8], a L4 kernel is ported to The resource that holds the fabric information of hardware manage hardware and software tasks, but without using dy- tasks is contained in a bitstream file. Different bitstream namic reconfiguration. In parallel, research in [9] discussed files can be stored in various memory devices and be ac- the reconfiguration management on Zynq platform at the cessed via a simple look-up table. Note that, the container application level, without using any operating system and corresponding to each HW task has always the same con- thus with poor flexibility. strained location in the FPGA. A HW task is dispatched by transferring the corresponding bitstream file to the assigned PRR. Normally, HW tasks with similar or close function- 3. PROPOSED PLATFORM alities should be distributed to the same PRR, so that the The motivation of the proposed hybrid ARM-FPGA plat- coherence of HW task interfaces can be guaranteed. Each form framework is to establish a user-practical environment HW task should have one corresponding SW application to with a highly abstract microkernel. The management of monitor and control its behaviour. hardware resources is integrated as a user application, with One of the crucial features regarding hardware tasks is relatively easy access. Both software and hardware tasks are the reconfiguration overhead, which is linearly correlated to registered and scheduled by a custom microkernel. A block the size of the bitstream, thus, the PRR size. This means diagram of the proposed platform is shown in Fig. 1. all HW tasks implemented in the same PRR will have the On the proposed platform, computing resources are divid- same time overhead for reconfiguration. ed into Processing System (PS) and Programmable Logic (PL). On the PS side, a simplified microkernel hosts mul- 3.2 HW/SW Task Communication tiple software applications, including Guest OSes and user To connect PL with PS, two interface types based on the applications executing within the user space. Each appli- standard AXI bus protocol are employed. Offering a unified cation is housed in an individual isolated space of the mi- mapping to the processor and being accessed as a normal crokernel, which is referenced as an execution context (EC). memory access, the AXI GP is intended for low-speed gen- By scheduling and switching ECs, the ARM processor is eral purpose communication. As in Fig. 1, the processing shared among guests according to time multiplexing. The system takes control of two master AXI GP interfaces as main methods to configure and read back the states of the HW ta sk data Use r I/O HW Task HW tasks. GIC Bit Bit Bit App Driver Manager AXI HP is aimed for high performance data exchange file file file Microke rne l DecCfg with burst transfer, which may transfer data blocks as large DDR ARM Cor tex-A9 as 4KB in one burst, and is sufficient for generic data pro- AXI Inter connection PCAP cessing applications. On our platform, 4 AXI HP interfaces PS are used and in charge of accessing both on chip memo- AXI4-HP M aster AXI4-Lite S lave ry(OCM) and DDR. Since HW tasks access AXI HP as mas- PRR Controller Config. Reg1 ters, data is fetched and written back without acknowledging PPR reg group PPR reg group PPR reg group Config. Reg2 the processor, allowing the processor to run simultaneously Config. Reg3 with HW tasks. PL PRR1 PRR2 PRR3 Config. 3.3 Reconfiguration Interface Two methods for partial reconfiguration are supported on Figure 1: Diagram of the Proposed Hybrid Platform connected to the generic interrupt controller (GIC). 8 SPI Table 1: Description of PRR Configuration Regis- resources are used to provide the PS with different HW task ters information such as task completion or critical errors. Reg Name Width Description Mark process status: Bit[0]: start data processing Proc status 16 Bit[1]: pause data processing 4. REAL TIME MICROKERNEL Bit[2]: interrupt handling over To facilitate the management of multiple guest SW appli- Mark interrupt status: PRR Int status 16 Bit[0:7]: PRRs interrupt enable cations and HW tasks, we developed a simplified microkernel Bit[8:15]: PRRs interrupt status based on Mini-NOVA, one revision of the NOVA hypervi- Mark PRR enable status: sor. In this section, we propose a specific HW task manager PRR status 16 Bit[0:7]: PRRs enable Bit[8:15]: PRRs switch enable. service and a scheduling strategy to support dynamic PR PRR’s status for reconfiguration: management. PRR Reco rdy 16 Bit[0:7]: PRR ready PRR delay 32 Time overhead for current reconfiguration 4.1 Microkernel Description General-purpose registers defined by user: HW PRR gpr[7:0] 32 task ID, working mode, parameters, etc. The microkernel runs on top of bare-metal hardware. By implementing the basic OS functionalities, the microkernel the Zynq platform: Processor Configuration Access Port (P- establishes an abstraction layer of the hardware platform to CAP) and Internal Configuration Access Port (ICAP). Us- user applications. The application of microkernel benefits ing PCAP, as shown in the datapath of Fig.1, PS is en- in the way that higher security level can be achieved with abled to initialize bitstream transfers from memory to PL virtualization technique. One of the essential features of this through the Device Configuration Interface (DevCfg) at high microkernel is security, so the principle of least privilege is throughput (130MB/s). In contrast, ICAP is designed for strictly followed in our framework to make sure that a min- self-configuration from the PL side with a AXI4-Lite as imal tested computing base (TCB) is achieved. Such a fea- transfer port. Such a mechanism severely limits the recon- ture will also improve performance with a quicker execution figuration speed (19MB/s). ICAP is less interesting also of context switches. because it requires additional hardware resources and will The proposed microkernel has simplified functionality and occupy at least one AXI interface. On our platform, PCAP reduced complexity, which makes it more suitable for em- is selected for its better compatibility with software appli- bedded systems and also more adaptable. Since the initial cations and higher throughput. Mini-NOVA is designed for x86 architecture, several modifi- cations have been made to execute on the ARM Cortex-A9 3.4 PRR Controller Block which is available on the Zynq-7000 platform. Besides, addi- As shown in Fig. 1, a PRR controller block is introduced tional mechanisms and a new scheduling strategy have also to monitor and manage the states of HW tasks. This block been provided to the system. The main features of the pro- runs as a state machine under the supervision of the HW posed microkernel are: task manager. Through the AXI GP interface, we have im- plemented a group of configuration registers(PPR reg group) • Modified bootloader and boot sequence for both Zynq which are mapped into memory space and accessible to the platform (e.g. FPGA initialization, DDR initializa- processor. By configuring these registers, a SW service is tion, etc.) and ARM Cortex-A9 processor(e.g. kernel able to set up HW tasks, such as defining working modes, boot, user boot, paging table, exception vector, etc.) and data address. Since the number of PRRs is pre-fixed, we • Separate virtual memory mapping for kernel and user provide each PRR a PPR reg group for configuration. The space while providing isolated execution context for context of the registers is left for user-definition to adjust to each user application different HW tasks. Table 1 describes the configuration of • System calls and IRQs provided to user applications this register group. • Specific Priority-based round-robin to support PR 3.4.1 Reconfiguration security • Supporting virtualized OS(e.g. uC/OS-II) In case of a PRR reconfiguration, a switch of HW task is normally required. The PRR Controller is proposed to We should note that, to minimize the TCB size of the guarantee the HW task security, avoiding invalid data out- kernel and guarantee system security, most board-specific put and undesired task state. Based on these considerations, support APIs and services are implemented in user space, following features are included: including HW task manager, AXI support, and supports for • In case of a certain multi-block pipeline structure, the on-board peripheral resources (UART, SD card, interrupt pipeline should be emptied before any HW task switch, controller, TCC Timer, etc.). so that invalid output data are avoided. The virtual memory space of our system is divided into several domains. As described in Table 2, the kernel space • To maintain the integrity of the data structure being and user space are access-isolated by virtual mapping. A processed, the PRR controller avoids reconfigurations range of 256MB memory space on the upper side is dis- interrupting of data frames. tributed to the microkernel, whereas user applications exe- • A reset should be asserted to initialize the reconfigured cute in the lower memory space. Besides the user space and PRR before being allowed to be activated. kernel space, an extra space up to 256MB is allocated to store the bitstream files dealing with HW tasks. This area 3.4.2 Interrupts Management is programmed to be only accessible from the user space. The PRR controller is able to generate general-purpose The execution context (EC) is the major kernel object, interrupts through the Shared Peripheral Interrupts (SPI) which is the abstraction of user threads or applications in Bitfile descriptor table Syscall_HW_Manager(1, 0, arg01, arg02, arg03) id addr length dalay prr_id Table 2: System Address Mapping 1 A1 L1 D1 1 Name Addr Range Accessibility Description EC SW HW task 0xC0000000 - SW App Application Kernel Kernel Kernel space data 0xDFFFFFFF Rescheduling() Syscall_yield() User 0x0 - 0x2FFFFFFF Kernel, User User space Bitfile1 0x30000000 - Kernel, User, Bitstreams, Bitfile2 HW Task EC HW 0x3FFFFFFF PL HW task data HW Manager Manager PCAP Bitfile3 0x40000000 - PL User (AXI GP) PL Memory Space prr_ transferbitfile() 0xBFFFFFFF kernel user 0xE0000000 - Platform and AXI4-Lite Config AXI4 Peripheral Kernel, User 0xFDFFFFFF Peripheral regs arg01 ~arg03 Proc_status Table 3: Structure of the bit descriptor Class PPR_stastus Regs1 Regs2 Regs3 Obj. member id addr len delay prr id Int_status PRR1 PRR2 PRR3 HW task Bitfile Bitfile Reconfig. PRR PPR_delay Contents ID Address Length Overhead ID the kernel space. Each EC is exclusively attached to one us- Figure 2: Execution of the HW Task Manager er application and is able to maintain and manipulate user applications’ features such as the CPU/FPU register state, A block diagram describing the execution of the HW task stack location, and scheduling sequence. By resuming its manager is shown in Fig. 2. As demonstrated, a specific EC, a given task can be completely restored. When sensi- system call from user space will require the kernel to launch tive operations (page allocation, thread creation, cache op- the HW task manager. Arguments are passed through to eration, etc.) are required, the user space may access the the HW manager. The prototype of this specific system call kernel services by generating system calls, which are also is: handled through an EC. Syscall HW Manager(HW id, irq en, arg01, arg02, arg03) 4.2 HW Task Manager By handling this system call, the kernel invokes a resched- The HW task manager is defined as a special user appli- ule process and returns to user space, passing control to the cation serving other applications. Though executed in user HW task manager. In this process, arguments are also de- space, this service cooperates closely with the kernel and is livered to the HW manager. The HW manager will compare an essential part of the PR control flow in the system. In the HW id with the executing HW task. If it is already im- the following, we describe its different features. plemented in PRR, then only the parameters are changed by writing arguments to the register group, otherwise a PCAP 4.2.1 Bitstreams Management transfer will be configured to reload the target PRR with The switch of HW tasks is based on the download of dif- the desired HW task. The irq en argument will indicate ferent bitstream files. As introduced in Section 3, each bit- whether the PL interrupt is enabled for the corresponding stream corresponds to one HW task, and its PRR container PRR by setting values in the PRR Int status register. After is pre-fixed (but not exclusive), which also determines its accomplishing the required operation, the HW task manager reconfiguration overhead. All bitstream files are loaded to gives back control to the previously interrupted application. the HW task memory space shown in Table 2, at the kernel In some cases, a PR request cannot be acknowledged im- bootload stage. A descriptor is provided to each available mediately. As the scenarios described in Section 3, a HW bitstream file by defining a bit descriptor class. We also cre- task may be in the middle of a data frame process and not ated a look-up table for all bit descriptor objects indexed by ready for reconfiguration. In such situations, to avoid mo- a unique ID number. In fact, the object members given in nopolizing the CPU, the HW task manager will be pulled Table 3, bit descriptor::id is the only information that a nor- up and give up its CPU usage to other SW applications. mal user application should know about HW tasks. Other When the data frame is completely processed, the target pieces of information such as location and length are only PRR informs the HW task manager by triggering an IRQ used by the HW task manager. IRQ Reco rdy, then the service will be relaunched to start the PCAP bitstream download. 4.2.2 Calling the HW Task Manager One major drawback of the PR technique is its significant Any attempt to dispatch, reconfigure, modify or disable reconfiguration time overhead. To reduce its effect on perfor- HW tasks should be accomplished by the HW task manag- mance, we abort the polling-for-done mechanism. Instead, er. In other words, operations towards HW tasks are isolated the completion of a PCAP transfer is not acknowledged to from other user applications. We employed this mechanism the HW task manager. Once the HW task manager launch- to ensure the security of the FPGA fabric. For user ap- es the PCAP transfer, it gives up the CPU control and wait plications which are cooperating with HW tasks, the only for the next call. A HW task is set to automatically start accessible memory space is the HW task data section, which an operation as soon as reconfiguration is done, thereby the is used for massive SW/HW data exchange. reconfiguration time overhead is overlapped by CPU opera- As described in Section 3, the behavior of HW tasks are tions. SW applications are able to be synchronized with a controlled by writing parameter values to their correspond- HW task state by its general-purpose IRQ. This functional- ing PRR configuration registers, for which the contexts of ity is enabled by the PRR Int status register. For example, parameters are defined by user application and are not in imagine a simple application with an image displayer SW the concern of the HW task manager. All the information task that is using a HW Image filter accelerator. It will required by the HW manager are the ID of HW task and fetch the target image and write the results back to memory the arguments to be transferred to the register group. through AXI4 automatically. Once the image processing is List_prio[prio_top] run_queue Table 4: HW Task Manager API List_prio[2] API Description List_prio[1] XDcfg Initialize(); Instantiate DevCfg AXI4 lite Init(); Instantiate master AXI4-lite prio_top = 1 Task1 Task2 Task3 fpga start(); fpga pause(); Pass signals to whole fpga fabric by writing fpga interrupt done() values to corresponding PPR controller regs. (a) Check current implemented HW tasks’ IDs HW_Manager_Dequeue() HW_Manager_Enqueue() check current ppr(HW id) to determine whether PR is necessary. Check if target PRR is ready for PR, if not, check reco rdy(HW id) List_prio[prio_top] run_queue use sys yield() to quit HW manage. HW prr set mode(HW id,irq en, Set up PRR Int status and register group List_prio[2] Manager arg01,arg02,arg03 ) of specific PRR. prr transferbitfile(HW id) Launch PCAP to transfer target bitstream. List_prio[1] prr register read(off,val) Basic access method to all prr controller prio_top = 2 Task1 Task2 Task3 prr register write(off, val) registers by master AXI4-Lite (b) finished, the HW filter will generate an IRQ to inform the Activated Suspended displayer task that another operation can be executed. Figure 3: Microkernel Scheduling Mechanism. (a) 4.2.3 HW Task Manager API prio top=1; (b) prio top=2 The driver API of DevCfg is supported by the Xilinx SDK tate, HW Manager Dequeue() is called to remove the EC of tool, which deals with the non-secure/secure PCAP transfer. the HW task manager from the run queue, as shown in Fig. Besides of the DevCfg API, several additional functions are 3(a), thus low-priority SW tasks are permitted to execute. developed to facilitate and simplify the HW management. Through this strategy, the PR of an HW accelerator is able In Table 4, the API supporting HW task management is to preempt other SW tasks and a quick response for the HW listed and described. task management is guaranteed. 4.3 SW Tasks Scheduling The scheduling strategy of SW tasks in Mini-NOVA is a 5. USE-CASE IMPLEMENTATION priority-based round-robin mechanism. The scheduler man- In order to test the SW/HW scheduling mechanism on the ages the execution sequence by manipulating ECs. Each EC platform, a use-case application based on a real scenario has obtains its own priority level at its creation, which is change- been proposed. In this scenario, a mobile wireless terminal able afterwards. Within the same priority level, SW tasks is capable of dynamically change its configuration in order to share the CPU through round-robin scheduling. Among dif- obtain the best level of performances according to the chan- ferent priority levels, high-priority tasks will always preempt nel conditions. For example, if the channel is very noisy, the low-priority tasks since the scheduler always selects the high- transmitter will deal with a simple but very efficient QAM est priority EC and dispatches the SW task attached to it. modulation to the detriment of the throughput. As soon Basically, all general SW tasks execute at the same prior- as the channel conditions allow to increase the throughput, ity level (1 by default). However, to fulfill the timing con- the mobile device may reconfigure itself to change its inner straints for specific requests such as real-time tasks and PR hardware modulator and rapidly adapt to the environment. requests, different priority levels are introduced. In this case, specific tasks should be of higher priority so that they can be 5.1 Implementation Description dispatched in time. Since our current system mainly deals In the proposed use-case scenario, the application is di- with HW management, only the HW task manager is being vided into two main software tasks running on the processor discussed here. and two additional hardware tasks running in the FPGA. Fig. 3 presents the scheduling mechanism based on prior- The SW ChannelSensor task performs a channel estima- ity. At each priority level, ECs are organized as a double- tion in order to evaluate the maximum level of performance linked queue, which is indexed by a list prio[] structure. to be obtained in terms of throughput and error rate. The list prio[] is a list of EC pointers indexed by a priority lev- SW HardwareManager is an instance of the HW task man- el. Each list prio[] element points to a certain priority level ager, as described in Section 4. EC queue. The run queue is composed of different priority Concerning the hardware parts, two reconfigurable HW level EC queues, and the prio top signal identifies the high- tasks sets have been considered which respectively deal with est priority level in current run queue. When reschedule() the modulation scheme and the IFFT used in the OFDM is invoked, prio top is used to access the highest-priority- context. The HW Modulation task deals with the nature level EC queue by dispatching list prio[prio top]. Once dis- of modulation to be implemented i.e. the constellation size. patched, the queue will keep executing until another resched- In this work, three constellations sizes have been considered: ule() is invoked. 4-QAM, 16-QAM and 64-QAM. Regarding the second hard- As shown in Fig. 3, the EC of the HW task manager is ware task, HW IFFT, several configurations have also been registered in the microkernel at its creation with a default implemented according to the number of points to consider. priority level 2. Initially, the HW manager is not included in In our application, a range of number of points for I-FFT run queue as Fig. 3(a). When Syscall HW Manager() is ex- (from 256 points to 8192 points) was implemented depend- ecuted, the microkernel will launch HW Manager Enqueue() ing on the channel bandwidth to be considered. All HW to add the HW task manager into the run queue as shown task execute in their corresponding PPR (PRR0 - PRR3). in Fig. 3(b). Then, the reschedule() function is launched to Since HW Modulation and HW IFFT execute in pipeline, update the schedule and dispatch the HW task manager as the reconfiguration of these HW tasks will suspend the entire the highest priority EC by selecting list prio[2]. When the pipeline. To minimize the significant time overhead, we pro- HW task manager finishes its task or enters the pull-up s- pose a multiple-path structure. A block diagram depicts this HW_QAM4 HW_IFFT256 AXI4 HW_QAM16 ... Table 5: SW/HW Tasks’ Attributes HW_QAM64 HW_IFFT8192 Execution Reconfig. Resource Task name Type Time(ms) Time(ms) Usage PRR0 PRR2 SW ChannelSensor SW 3 no no QAM1 IFFT1 SW HW Manager SW 0,0096 no no FIFO1 Q EC Switch SW 0,00232 no no HW QAM (4/16/64) HW 0,09-0.03(1 frame) 0.231 2% FIFO2 PRR1 PRR3 HW IFFT (256-8192) HW 0,006-0,168(1 frame) 2.72 13% I QAM2 IFFT2 resources on chip. For static FPGA circuits, implementing CrossBar multiple I-FFT blocks with different points will cost consid- HW_Modulation HW_IFFT erable FPGA area, while on our platform only 26% FPGA Figure 4: Use-Case Implementation resources (2 I-FFT blocks) are used to hold multiple I-FFT blocks. Thus the chip cost is significantly reduced. task SW_HWManager SW_ChannelSensor Microkernel 6. CONCLUSION t PCAP Transfer t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 In this paper, we have presented a custom ARM-specified PRR0 (HW_QAM4) microkernel on a partially reconfigurable FPGA platform. PRR1 (HW_QAM16) This approach allows to dynamically manage reconfigurable PPR2 (HW_IFFT256) PPR3 (HW_IFFT512) HW accelerators and SW tasks by developing a specific schedul- ing mechanism. Efforts have been made to maximize the SW/HW Execution Reconfiguration Pipeline suspension performance of the FPGA fabric and minimize the overhead t1: syscall_HW_Manager() t2: reschedule() t3: prr_transferbitfile(); syscall_yield() caused by partial reconfiguration. We are currently working t4: reschedule() t5: PCAP_Done t6: Data frame over on the virtualization of guest OS. By implementing differ- Figure 5: Gantt Chart of the Tasks’ Execution ent OSes based on the microkernel, we intend to establish a complete virtualizable embedded system structure in Fig. 4. Both HW Modulation and HW IFFT consist of a pair of identical PRRs. While the current PPR 7. REFERENCES continues working, SW ChannelSensor may alter the HW [1] D. Thomas, J. Coutinho, and W. Luk, “Reconfigurable task by reconfiguring the other PRR, and activating the new computing: Productivity and performance,” in datapath after reconfiguration. Thus, the overhead caused Asilomar Conference on Signals, Systems and by reconfiguration is reduced. Computers, pp. 685–689, 2009. With a 18,800 bits data frame size and 100MHz FPGA [2] G. Heiser, “The role of virtualization in embedded clock frequency, a Gantt chart for the result of proposed s- systems,” in Proceedings of the 1st workshop on cenario on our platform is given in Fig.5. The application Isolation and integration in embedded systems, begins with the SW ChannelSensor task deciding to change pp. 11–16, ACM, 2008. the hardware configuration because the the channel’s condi- [3] “Ug585: Zynq-7000 all programmable soc technical tions are not suitable for the default configuration (a QAM4 reference manual,” Xilinx Inc., March 2013. modulation scheme and a 256 points I-FFT). In this case, the task calls the SW HardwareManager to manage its request [4] U. Steinberg and B. Kauer, “Nova: a microhypervisor of switching I-FFT mode to 512 points (t1 - t2 ). Since PRR3 based secure virtualization architecture,” in Proceedings is idle and ready for reconfiguration, SW HardwareManager of the 5th European conference on Computer systems, launches the PCAP transfer to implement HW IFFT512 to pp. 209–222, 2010. PRR3 while the QAM4-IFFT256 pipeline continues comput- [5] C. Claus, B. Zhang, W. Stechele, L. Braun, M. Hubner, ing (t2 - t5 ). After the completion of PCAP transfer (t5 ), and J. Becker, “A multi-platform controller allowing for the pipeline holds for currently-processed data frame to be maximum dynamic partial reconfiguration throughput,” completely processed (t5 - t6 ) before the HW IFFT512 is ac- in Field Programmable Logic and Applications, tivated at t6 . The same procedure is executed again, when pp. 535–538, IEEE, September 2008. SW ChannelSensor decides to switch from QAM4 modula- [6] D. Göhringer, M. Hübner, E. N. Zeutebouo, and tion to QAM16 (t7 - t12 ). Some attributes of SW/HW tasks J. Becker, “Operating system for runtime reconfigurable are listed in Table 5. multiprocessor systems,” International Journal of Reconfigurable Computing, vol. 2011, January 2011. 5.2 Discussion [7] J. C. Prevotet, A. Benkhelifa, and e. a. B. Granado, “A As shown in the Gantt chart, the major overhead of re- framework for the exploration of rtos dedicated to the configuration is fully circumvented by both SW and HW management of hardware reconfigurable resources,” in tasks running in parallel. For data processing, the only International Conference on Reconfigurable Computing overhead caused by the HW task switch is the delay re- and FPGAs, pp. 61–66, IEEE, 2008. quired to process a complete data frame(worst case 0.168 [8] K. D. Pham, A. K. Jain, J. Cui, and et al, “Microkernel ms, in case of 8096 points I-FFT). Due to the simplified hypervisor for a hybrid arm-fpga platform,” in 24th kernel and scheduling mechanism, a quick response to PR International Conference on Application-Specific is achieved (0.0119 ms). We should note that the tremen- Systems, Architectures and Processors, pp. 219–226, dous reconfiguration overhead of I-FFT tasks result from the IEEE, 2013. massive computing-intensive structure of I-FFT blocks. Im- [9] K. Vipin and S. A. Fahmy, “A high speed open source plemented by Xilinx Planahead synthesis tool, it consumes controller for fpga partial reconfiguration,” in FPT, 5600 LUTs and 1600 SLICEs, which takes up to 13% FPGA pp. 61–66, IEEE, 2012.