{'fraction_non_alphanumeric': 0.019604488846311757, 'fraction_numerical': 0.009066648419323937, 'mean_word_length': 4.716604733033444, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
PCIE IP Validation Process Across Process Corner, Voltage and Temperature Conditions

IP validation has become more challenging for FPGA device as it supports high operating speed. PCIe is an IP used for high-speed data transfer. The link training and Initialization takes place at physical layer to initialize the link width and link data rate. The physical layer is getting more complex when it supports higher speed. The stability of link training is improved by optimizing the soft logic design in application layer. Two protocol tests usually validated in industry are link up testing and link & higher layer testing. Debugging tools supported by Quartus are fully utilized to detect any failure during link training. The characterization of link performance covers process corners, voltage and temperature conditions are hard to analyse. The H0 statement shows a significant difference for passing and failing case. The p-value is greater than 0.05 proved H0 statement is accepted. The difference on passing and failing percentage is insignificantly impacting overall link performance of PCIe. It concludes that the bug is random and not caused by any defects on the silicon layout of FPGA device. Thus, IP validation shows the robustness of the device and able to comply with base specification of PCIe.


Introduction
Electronic fabric design gets more complex when the size of electronic technology gets smaller making it harder to verify the specifications of analog and digital signal. Intellectual Property (IP) validation used to confirm the function of the IP and to regulate the robustness of the device. Post-Silicon IP verification for microprocessors summarizes key areas like system validation (SV), compatibility validation (CV), and electrical validation (EV) [1]. The selection made for Product Release Qualification (PRQ) turning point are dependent on results obtained from the verification. The PRQ turning point is to make sure the device is functioning and prepared to be out in market. In the rise of advanced technology in this day and age especially in communication fields, sophisticated military equipment and autonomous driving in automotive industry have push the demands of advanced semiconductor technology that competent of switching and carry out delicate function that can be done by a tiny yet powerful Intel FPGA chip. These demands had steer data transmission consortium a crucial part in exclusive technology specially to match with industrial standards. Peripheral Component Interconnect Express (PCIe), Universal Serial Bus (USB), Synchronous Optical Network (SONET) and Universal Asynchronous Receiver Transmitter (UART) are part of the industrial level in data transmission interface protocols. PCIe IP is a high-speed serial interface usually utilized as an interface for flash storage in industrial implementation like data center, cloud computing, sever and Ultrabook. For example, SATA-based interface that can be develop in Solid State Drive (SSD) in a lot of computer and electronic devices has restricted capacity of data transfer. PCIe technology is most favored protocol utilized in communication application since it offers higher speed and greater output. The outcome of PCIe link training is verified across numerous process corner, voltage, and temperature conditions. Process corner is a skew unit disparity in a silicon fabrication. Those disparity influence the outcome of the device relying on the operating voltage and temperature. Process corners make up for three dissimilar skew units, which are typical device (TT), slow device (SS) and fast device (FF). The timing calibration of Intel FPGA utilizing Nios technology help make up for changes in process corners, voltage, or temperature. The breakthrough calibration algorithms secure highest bandwidth and strong timing margin over all operating circumstances.
The serial protocol like PCIe has developed throughout the years to supply very high operating speeds and better outcome. This evolution has caused their physical layer protocol such as physical medium attachment (PMA) and physical coding sublayer (PCS) architecture became more complexed. The link training and link initialization is a very important processes at physical layer. This process fixed a lot of major function like link width negotiation and link data rate negotiation. All these functions are achieved by Link Training and Status State Machine (LTSSM). The LTSSM adjusts and instructs the PCIe link for dependable data transfer. L0 state is the normal operational condition where data and control packets are transitioned. The transition state from LTSSM to get to L0 state is tough to accomplish because of complexity of physical layer formation when the operating speed increase. Plus, the implementation layer of PCIe protocol stack was patterned using soft logic circuit. The design gets more complex and tough to follow the timing margin of the circuit to obey with PCIe protocol. As a result, more complicated routing path in the FPGA used when more protocol testing is demanded. For IP validation, protocol testing is tested at 1000 iterations with different process corners, voltage, and temperature settings. The link performance differences at vary link speed and lane width throughout the process, voltage, and temperature (PVT). The differentiation is tough to be examined and summarized since it does not convey any big remarkable different for each test. Thus, an precise data analysis methodology is required to understand the difficulty, and these will be the focus of this research.

Importance of post-silicon IP validation
Post-silicon IP validation is a very important part of device characterization mainly in FPGA devices. The function is to find out and resolve bugs in complicated integrated circuit that impossible to be caught during pre-silicon validation [2]. The complication in layout of integrated circuits rise the bugs for fabricated silicon. The restricted time-to-market period worsen the circumstances, as the quality assurance of the device is ignored [3]. The observation and identification are needed to put out debug operation prior to high volume production. The reliance on pre-silicon validation collected data is not well founded as the coverage is lesser. During pre-silicon validation, test cases will be running on virtual environment with advanced software tools are utilized [4]. Simulation and emulation are one of the appliances utilized in pre-silicon validation still, it limits the precision of the device. Worst test case coverage on the silicon will influence the probability of appearing the bugs thus a well-constructed data analysis methodology is crucial to observe the performance of the new silicon. There are few levels of debugging in post-silicon IP validation, which incorporate the software and hardware preparation. These propositions will help to run task in timely manner and easier to take out the failures. During bugs detection phase, the right stimuli must be used, registering all the critical bugs that are hard to resolve. This kind of way is susceptible to be able to cut up the bugs into smaller regions thus easy to detect the circumstances of the bugs. Once the bugs are recognised, software patching, silicon re-spin and editing the baseline design [4] are the only ways left to cremate the bugs. There are few kinds of bugs that can be categorized as well as electrical bugs and logic bugs [5]. Electrical bugs are closely connected to signal integrity of the board such as crosstalk between traces and the length of the traces induced noise. The power supply shortage and thermal effects in experimental setup also create electrical bugs. The interfaces of digital and analogue circuitries considered to be very hard component to dealt with [6]. Process corner variations which cover slow and fast corner devices will differ the electrical bugs in post-  PCIe is a high performance, general aim I/O inter-connect technology interpreted for a huge variety of future computing and communication platforms. Data transaction is summarized in packets. The data produced in transaction layer is transmitted throughout data link layer. The physical layer help transfer the data which is encrypt following type of data transfer by transmitter port, through physical link cable to the end of the device. In PCIe IP validation, two components are needed to have PCIe supported interface which is the root port and the endpoint. Intel FPGA device families assist PCIe IP interface, that can be depicted as the two components. The PCIe design follow the industry level that is introduced in Base Specification of PCIe Revision 3.0. In any device helped by PCIe protocol, link training process is the fundamental element of PCIe. Besides, the packet created in transaction layer and any transaction of packet through layer by layer, there is a mechanism must begin as it will decide the stability of the link. The link training process can forecast PCIe link stability when it stables at its anticipated outcome as well as speed rate, lane width and link up status. Ordered sets are utilized for link training, it will be transferred as a bunch of characters on all the lanes. Ordered sets are packets that originate and end in the physical layer. To perfectly go over lane initialization and build an anticipated link to begin a link training process, ordered sets are transferred as a mark for the lane readiness. Before a packet of data is transmitted on the link, the link training process will kick starts the PCIe engine. Link training process is taking place after FPGA being organized. PCIe protocol is a handshaking protocol of two devices, where both devices must act as a pillar to PCIe protocol. The endpoint and the root port need to be configured before a link training starts and arrives at L0 state. The normal operating PCIe only occurs when it reaches L0 state. The new silicon will be the endpoint while the root port will be the matured silicon. This is caused by matured product that has its outcome boosted over time and decrease debugging steps that is very time wasting. The handshaking process to start PCIe link training requires two basic components, which are a transmitter, controlling the status bit throughout link up process, and a receiver that control the error status bits. A successful status of link up process can be decide based on PCIe protocol specifications provided by PCI-SIG. In high-level overview, link training process is happening between data link layer and physical layer which also known as PHYMAC layer. DETECT, POLLING, CONFIGURATION and L0 are four states component present during link training process. 4 Each state initiates their task alone and competent of realize the failure origins during link training which in turn relieved the debugging attempt. The design is produced prior to the first fabricated silicon ready to be validate. The preparation comprised of software readiness including the automation script to run the whole test automatically. The task gets easier when it deals with software and hardware in parallel.

Debugging
During protocol test, many issues are found related to FPGA configuration, link up training problem, data reliability and so on. The protocol tests for this research include the link up testing and link & higher layer testing. There are several tools provided by Quartus software including SignalTap Logic Analyzer and system console. The tools used to illustrate the link training issue by observing the accuracy of the data in hardware level. In PCIe, the physical layer initiated the link training to configure the link before the data can be transmit. Thus, debugging method in PCIe link training is giving a guideline to solve any issue seen on device. After FPGA is configured, the system is reset to link up correctly. The link training is observed and controlled using the tools explained, SignalTap and System Console. If the link training is not successful, there are few paths to identify the failures. The link up signal indicates the link is up or not stable. Firstly, the LTSSM state transition is checked. The transition state from detect to polling to configuration to L0 state determined the sub-state requirement met the base specification of PCIe revision 3.0 [7]. The design is implementing a counter to count on the LTSSM state transition time. The requirement to transition to another sub-state is quantified in time manner. The timeout is set on each state. If the timeout happen before the link is trying to pick up, the link training is not successful. The whole process is repeated until stable link is achieved. Second method of debugging link training issue is by using PCIe Protocol Analyzer. It also provides the visibility of hardware and help determine the solution such as LeCroy Analyzer. The features include LTSSM flow graph and advanced triggering are easier to isolate and analyze the error pattern. It visualized the entire trace activities and help in understanding the overall PCIe handshaking concept. There are three layers in PCIe protocol which are transaction layer, data link layer and physical layer. The physical layer has the physical interface to another block. The PIPE interface signals are very helpful to identify the failure causes during link training issue. Signal stated in Intel FPGA Avalon-ST Interface Solution User Guide which useful for debugging is txdetectrx0. The signal tells the physical layer to start receiving detection operation on far end device. At detect state, the signal is asserted to recognize the far end device. Thus, the debugging is drived from there. Once the link is up and stable at L0 state, data packets are ready to be transmit. Thus, second level of validation begins. The configuration space register is providing the information of the link status by accessing the specific register through System Console.

Data analysis method
The collected data during link up testing and link & higher layer testing are analysed. Decision is made based on the evidence of analysed data. The link training performance across various process corners, voltage and temperature conditions need to be analysed using hypothesis testing. It is because the failure pattern is obviously seen using this method of decision-making. Besides the supported tools to analyse the performance of the link training, hypothesis testing gives additional confidence level towards the behaviour of the link training. The analysed data gives an extra evidence for the stability of PCIe link performance. The hypothesis testing is proven using the equation to measure the significant different for all the test cases. The passing criteria ensures the evidence is enough before a conclusion is made on the overall performance of PCIe across process corner, voltage and temperature conditions. The parameter identification is to identify the characterization that need to be tested. In analysing stability of PCIe across different speed and lane width, the variation of process corner, voltage and temperature are set as the parameter. In 3.1 shows the null hypothesis which also known as H-naught (H0) is a true assumption statement that need to be carried out. The alternative hypothesis illustrated in 3.2 is called as H-one (H1) is finding the prove of an alternative statement.
The significant level is chosen based on types of data characterization. For quality assurance work, 0.05 is set as the level of significance. The variance, α is assumed to be equal for the link stability analysis. The critical region is based on the level of significant which reject or non-reject the null hypothesis. In stress test data, the critical region is based on the performance across process corner, voltage and temperature. The appropriate test static is selected based on the computed value. If the value is in the rejected region, it rejects the null hypothesis and non-reject the alternative hypothesis and vice versa. The decision is made with enough evidence to reject or not reject H0 statement. The stress test data is analysed with respect to its link speed and lane width at low and high voltage and temperature. A single factor of analysis of variance (ANOVA) is used to verify the null hypothesis as some populations are all equal. For stress test collected data, there is no difference across process corner, voltage and temperature through an eyeball-checking. The hypothesis testing helps to see the significant difference even if the results looked all equal.

Results and discussions
The link up testing is captured after reset applied at endpoint device. The SignalTap Logic Analyzer is used to tap the signal status during link up testing. System console is used for handling the command to control reset signal. The Gen1 has a very straight forward transition from detect state until it reaches L0 state without going through equalization phase. There is an additional transitional state during link up to Gen3 which enters the equalization phase1, phase2 and phase3 before reaches L0 state. The electrical validation finalized all the settings supported by PCIe protocol especially for higher data rate. After reset is applied for 1000 iterations, the link achieved L0 state with correct configuration.  Table 1 shows the Gen1 link up to L0 state with 2.5GT/s. The Gen2 at 5.0 GT/s is link up successfully after reset is applied for 1000 times. The endpoint device shows a highest confident on the link training stability as it achieved highest PCIe supported data transfer rate at 8.0 GT/s at nominal voltage and temperature. Based on the results, analysis of variances is made to gain more confident on the stability of link training performance during enabling. As the overall overview on link up testing performance, there is no significant different for three different PCIe supported data rate, gen1, gen2 and gen3. This is expected for link up testing as it not stress testing the device yet. Thus, the H0 statement in (3) is non-rejected which gives strong evidence on the stability of link training is achieved during protocol test.
The stability of PCIe link training for link up testing at nominal voltage and temperature are not giving much impact on the digital circuit inside FPGA as it is operating at its typical process. The link & higher layer testing changed the speed of data transfer as per requested. Based on PCIe specification Revision 3.0, Gen3 capable supported to change speed to lower speed rate including Gen2 and Gen1 regardless of any lane width as long the root port device is supporting the highest speed and lane width. The link goes to recovery state to speed change to another speed. These transitions are seeing across multiple iteration of speed changes at any speed changes combinations. The robustness of link training is tested through link & higher layer testing as it has an additional LTSSM sub-state to go through compared to link up testing. The link & higher layer testing result shows in table 4.2 described the performance of link training at multiple speed changes across three combinations of speed. The protocol test result is collected at nominal voltage and nominal temperature to see the performance of link training is achieving L0 state for every speed change. The differences are observed at gen3x4 and gen3x8. Gen2 bidirectional speed changes to Gen3 and Gen3 bidirectional speed changes to Gen1 are expected to have error as it handles extra mechanism compared to Gen2 bidirectional sped changes to Gen1. Any speed changes to Gen3 need to go three phases of equalization. The adaptation engine is running during speed changes to Gen3 as it has higher data transfer rate, 8.0 GT/s.
From Minitab output, p-value is equal to 0.95 which is more than the significance level set for this research. The result proved that H0 statement in (5) is not rejected and indicated that there is no significant different between all lane width for the combinations of speed changes. Thus, the data point is strong enough to proceed for characterizing link training of PCIe across process corner. Slow-Slow (SS) process device is tested to further characterize the performance of the link training of PCIe at slower grade device. The performance of digital circuit may vary across process corners as the transistor switching speed is slower from one state to another state. The link training has a lot of switching mechanism to achieve L0 state. There are a lot of interference for the link up testing and link & higher layer testing. The failures are easily identified especially at a worse case condition.  Table 3 shows the link up testing at Gen1 achieved L0 state across all condition for 1000 system reset iteration. There is no different for x1, x2, x4 and x8 lane configurations as at Gen1, 2.5 GT/s the link up training is having less reset mechanism and the transition state of LTSSM to achieve L0 state is lesser. Thus, process corners are not affected on Gen1 link up testing. The link training hits a failure at Gen2x8 at low voltage and low temperature as shown in Table 4. The worst case for link up testing for Gen2 is expected as Gen2 speed require additional LTSSM sub state transition. The failure is caused by the link retrained and not achieved L0 state for few iterations of system reset. H0 : µLL = µHL = µLH = µHH (7) H1 : µLL ≠ µHL ≠ µLH ≠ µHH (8) Figure 2 shows the p-value=0.426 is greater than α=0.05 then evidence is not enough to reject H_o statement in 4.5. Thus, the failure happened at Gen2x8 is valid and gives a confidence on the performance is dependent on process corner device. Gen3x4 has two point of failures at low temperature but the occurrence rate is lower than gen3x8 failures. The failure at Gen3x8 is expected as it increased the speed of data transfer, a lot of switching mechanism need to fulfill. The Gen3x4 failure are caused by the link retrained for few cycles and achieved L0 state. The failures are not a critical bug as the percentage is marginally passed.  Figure 3. Snapshot from Minitab for one-way ANOVA for Gen3 link up testing. Figure 3 shows p-value=0.346 is greater than α=0.05. Thus, there is not enough evidence to reject H0. The H0 statement in 7 is valid to summarize the failures seen are acceptable and within the good range. The link & higher layer testing is tested and analyzed with respect to speed change combinations. Link training for each speed has different transition of LTSSM sub-state. As the speed changes are initiated from the root port device, the link speed is expected to support requested speed. The endpoint device is stressed at low and high voltage and temperature to observe the dependencies on the link training stability. Table 6. The bidirectional Gen2 to Gen1 speed changes across voltage and temperature variations. Table 6 shows the raw data for speed changes test at SS unit for 1000 iterations at voltage and temperature variations. Gen2 to Gen1 speed changes at Gen3 capable is having a failure rate less than 80 %. The failures are not significant to claim as the silicon bugs as the changes of Gen1 and Gen2 required less mechanisms involved. The failures are caused by signal integrity issue on experimental setup. After some time of running under stress condition, it started to behave unexpectedly including the current leakage is too high.     It helps to see the failure pattern as the failure rate is very high. The discrepancy of failure rate needs to be debugged. The link training sequence are observed and the timeout happened at few LTSSM sub-state has caused infinite retrain on the link. The adaptation engine for Gen3 is slightly off phase for SS unit when tested under stress condition. Thus, the electrical validations are required to optimize the link performance to match the digital logic supported by PCIe. Figure 5. Snapshot from Minitab for one-way ANOVA for Gen2 →Gen3 Figure 5 shows p-value = 0.424 is greater than α=0.05 then H0 statement in 7 is not rejected. The failure rate is still in acceptable range. The assumption on stability of PCIe link training performance is valid and minor bug can be tuned through link optimization.   Figure 6. Snapshot from Minitab for one-way ANOVA for Gen3 →Gen1 Figure 6 shows the p-value=0.695 is greater than α=0.05 then evidence is still not enough to reject H_o statement. Thus, data point is still valid and not gating any progress on data collection. Fast-Fast (FF) process corner is tested to further characterize the performance of the link training of PCIe at faster grade device. Theoretically, the performance of digital circuit may vary across process corners as the transistor switching speed is faster from one state to another state. Fast skew device is expected to meet the worst condition. The L0 state is expected to be achieved for link up testing and link & higher layer testing. Table 9. Gen1 link up testing at voltage and temperature variation across all lane configuration. Gen2 VT Condition Table 11 shows the performance of Gen3 link up testing at different voltages and temperatures. Gen3x1 has the lowest passing percentage. The failure is caused by the link is stuck at detect state which indicate that the root port device is dead. The endpoint device cannot proceed to the next LTSSM substate as it did not receive the other end signal. The failure is happened at random lane width regardless of any voltage and temperature condition. Thus, the stability of PCIe of endpoint device must ensure the link stability at root port device.  Figure 7 shows the p-value = 0.426 is greater than α=0.05. The H0 statement in Eq. 7 is not rejected as no evidence to reject the performance of Gen3 link stability. Thus, the stability of PCIe link training for link up testing is validated and achieved expected link speed and lane width. The link & higher layer testing of FF unit is tested and analyzed with respect to speed change combinations. The link training for multiple iterations of speed changes at faster skew device are expected to have a bug especially during Gen3 speed change. Theoretically, the higher the data transfer rate, the signal easily drop over transmission line. The electrical setting for equalization need to be optimized to get a stable link training at Gen3.  Table 12 shows the bidirectional speed changes of Gen2 and Gen1 are passing for across all configurations. The faster corner is not causing any failure at Gen2 and Gen1 speed as the mechanism is not complicated as at Gen3.  Table 13 shows the result of getting link failures during speed changes between Gen2 and Gen3. The failures are caused by the combination of link training issue. As the speed getting faster, the faster corner easily hit the failure of link training. Root causing the bugs are easier using the faster corner as it easier to reproduce the error for debugging.    Table 14 shows the raw data of speed changes is hitting 0% failure at low voltage and low temperature. The worst condition for Gen3 to Gen1 speed changes need further analysis whether it is in the acceptable margin. The failure is not lane width dependent, it happened at random configurations. The failures are caused by root port device having a link up problem as it cannot receive any signal from endpoint transmitter. As the IP validation is only concerned on endpoint device performance, the failure condition can be neglected. Figure 9. Snapshot from Minitab for one-way ANOVA for Gen3 →Gen1. Figure 9 shows the p-value=0.461 is greater than α=0.05 then H0 statement in Eq. 7 is not rejected for the speed changes between Gen3 and Gen1. The bugs are still acceptable and its likely a device dependent kind of performance.

Conclusion
In this research, the stability of PCIe link training of endpoint device across process corner, voltage and temperature is presented. The debug method is applied for the link training issue found in validating 20nm technology of IntelFPGA. Hypothesis testing is applied on the collected data to have extra evidence on the link performance. The stability of PCIe protocol link training is improved by applying an optimized design of soft logic application layer. The design minimized the routed path in FPGA device and abled to meet the timing margin. The resource utilization of the core in FPGA device used to design for protocol test is reduced by half. The expected operating speed and link width can achieve L0 state without having major link training issue. The analysis method used in this research clearly illustrated the variation of passing percentage across voltage and temperature at random link speed and lane width. The hypothesis testing shows a significant different as some of the results are all equal. The significance of level for this research is set to 95% confidence level. The p-value from Minitab for all test cases are at 0.6-0.3 range which higher than α=0.05, thus the evidence to reject H0 statement is not valid. The analysis proves that the performance of PCIe link training across PVT depends on the device characteristic. The objectives of this research are successfully achieved and analysed using a reliable method.