Randomized Benchmarking

Randomized benchmarking (RB) is a popular protocol for characterizing the error rate of quantum processors. An RB experiment consists of the generation of random Clifford circuits on the given qubits such that the unitary computed by the circuits is the identity. After running the circuits, the number of shots resulting in an error (i.e. an output different from the ground state) are counted, and from this data one can infer error estimates for the quantum device, by calculating the Error Per Clifford. See the Qiskit Textbook for an explanation on the RB method, which is based on Refs. [1] [2].

Note

This tutorial requires the qiskit-aer and qiskit-ibm-runtime packages to run simulations. You can install them with python -m pip install qiskit-aer qiskit-ibm-runtime.

import numpy as np
from qiskit_experiments.library import StandardRB, InterleavedRB
from qiskit_experiments.framework import ParallelExperiment, BatchExperiment
import qiskit.circuit.library as circuits

# For simulation
from qiskit_aer import AerSimulator
from qiskit_ibm_runtime.fake_provider import FakePerth

backend = AerSimulator.from_backend(FakePerth())

Standard RB experiment

To run the RB experiment we need to provide the following RB parameters, in order to generate the RB circuits and run them on a backend:

• qubits: The number of qubits or list of physical qubits for the experiment

• lengths: A list of RB sequences lengths

• num_samples: Number of samples to generate for each sequence length

• seed: Seed or generator object for random number generation. If None then default_rng will be used

• full_sampling: If True all Cliffords are independently sampled for all lengths. If False for sample of lengths longer sequences are constructed by appending additional Clifford samples to shorter sequences. The default is False

The analysis results of the RB Experiment may include:

• EPC: The estimated Error Per Clifford

• alpha: The depolarizing parameter. The fitting function is \(a \cdot \alpha^m + b\), where \(m\) is the Clifford length

• EPG: The Error Per Gate calculated from the EPC, only for 1-qubit or 2-qubit quantum gates (see [3])

Running a 1-qubit RB experiment

The standard RB experiment will provide you gate errors for every basis gate constituting an averaged Clifford gate. Note that you can only obtain a single EPC value \(\cal E\) from a single RB experiment. As such, computing the error values for multiple gates \(\{g_i\}\) requires some assumption of contribution of each gate to the total depolarizing error. This is provided by the gate_error_ratio analysis option.

Provided that we have \(n_i\) gates with independent error \(e_i\) per Clifford, the total EPC is estimated by the composition of error from every basis gate,

\[{\cal E} = 1 - \prod_{i} (1 - e_i)^{n_i} \sim \sum_{i} n_i e_i + O(e^2),\]

where \(e_i \ll 1\) and the higher order terms can be ignored.

We cannot distinguish \(e_i\) with a single EPC value \(\cal E\) as explained, however by defining an error ratio \(r_i\) with respect to some standard value \(e_0\), we can compute EPG \(e_i\) for each basis gate.

\[{\cal E} \sim e_0 \sum_{i} n_i r_i\]

The EPG of the \(i\) th basis gate will be

\[e_i \sim r_i e_0 = \dfrac{r_i{\cal E}}{\sum_{i} n_i r_i}.\]

Because EPGs are computed based on this simple assumption, this is not necessarily representing the true gate error on the hardware. If you have multiple kinds of basis gates with unclear error ratio \(r_i\), interleaved RB experiment will always give you accurate error value \(e_i\).

lengths = np.arange(1, 800, 200)
num_samples = 10
seed = 1010
qubits = [0]

# Run an RB experiment on qubit 0
exp1 = StandardRB(qubits, lengths, num_samples=num_samples, seed=seed)
expdata1 = exp1.run(backend).block_for_results()
results1 = expdata1.analysis_results()

# View result data
print("Gate error ratio: %s" % expdata1.experiment.analysis.options.gate_error_ratio)
display(expdata1.figure(0))
for result in results1:
    print(result)

Gate error ratio: {'x': 1.0, 'rz': 0.0, 'sx': 1.0}

AnalysisResult
- name: @Parameters_RBAnalysis
- value: CurveFitResult:
 - fitting method: least_squares
 - number of sub-models: 1
  * F_rb_decay(x) = a * alpha ** x + b
 - success: True
 - number of function evals: 143
 - degree of freedom: 1
 - chi-square: 0.5887493022524771
 - reduced chi-square: 0.5887493022524771
 - Akaike info crit.: -1.6641967196235967
 - Bayesian info crit.: -3.5053136362639252
 - init params:
  * a = 0.47151336778158165
  * alpha = 0.999583692373827
  * b = 0.5
 - fit params:
  * a = 0.9715936998628126 ± 1.9490519513784161
  * alpha = 0.9998130544423283 ± 0.0003945368309957095
  * b = 3.0410068048442503e-15 ± 1.9495115964881315
 - correlations:
  * (a, b) = -0.9999996846083852
  * (alpha, b) = -0.9999029224307697
  * (a, alpha) = 0.9998957008123674
- quality: good
- extra: <2 items>
- device_components: ['Q0']
- verified: False
AnalysisResult
- name: alpha
- value: 0.9998+/-0.0004
- χ²: 0.5887493022524771
- quality: good
- extra: <2 items>
- device_components: ['Q0']
- verified: False
AnalysisResult
- name: EPC
- value: 0.00009+/-0.00020
- χ²: 0.5887493022524771
- quality: good
- extra: <2 items>
- device_components: ['Q0']
- verified: False
AnalysisResult
- name: EPG_x
- value: 0.00012+/-0.00025
- χ²: 0.5887493022524771
- quality: good
- extra: <2 items>
- device_components: ['Q0']
- verified: False
AnalysisResult
- name: EPG_rz
- value: 0.0+/-0
- χ²: 0.5887493022524771
- quality: good
- extra: <2 items>
- device_components: ['Q0']
- verified: False
AnalysisResult
- name: EPG_sx
- value: 0.00012+/-0.00025
- χ²: 0.5887493022524771
- quality: good
- extra: <2 items>
- device_components: ['Q0']
- verified: False

Running a 2-qubit RB experiment

In the same way we can compute EPC for two-qubit RB experiment. However, the EPC value obtained by the experiment indicates a depolarization which is a composition of underlying error channels for 2Q gates and 1Q gates in each qubit. Usually 1Q gate contribution is small enough to ignore, but in case this contribution is significant comparing to the 2Q gate error, we can decompose the contribution of 1Q gates [3].

\[\alpha_{2Q,C} = \frac{1}{5} \left( \alpha_0^{N_1/2} + \alpha_1^{N_1/2} + 3 \alpha_0^{N_1/2} \alpha_1^{N_1/2} \right) \alpha_{01}^{N_2},\]

where \(\alpha_i\) is the single qubit depolarizing parameter of channel \(i\), and \(\alpha_{01}\) is the two qubit depolarizing parameter of interest. \(N_1\) and \(N_2\) are total count of single and two qubit gates, respectively.

Note that the single qubit gate sequence in the channel \(i\) may consist of multiple kinds of basis gates \(\{g_{ij}\}\) with different EPG \(e_{ij}\). Therefore the \(\alpha_i^{N_1/2}\) should be computed from EPGs, rather than directly using the \(\alpha_i\), which is usually a composition of depolarizing maps of every single qubit gate. As such, EPGs should be measured in the separate single-qubit RBs in advance.

\[\alpha_i^{N_1/2} = \alpha_{i0}^{n_{i0}} \cdot \alpha_{i1}^{n_{i1}} \cdot ...,\]

where \(\alpha_{ij}^{n_{ij}}\) indicates a depolarization due to a particular basis gate \(j\) in the channel \(i\). Here we assume EPG \(e_{ij}\) corresponds to the depolarizing probability of the map of \(g_{ij}\), and thus we can express \(\alpha_{ij}\) with EPG.

\[e_{ij} = \frac{2^n - 1}{2^n} (1 - \alpha_{ij}) = \frac{1 - \alpha_{ij}}{2},\]

for the single qubit channel \(n=1\). Accordingly,

\[\alpha_i^{N_1/2} = \prod_{j} (1 - 2 e_{ij})^{n_{ij}},\]

as a composition of depolarization from every primitive gates per qubit. This correction will give you two EPC values as a result of the two-qubit RB experiment. The corrected EPC must be closer to the outcome of interleaved RB. The EPGs of two-qubit RB are analyzed with the corrected EPC if available.

lengths_2_qubit = np.arange(1, 200, 30)
lengths_1_qubit = np.arange(1, 800, 200)
num_samples = 10
seed = 1010
qubits = (1, 2)

# Run a 1-qubit RB experiment on qubits 1, 2 to determine the error-per-gate of 1-qubit gates
single_exps = BatchExperiment(
    [
        StandardRB((qubit,), lengths_1_qubit, num_samples=num_samples, seed=seed)
        for qubit in qubits
    ]
)
expdata_1q = single_exps.run(backend).block_for_results()

# Run an RB experiment on qubits 1, 2
exp_2q = StandardRB(qubits, lengths_2_qubit, num_samples=num_samples, seed=seed)

# Use the EPG data of the 1-qubit runs to ensure correct 2-qubit EPG computation
exp_2q.analysis.set_options(epg_1_qubit=expdata_1q.analysis_results())

# Run the 2-qubit experiment
expdata_2q = exp_2q.run(backend).block_for_results()

# View result data
print("Gate error ratio: %s" % expdata_2q.experiment.analysis.options.gate_error_ratio)
display(expdata_2q.figure(0))
for result in expdata_2q.analysis_results():
    print(result)

Gate error ratio: {'cx': 1.0}

AnalysisResult
- name: @Parameters_RBAnalysis
- value: CurveFitResult:
 - fitting method: least_squares
 - number of sub-models: 1
  * F_rb_decay(x) = a * alpha ** x + b
 - success: True
 - number of function evals: 16
 - degree of freedom: 4
 - chi-square: 0.014768425008341263
 - reduced chi-square: 0.003692106252085316
 - Akaike info crit.: -37.128217801804595
 - Bayesian info crit.: -37.29048735463866
 - init params:
  * a = 0.6915205739921371
  * alpha = 0.9760745162786324
  * b = 0.25
 - fit params:
  * a = 0.6912379592581522 ± 0.0057273836192850095
  * alpha = 0.9758102232867637 ± 0.0006553721847698429
  * b = 0.2504749287678986 ± 0.003451267746133811
 - correlations:
  * (alpha, b) = -0.7649100616097434
  * (a, b) = -0.4753017927930775
  * (a, alpha) = 0.18238340845472192
- quality: good
- extra: <2 items>
- device_components: ['Q1', 'Q2']
- verified: False
AnalysisResult
- name: alpha
- value: 0.9758+/-0.0007
- χ²: 0.003692106252085316
- quality: good
- extra: <2 items>
- device_components: ['Q1', 'Q2']
- verified: False
AnalysisResult
- name: EPC
- value: 0.0181+/-0.0005
- χ²: 0.003692106252085316
- quality: good
- extra: <2 items>
- device_components: ['Q1', 'Q2']
- verified: False
AnalysisResult
- name: EPC_corrected
- value: 0.0174+/-0.0007
- χ²: 0.003692106252085316
- quality: good
- extra: <2 items>
- device_components: ['Q1', 'Q2']
- verified: False
AnalysisResult
- name: EPG_cx
- value: 0.0124+/-0.0005
- χ²: 0.003692106252085316
- quality: good
- extra: <2 items>
- device_components: ['Q1', 'Q2']
- verified: False

Note that EPC_corrected value is smaller than one of raw EPC, which indicates contribution of depolarization from single-qubit error channels. If you don’t need EPG value, you can skip its computation by exp_2q.analysis.set_options(gate_error_ratio=False).

Displaying the RB circuits

The default RB circuit output shows Clifford blocks:

# Run an RB experiment on qubit 0
exp = StandardRB(physical_qubits=(0,), lengths=[2], num_samples=1, seed=seed)
c = exp.circuits()[0]
c.draw(output="mpl", style="iqp")

You can decompose the circuit into underlying gates:

c.decompose().draw(output="mpl", style="iqp")

And see the transpiled circuit using the basis gate set of the backend:

from qiskit import transpile
transpile(c, backend, **vars(exp.transpile_options)).draw(output="mpl", style="iqp", idle_wires=False)

Note

In 0.5.0, the default value of optimization_level in transpile_options changed from 0 to 1 for RB experiments. Transpiled circuits may have less number of gates after the change.

Interleaved RB experiment

The interleaved RB experiment is used to estimate the gate error of the interleaved gate (see [4]). In addition to the usual RB parameters, we also need to provide:

• interleaved_element: the element to interleave, given either as a group element or as an instruction/circuit

The analysis results of the RB Experiment includes the following:

• EPC: The estimated error of the interleaved gate

• alpha and alpha_c: The depolarizing parameters of the original and interleaved RB sequences respectively

Extra analysis results include

• EPC_systematic_err: The systematic error of the interleaved gate error [4]

• EPC_systematic_bounds: The systematic error bounds of the interleaved gate error [4]

Let’s run an interleaved RB experiment on two qubits:

lengths = np.arange(1, 200, 30)
num_samples = 10
seed = 1010
qubits = (1, 2)

# The interleaved gate is the CX gate
int_exp2 = InterleavedRB(
    circuits.CXGate(), qubits, lengths, num_samples=num_samples, seed=seed)

int_expdata2 = int_exp2.run(backend).block_for_results()
int_results2 = int_expdata2.analysis_results()

# View result data
display(int_expdata2.figure(0))
for result in int_results2:
    print(result)

AnalysisResult
- name: @Parameters_InterleavedRBAnalysis
- value: CurveFitResult:
 - fitting method: least_squares
 - number of sub-models: 2
  * F_standard(x) = a * alpha ** x + b
  * F_interleaved(x) = a * (alpha_c * alpha) ** x + b
 - success: True
 - number of function evals: 25
 - degree of freedom: 10
 - chi-square: 7.111268423386916
 - reduced chi-square: 0.7111268423386916
 - Akaike info crit.: -1.483273825167938
 - Bayesian info crit.: 1.0729554932930956
 - init params:
  * a = 0.6859232480592643
  * alpha = 0.9795725844757435
  * b = 0.25
  * alpha_c = 0.9897501197945084
 - fit params:
  * a = 0.6871274818583644 ± 0.004184053015591246
  * alpha = 0.9752076047168833 ± 0.0005607844499474046
  * b = 0.25265746689130797 ± 0.0020896943708809115
  * alpha_c = 0.9869926839039223 ± 0.0008943357167801021
 - correlations:
  * (alpha, b) = -0.6390432076041747
  * (a, b) = -0.38095143410205834
  * (alpha, alpha_c) = -0.25235335423912797
  * (b, alpha_c) = -0.1350561775299012
  * (a, alpha_c) = -0.05146359498906435
  * (a, alpha) = 0.08758899323218776
- quality: good
- extra: <4 items>
- device_components: ['Q1', 'Q2']
- verified: False
AnalysisResult
- name: alpha
- value: 0.9752+/-0.0006
- χ²: 0.7111268423386916
- quality: good
- extra: <4 items>
- device_components: ['Q1', 'Q2']
- verified: False
AnalysisResult
- name: alpha_c
- value: 0.9870+/-0.0009
- χ²: 0.7111268423386916
- quality: good
- extra: <4 items>
- device_components: ['Q1', 'Q2']
- verified: False
AnalysisResult
- name: EPC
- value: 0.0098+/-0.0007
- χ²: 0.7111268423386916
- quality: good
- extra: <4 items>
- device_components: ['Q1', 'Q2']
- verified: False

References

[1]

Easwar Magesan, J. M. Gambetta, and Joseph Emerson, Robust randomized benchmarking of quantum processes, https://arxiv.org/abs/1009.3639.

[2]

Easwar Magesan, Jay M. Gambetta, and Joseph Emerson, Characterizing Quantum Gates via Randomized Benchmarking, https://arxiv.org/abs/1109.6887.

[3] (1,2)

David C. McKay, Sarah Sheldon, John A. Smolin, Jerry M. Chow, and Jay M. Gambetta, Three Qubit Randomized Benchmarking, https://arxiv.org/abs/1712.06550.

[4] (1,2,3)

Easwar Magesan, Jay M. Gambetta, B. R. Johnson, Colm A. Ryan, Jerry M. Chow, Seth T. Merkel, Marcus P. da Silva, George A. Keefe, Mary B. Rothwell, Thomas A. Ohki, Mark B. Ketchen, M. Steffen, Efficient measurement of quantum gate error by interleaved randomized benchmarking, https://arxiv.org/abs/1203.4550.

See also

• API documentation: randomized_benchmarking

• Qiskit Textbook: Randomized Benchmarking