Real World HF Tests
In these tests we tested the Airspy and RSP on the HF (0 - 30 MHz) band. We mostly ignored the HackRF for these tests as its performance on HF was quite bad and not really worth our time to compare. The HackRF may perform better with the use of an upconverter, like the SpyVerter but we did not test this. The HackRF is also very difficult to use at HF frequencies due to its lack of decimation or bandwidth options lower than 8 MHz. This means that the visual FFT resolution is very bad, making it difficult to visually identify signals.
The Airspy can only receive HF with the help of an upconverter. We used the recommend partner upconverter, the Spyverter and powered it via the Airspy's bias tee. Using an upconverter introduces some losses, probably around 6dB, so our expectation was that the RSP would have better SNR at HF.
We also noticed that the RSP had significant imaging problems when the Zero IF (ZIF) mode was used. However, when the Low IF (LIF) mode was used there were no images. In the LIF mode the maximum bandwidth is restricted to 1.536 MHz, but for HF this is okay, since we need to use lower bandwidth's to be able to accurately view the very narrow band HF signals. Thus all HF tests performed on the RSP will mainly use the LIF mode to get the best results, but we did use ZIF mode in some tests where no images were seen.
Here we tested the maximum SNR obtainable on the AM band with a long wire antenna. We ran the RSP at 1.536 MHz (actual bandwidth was actually around 1.646 MHz) and the Airspy at 10 MHz with decimation 4 or 2.5 MHz, cropped down to 1.6 MHz. The FFT interpolation addition for the Airspy was then 3 * (2.5 - 1.6) / 1.25 = +2.16 dB.
Even with the interpolation adjustment for the Airspy, the results showed that the RSP and Airspy had very similar results as their spectrum's were almost identical.
One major issue with the Airspy during this review that we had to address was what seemed to be USB or general PC noise. Originally, we had been running both the RSP and Airspy through a 10 meter long active USB extension cable. However, on the Airspy the use of this active extension cable caused some interference issues. The RSP also experienced a little interference from the active cable, but it was almost negligible. This interference only appeared on some bands, like the AM band and did not occur for VHF and above frequencies.
When the active USB cable was not used, and the device was plugged directly into the PC the noise disappeared. However the Airspy was also affected by which USB port was used. Some USB ports were more noisy than others, but this did not cause as much noise as with the active USB cable.
We also tested both SDR's with a galvanic isolator (the one from Nobu). This isolates the antenna from the device and PC, but at the same time introduces a little loss. Reception in the AM band was significantly improved with the galvanic isolator, but we didn't notice much difference when it was used on the other bands.
We demonstrate the PC interference issue in the images below.
The USB noise issue originally made us think that the Airspy/SV was poorer than the RSP at BCAM reception. However, once we stopped using the active USB extension cable the RSP and Airspy were pretty much identical. Adding in a galvanic isolator helped solve even more noise problems, but even with the isolator both the Airspy and RSP had similar performance.
For this test we used a Magnetic Loop antenna.
Here the Airspy/SV and RSP seemed to perform very similarly and we could not really determine any difference between the two in terms of sensitivity or overloading. We also tested the HackRF here but it was always about 10 dB poorer than the Airspy and RSP.
We also redid this test with a long wire antenna on the Airspy and RSP. Again the results where almost identical.
Here we tested reception on a STANAG signal. Again in this test we could not see any real difference between the Airspy/SV and RSP. The SNR's were nearly identical on average (we must take an error of +-6dB in these images as the HF signals in this band were rising and falling in strength fairly fast over a period of seconds).
In this test we used a long wire antenna and again results were pretty much identical. We note that 9 MHz was the only band other than BCAM that was significantly affected on the Airspy by the active USB cable issue.
This test was performed with a long wire antenna and once again no real difference between the Airspy and RSP could be detected.
At 15 MHz there was again no real discernible difference between the Airspy and RSP.
Once again, no real discernible difference could be seen.
It seems that both receivers were pretty much identical on these HF tests. The Airspy however had some noise issues when running it with an active USB cable and was sensitive to which USB port was used.
The RSP should be used in LIF mode to get the best results as the ZIF modes produced significant imaging, however this is okay as there is unlikely to be a situation where you need 8 MHz of bandwidth for HF.
Strong Signal Test: Airspy v.s. SDRplay RSP
Leif has already done some good tests regarding the dynamic range on the Airspy and the SDRplay RSP (YouTube) which we show again below. The test he did was to inject a strong interfering signal into the signal path and measure its effects on a nearby broadcast FM station. As the interfering signal increases in strength we can expect the FM signal to be degraded and fall into the noise floor. SDR's with better dynamic range specs will only degrade at stronger levels of interference.
Leif's results showed that the Airspy generally wins in terms of dynamic range. However, the SDRplay team has critiqued this test as Leif did not use the official drivers released by the SDRplay team which they write should give better performance. He also did not use the IF filter bandwidth reduction which might improve the RSP's max dynamic range by about 5 dB, at the expense of reducing the bandwidth.
Below are the results for the two tone tests done by Leif. Leif also made some two tone tests where the two tones were arranged for the 3rd order intermodulation product to appear in the passband of the desired signal. Here the Airspy was still better in most tests, but the SDRplay was better in some too.
Our own loss of reception (Dynamic Range) test
In this part of the review we wanted to compare the dynamic range of the Airspy and RSP ourselves. To do this we devised a simple test involving a DMR trunking channel and an interferer generated by the HackRF. The trunking channel was received by an antenna whilst the HackRF was used to inject a simulated GSM interferer at various offsets from the DMR signal. We used the DMR decoder program to monitor the DMR signal, and steadily increased the interferer gain on the HackRF. When a loss of synchronisation occurred on the DMR signal we recorded the HackRF gain value at which this happened. During the test we were free to adjust the Airspy and RSP gain settings and center frequencies (to move intermodulation products out of the way) to try and obtain the best reception possible.
Please remember when testing like this only the relative measurements are meaningful, the absolute numbers mean nothing. The test is simply to show which receiver works better in the presence of strong signals, not to show any quantitative difference.
|Offset (MHz)||Airspy (Higher is better)||RSP (Higher is better)|
|-50||23 (Interefer Harmonic directly on top of signal)||14|
|50||22 (Interefer Harmonic directly on top of signal)||16|
In these tests the Airspy performed significantly better. It was able to tolerate much stronger interference at all offsets.
The RSP has the option to reduce its IF filter down to 200 kHz in order to improve dynamic range by better blocking in band interferers. However, we didn't really see much improvement when reducing the bandwidth like this. There was maybe at most a 1-2 dB improvement.
The RSP had issues with the strong interferer, even at offsets far from the centre. We believe that this problem is caused by reciprocal mixing, which is when the phase noise of the local oscillator mixes with a strong signal (the interferer) and causes interference to other weak signals. The Airspy with its low phase noise clock exhibited this problem significantly less.
SNR Lab Tests (Added 22 Feb 2016)
Some commenters of this review have pointed out the fact that the RSP has better sensitivity, and that this was not highlighted in our review. Since we tested in a real environment with several blockers (strong out of band signals) the RSP's edge in sensitivity could not be noticed when it suffered from intermodulation problems as intermodulation interference meant we had to reduce the gain or loose the signal.
Below we present a lab measurement of sensitivity. In this test we measure the SNR of a single tone on various frequencies on both the RSP and Airspy. No external interferers are present. We didn't have an accurate signal generator available, so instead we used the HackRF which transmitted directly into the two units via a 32dB attenuator and short cable connection. The HackRF is not very accurate, but we believe that it is useful enough to make relative measurements as we do here. The HackRF settings where: Amplitude 10m, RF 0, IF 0.
If you want to perform your own SNR tests remember the following tips:
- Shield the RSP as its plastic case is prone to near field leakage adding to the signal strength. We put ours in a simple metal box that used to contain biscuits and this blocked all near field interference.
- Do not measure the signal in the center of the RF spectrum as any DC spike will add to the SNR. Also ensure that there are no spurs under your test signals which can add to the SNR.
- If measuring at 10MSPS and 8MSPS on the Airspy and RSP, you must interpolate the Airspy result by adding 1.2dB to its measured SNR.
- Measure fully zoomed out so that you can be sure that you have not changed the FFT bin density more than the difference between 10MSPS and 8MSPS.
- Measure after warming up the devices for at least 30 minutes, the Airspy seems to loose about 0.5 dB on some frequencies when it is warm.
Below are some lab SNR results tested at every 25 MHz.
We should point out that the RSP seems to get its sensitivity edge due to its use of a low noise amplifier on its front end with a 1dB noise figure. We postulated that if we placed an LNA4ALL (~0.75 dB NF) on the front end of the Airspy and RSP and redid the SNR testing, we'd see that the two units are very similar. The test is done below at 100 MHz spacings.
|Frequency||RSP w/ LNA||AS w/ LNA (Interpolated)||Difference|
As predicted the RSP is indeed more sensitive than the Airspy in this lab test which measures the SNR of a single tone without any interferers present. Below 476 MHz the difference is not great, but above the difference becomes quite noticeable. The Airspy especially suffers in sensitivity above about 1.2 GHz. The reason the SNR advantage was not seen in the real world tests is because the RSP tended to overload on strong out of band signals much easier than the Airspy, meaning that the SNR could not reach its full potential.
We also believed that the better SNR in the RSP was due to its use of a built in LNA on its front end. Our tests with an external LNA on the front of both devices show that this seems to be true. With the external LNA the sensitivity of both devices was nearly identical with the Airspy having a slight edge.
To be clear about this test: We don't think a lab SNR test by itself can give a very good picture on what SDR is best as other factors like dynamic range etc also play big parts. This is why we prefer to do real world tests which test all factors at once. In the future we hope that a professional agency like the ARRL can do a review that includes full set of lab tests including tests like two tone, multitone and more.
The current consumed by the SDR is important if you wish to use it on a battery powered device. From the results we see that the RSP is the most power efficient device, with the Airspy and HackRF requiring about double the current.
0.4A (with SpyVerter)
0.44A (LNA On)
PPM Drift and Offset Test
To test the oscillator drift we tuned to a 1.5 GHz L-Band signal and watched the drift for about 40 minutes using spectrum lab. A stable signal is important for decoding signals as many digital decoders cannot handle signals that drift too fast.
The Airspy with its temperature compensated oscillator (TCXO) had an initial 0 PPM offset and drifted the least as is expected with the TCXO. It drifted about 500 Hz over 40 minutes from a cold start giving a PPM drift of about 0.3 PPM, which is within the 0.5 PPM TCXO spec. After 5 minutes of warm up the drift was only about 150 Hz, which is about 0.1 PPM.
The RSP has a crystal oscillator rated at 10 PPM. Our unit had an initial offset of -5 PPM and drifted about 1.6 kHz after 40 minutes from a cold start giving a PPM drift of about 1 PPM. After about 10 minutes the RSP drift stabilised down to about 0.1 PPM. Even though it does not use a TCXO the RSP drift is quite low, probably because it is power efficient and does not generate much internal heat, as well as having a much large PCB to dissipate the heat into. However, since no TCXO is used external temperature changes from night to day for example could affect drift.
The HackRF has a crystal oscillator rated at 30PPM. Our unit had an initial offset of -18 PPM and a larger drift of about 4 kHz after 40 minutes from a cold start which is about 3 PPM. Like the RSP it also does not use a TCXO. But compared to the RSP its current usage is much higher, possibly creating more heat which makes the oscillator drift much more.
Screenshots of the HackRF at 20 MSPS
As a bonus to highlight a good feature of the HackRF we show some screenshots showing wideband reception of some signals with the HackRF running at 20 MSPS.
It is clear from our review that there is no overall "winner", each SDR has their own strengths and weaknesses and what you choose will depend on your needs and budget. They are all designed for different markets. The Airspy clearly works significantly better in tough RF environments than the RSP does. However the RSP comes in at over $50 to $100 less and does not require an add on upconverter to listen to HF. The HackRF has poor RX performance, but has the widest bandwidth, tunable range and can transmit.
If we were to choose a unit we would say between the Airspy/RSP and HackRF, pick the Airspy/RSP if you are interested in scanning, DXing or just browsing the radio spectrum. Pick the HackRF if you are more interested in experimenting with locally generated radio signals/devices (such as for reverse engineering wireless devices). Between the Airspy and RSP, pick the Airspy if you live in suburban/city areas and want the best reception, or pick the RSP if you live rural or are more concerned about budget.
In table form based on the results of this review we make the following recommendations:
Advantages: The Airspy is the clear winner in terms of overall RX performance. Its natural high dynamic range allows for excellent SNR and reception of weak signals when in the presence of nearby strong signals. There are very few to no images caused by strong signals in the Airspy so the spectrum is very clean. It also works well with external filters and LNA's and has good official software support for Windows and the Raspberry Pi.
Disadvantages: Costs $50 more than the RSP. And another $50 if you want HF capability. Plus shipping costs. Slightly less upper frequency range than the RSP. Needs a fast modern PC to run.
Should you buy it?
The Airspy costs $199 USD, or $249 USD if you buy the SpyVerter for HF, plus $5-$20 shipping depending where in the world you are.
The Airspy is the best for users in need of the best RX performance.
This is the best unit if you live in a tough RF environment like in a city or suburbia or intend to use an external LNA. Note that you will need a fast PC to run the Airspy.
Seems to be marketed more towards professional RF users, but also has a strong amateur/scanner user community.
Advantages: The RSP is the winner in terms being the cheapest all-in-one RX unit that is much better than an RTL-SDR. The RSP can tune to HF frequencies out of the box without an add on, has a higher top frequency of 2 GHz, doesn't need a high end PC, is generally more sensitive than the Airspy due to its built in LNA and is $50 USD cheaper with free shipping in the USA (or $100+ cheaper if you only consider the Airspy & Spyverter combo for HF).
Disadvantages: Its capabilities in the presence of very strong signals are not as good as the Airspy so overloading in suburban/city settings is a problem.
Should you buy it?
The RSP costs $149 USD with free shipping in the USA, or £99 + VAT + ~£10 shipping in the EU.
The RSP is the best for users who want a low cost all in one RX device with decent, but not great RX performance.
The RSP does VLF to UHF, and can work with slower PCs. But don't buy this unit if you have problems with strong signals in your area or if you want to use an external LNA.
Seems to be mainly marketed towards amateur/scanner users.
Advantages: The HackRF is the winner in terms of being an all rounder. It can TX, it has the widest bandwidth and frequency range.
Disadvantages: Its RX performance is poor compared to the Airspy or RSP. Needs a modern PC for higher bandwidths. General RX software support isn't great.
Should you buy it?
The HackRF costs $299 USD + shipping costs.
The HackRF is the best for RF experimenters/people who want an all in one RX/TX device and don't need great RX performance for DXing. It is great for reverse engineering wireless devices.
We think it is more designed to be used with custom software written in GNU Radio or Python.
It's main selling point is is wide frequency range, wide bandwdith and TX capability. Don't buy the HackRF if you are looking for RX performance better than an RTL-SDR or want a DX radio.
Seems to be marketed more towards the hacker/security/electronics or reverse engineering crowd.
Replies from the Manufacturers of the Airspy and SDR Play
Before we posted this review we sent a copy to the manufacturers of the Airspy and the SDRplay RSP so that they could fact check our review for mistakes or bad testing methods. Here are their responses:
Initially we were confused about what sort of data came out of the RSP. The SDRplay team wrote:
The RSP does deliver raw samples from the ADC, but as the MSi001 is capable of delivering analog I/Q signals, you need dual ADCs to sample the output from the tuner. The max sample rate for the Airspy single channel ADC is 20 MS/s, which is necessary to deliver 10 MHz of bandwidth without major aliasing problems. The Dual ADCs on the RSP can each sample in excess of 10 MS/s which together means that the USB throughput needs to be 2 x 10 MS/s x 12 bits, which is the same as the Airspy. The Airspy then needs to de-rotate the sampled IF to digital I/Q, whereas the RSP does not need to do this as the information is in I/Q form. If the RSP is used in low IF mode, then only a single ADC is used and the USB throughput is halved, but as correctly observed, the IF bandwidth is limited to 1.536 MHz.
Regarding our LNA tests the SDRplay team wrote the following which prompted us to do LNA testing with higher loss coax cable:
Regarding the external LNA [LNA4ALL], we appreciate that people use off-the shelf LNAs, but a 20 dB LNA of gain is in excess of what is really necessary to overcome the loss of the coax cable. Our point is that when using an external LNA, the system still needs to be ‘designed’ or people might be tempted to cascade these LNAs mistakenly thinking that 40 dB of gain will improve performance. If the cable loss was in fact 14 dB (maybe 10-15m of cable), the performance of the RSP would actually be better than if it is only 4-5 dB.
Regarding the LNA tests the Airspy team write:
[In the LNA tests] the LNA [LNA4ALL] might not have enough dynamic range in a high performance setup since it overloads before the Airspy.
The Airspy team suggested that the LNA should improve SNR on the Airspy, but that the LNA we used was not suitable for our environment due to it overloading on BCFM signals. They suggested that we should have used a LNA with a much higher dynamic range such as the PGA103+.
When asked about the PC requirements of the SDR the SDRplay team wrote:
The benefit of the isochronous mode driver is that it reserves the necessary USB bandwidth, something that does not happen with a bulk mode driver. As a consequence, depending what else the PC is actually doing, there is a greater risk of buffer overflows and packet losses with a bulk mode driver than with an isochronous mode driver. We do occasionally get complaints from developers that we chose not to use a WinUSB (bulk mode) driver, but the reason for doing this was to open up the range of platforms capable of using the device’s full capability.
When we asked the Airspy team why they did not use an isochronous driver they replied:
[With an Isochronous driver] you have no means to know how many samples you lost if your system had a transfer error - which means you can't implement coherent receivers with it even if the sampling is synchronised.
Isochronous is limited in bandwidth and has no means for correcting streaming errors. That's why it is only used in non critical consumer products, like TV tuners etc. The high end SDRs all use the Bulk transfer mode..
About Leif's tests the SDRplay team wrote:
Regarding the tests performed by Leif Asbrink, our principle concern was that to interface the RSP to Linrad, he had used a driver which was not developed by either Mirics or SDRplay and was known to contain bugs which prevented proper control of the RSP and had a sub-optimal gain map. We felt that the use of this driver was likely to compromise the results he was able to achieve.
If you have any experiences or comments about one or more of these SDR's, or if you find any mistakes we have made then please let us know about them in the comments section.
Some commenter's have rightly pointed out the fact that the RSP has better sensitivity, and that this was not highlighted in our review. Since we tested in a real environment with several blockers the RSP's edge in sensitivity could not be noticed when it suffered from intermodulation problems as intermodulation interference meant we had to reduce the gain or loose the signal. We have now added a lab SNR test in the Other Tests section.
Some commenter's had concerns that we should not test in SDR#, as this is built for the Airspy and therefore all testing in SDR# should benefit it. However, as mentioned earlier we discovered that the RX results in SDR#, HDSDR and SDR Console appear to be identical. It seems that the only disadvantage SDR# gives to other devices is the inability to use third party plugins. We chose to test in SDR# because of its SNR meter and easy ability to control the FFT resolution and bandwidth crops for better screenshots.
Although every care was taken to be accurate in this review, please note that we are not a professional RF testing agency. We bought the Airspy R1, SDRplay RSP and HackRF SDR's with our own funds, however we received a complimentary Airspy R2 and Spyverter from the creators to use in our review as they wanted us to write the review using their newer hardware.
If you want to read other reviews, simply search for "airspy review", "sdrplay review" and "hackrf review" on this site using our search function, or on Google. Though we believe that we have the first review that actively compares the three SDR's together.