Understanding Lossless vs Lossy audio compression (MP3 vs FLAC vs WAV)

Introduction

I've recently spent some time reading through some audio forums to increase my knowledge of audio amplifiers. I was building a headphone amplifier and wanted additional input into how the achieve the best bang for buck.

What I noticed is that there are many misconceptions about various computer audio elements that are causing many people to spend lots of money and time trying to achieve benefits in the wrong places.

Over the next few weeks I'm going to write some articles to try and dispel some of these misconceptions. My articles are going to concentrate on what can and can't effect the audio quality in a computer system (your PC), and the technical reasons why this is so.

The idea here is not to attempt to prove any subjectivists wrong over their beliefs, but to provide the information to enable them to make their own decisions.

In this article, I'm going to start off with a rather low-tech explanation of the difference between MP3 (as an example of lossy compression), FLAC (as an example of lossless compression) and WAV (an example of no compression) files.

For simplicity sake, when I refer to audio files in this article, I'm referring to CD quality 44 KHz 16 Bit rips to WAV, FLAC and MP3.

What is digital audio?

Here is a little info to form a basis of what digital audio is to help understand the following parts of this article.

Audio typically starts by being produced in the analogue world, that is, what musicians play on instruments and what we hear with our ears - this is all analogue. This analogue sound is a bunch of sound waves that are processed by a ADC (an Analogue to Digital Converter) to create a bunch of samples using a process call pulse code modulation. Don't worry about the full technical details in those links unless you're interested in them, you won't need to read or understand them to follow this article. A great detailed page to start on if you're interested in the gory details is the Wikipedia page on digital audio, but again, this is only for information and is not required for this article.

To listen to this digital audio it must be converted back into an analogue wave (sound). This is done using the opposite process by a device called an digital to analogue converter (DAC). The DAC takes the data generated by the ADC and converts it back to analogue so we can hear it.

WAV Files - No compression

WAV files are simply a set of numbers that represent the wave form (or samples) taken from the analogue world and require little to no processing to deliver to a DAC (a Digital to Analogue Converter).

A wave file just contains the numbers that the ADC produced, saved to a file. To reverse the process, the numbers are read from the file and given to a DAC that converts it back to analogue. Quite simple, not much can go wrong here.

Lossless Compression - FLAC

Flac files contain the same data as WAV files except they are compressed using a lossless compression algorithm.

I consider the algorithm used for FLAC files too complicated for this article, so instead of explaining it's implementation I'll use a simple form of lossless compression to explain how lossless compression works and why it will never lose any information due to its compression.

I'll describe a technique of substitution which is simple to understand and illustrates lossless compression well.

Let's take this simple block of text that we want to compress, but we don't want any loss of information.

i have a feline, it sat on a mat. i like the feline because his fur is black. the feline that sat, is black. the mat is black, the feline is black. his fur is on the mat.

Now, I'm providing a very loose example here that will yield poor compression, but this is only to show you how the process works.

Let's assign a number to represent each word exists in that text, we'll call this the 'map':

feline = 1
i = 2
have = 3
a = 4
it = 5
sat = 6
mat = 7
on = 8
the = 9
like = 8
his = 10
fur = 12
is = 13
that = 14
black = 15
because = 16

Now that we have a number for each word, we can replace each instance of the word with its corresponding number.

2 3 4 1, 5 6 8 4 7. 2 8 9 1 16 10 12 13 15. 9 1 14 6, 13 15. 9 7 13 15, 9 1 13 15. 10 12 13 8 9 7.

Now, as you can see that the text is compressed into a much smaller sentence. Now this is rather poor compression as you need to include the 'map' of words to numbers. This will be more effective for larger blocks of text with more repeating words. For example if our text was repeated many times the compression is more effective. Remember this technique is only for illustration purposes and is not a practical implementation of lossless compression.

As you can see, to get the original text back you would just replace each number with its corresponding word. There will be no lost information and the result will be identical to the original block of text.

As shown in this example, the data in a Flac file when uncompressed is exactly the same as the data that was used before compression, that is, there is no loss of data what-so-ever.

This is very different from lossy compression.

MP3 lossy compression

MP3s use what is called lossy compression, that is, information is lost in the compression process and the original can never be recreated from the compressed version.

The main technique behind MP3 compression is to remove parts of the audio data that is believed to no be perceivable to humans, that is, we won't hear the difference. The higher the compression rate, the more information is removed and the lower the quality. Now whether we can perceive this removed data is not part of this article, although I may touch on that in a later article.

To illustrate lossy compression, I'm going to use an even less effective compression example (and somewhat ridiculous one at that) as I want to illustrate the idea in a simple manner.

Let's take our original block of text again

i have a feline, it sat on a mat. i like the feline because his fur is black. the feline that sat, is black. the mat is black, the feline is black. his fur is on the mat.

Now if I was to attempt to compress this block of text by removing (or changing) things that I don't think make a difference, the text can still convey its message but the original will be lost, here is my attempt.

i have a cat sat on a mat i like the cat coz his fur is blck the cat that sat is blck the mat is black the cat is blck his fur is on the mat

Now, in my poor attempt to compress this block of text, I've replaced 'feline' with 'cat', removed all punctuation, misspelt 'black' as 'blck' etc. Now if I sent that text to a friend and asked them to reproduce the exact original, this could not be done it might be close, but is very unlikely to be the same. But the text still portrays a very similar message (albeit poorly).

Summary

Ok, so I've attempted to show you how lossy versus lossless compression techniques differ and why lossless is lossless, there is never any degradation of the data, when played back it will be exactly the same as the original, every time.

If you came here believing that WAV files are superior to FLAC (or other lossless compressed files) then I hope I have dispelled that misconception and you can happily compress your WAV files to FLAC knowing that you're not losing any data in the process.

Note also, that the FLAC file format allows other additional meta data like ID3 tags to be embedded in the file, such as track name, artist info, images, etc. The WAV file format does not provide this facility and is a major drawback.

I hope this has helped you, if it has or if you think I've got something wrong or could explain something better, please leave a comment.

Choosing a USB DAC Headphone amplifier - Aune mini USB DAC mini review.

I don't normally do reviews, but sometimes you find a product that just deserves a positive review.

Before I get into the review - or plug may be a better description - I want to give my opinion (everyone has one) on some claims made in the audio industry.

Choosing a USB DAC Headphone Amplifier

Diminishing returns

From my own research - once subjective claims are removed from the mix - I found there there are some significant diminishing returns in this area - that is, don't expect the increase in performance (if any) to be proportional to the amount you spend - keep this in mind when shopping for audio gear - there are a lot of very expensive products that while they often look fantastic, and seem to contain awe inspiring components, their increase in performance is usually only via the placebo effect. Be a sceptic, and don't waste enormous amounts of money based on anecdotal subjective claims regardless of the number of followers - the audio world can be like a battle of religions with many followers.

Huge range

These days there is an enormous range of USB DAC Headphone amplifiers and what ever other combination of hardware you want to mix in. These range from as low as US$40 to many $1000's.. If you're like most online shoppers, you'll use eBay as a starting place for your USB DAC shopping - there are a lot of choices, so many it takes some work to understand the differences.

DACS, bits, hertz, my head hurts!

So let's go through what I have learnt in this area, I want to break this down so it's simple for you to decide what you really need or can and will use.

Different DACs can process different signal combinations. Common ones are:

  • 16 bit 48 KHz
  • 24 bit 96 KHz
  • 24 bit 192 KHz

If your source music is from CD or tracks ripped from CD to mp3, flac, wav, etc then this source audio is 16 bit 44 KHz, such, a device that can do 16 bit 48 KHz will be sufficient - there is no benefit of having a larger bit depth or higher sampling frequency here.

If you have sources that are 24 bit 96/192 KHz then you may want to put extra consideration into the DAC capabilities.

Typically devices that support 24 bit 96/192KHz over USB require additional drivers - this might be an issue if you want portability and ease of installation.

Whereas, devices that support 16 bit 48Khz over USB typically don't require additional drivers.

24 bit 96Khz is usually available via optical or coaxial SPDIF connectivity.

Amplifier Circuit Type

There are many types of amplifier circuits, valve, mosfet, opamp, etc - the most common in USB DAC Headphone Amps are OpAmps.

OpAmps are a good choice here as they offer very low distortion, have little colouration on the output signal are cheap and are often simple to replace and test various types to suit your taste.

The Aune mini USB DAC

With that out of the way, let's look at the Aune.

If you're in the market for a USB DAC and Headphone amplifier the Aune Mini USB DAC Headphone amplifier is an excellent choice.

I purchased mine directly from the manufacturer's eBay store which I highly recommend as their service is excellent - their store representative Candy is very helpful and responds very quickly to email - usually within minutes. I have no affiliation with these guys, I just like their products and service.

Here is a picture of mine:

As you can see the front panel is a solid 8mm thick piece of aluminium, is well finished and has aune engraved (CNC machined) on it.

The components and build quality are excellent for this price range, I was very surprised when I opened mine:

As you can see, it contains quality components and is well constructed.

The underside again shows quality construction, no dry dodgy solder joints (I can't say the same for some other cheap Chinese ones that I own):

The Aune specifications:

  • Originated from the DIY project HIFIDIY.NET Mini USB DAC
  • Compatible with Windows XP/Vista/7 & Mac OS X 10.x and Linux (most variants)
  • Can be used as a DAC or headphone amp
  • OPA chip: BB OPA2134
  • Headphone AMP: Burr-Brown OPA2604+TIP41C*2+TIP42C*2 (Pull and push)
  • Digital Coaxial/USB input, RCAx2/6.35mm stereo analog output
  • Headphone amplifier built-in (TIP41C*2+TIP42C*2 (Pull and push) )
  • USB controller chip BB PCM2707
  • DIR chip -- TI DIR9001
  • Main DAC chip -- Burr-Brown PCM1793:
    - 24bit/192kHz Sampling Advanced Segment Audio Stereo DAC
    - Dynamic Range: 113 dB and THD+N: 0.001%
    - 8x Oversampling Digital Filter (from Official data-sheet)

The Aune supports 16 bit 48 KHz over USB and up to 24 bit 192KHz over SPDIF so you have the portability to play 16 bit 44 KHz sources and the SPDIF available if you need the higher source input.

Technical Note

Unlike most USB DACS, the Aune uses the PCM2707 as a USB receiver only, it then passes the received data to the PCM1793 DAC via i2s for A/D conversion - the PCM1793 has a reputation as a higher quality DAC than is commonly used and includes 8x oversampling.

Now on to the performance

I'm using my Aune with a pair of AKG K702 headphones, the Aune has no problems driving the K702s and sounds great. It is very detailed, lots of separation and has a fantastic sound stage.

I won't go into any more detail trying to describe my interpretation of the sound in 'audiophile speak'. I don't consider myself an audiophile and prefer to judge things by their performance and less by their look and status or reputation - let's try and keep the placebo effect well away from our judgement.

Conclusion

My Aune is my favourite headphone amp, it's very versatile, attractive, cheap and a solid performer. You really can't go wrong with this little amp, it's excellent value for money! As I said before, you can purchase these directly from the manufacturer's eBay store where you're sure to get great service and prompt registered delivery.

Music Player Minion for Firefox 4 - mpd client

I've recently made the switch to Firefox 4, but today I found that Minion has not been updated for Firefox 4, but don't fear.

It seems that latest version does work fine* under Firefox 4.

You just need to change the maxVersion from 3.6.* to 4.* in the installers install.rdf file.

Just follow these steps:

1) Download the plugin from https://addons.mozilla.org/en-us/firefox/addon/music-player-minion/

2) Rename the .xpi file to .zip

3) Open the .xpi file

4) Open install.rdf in a text editor (notepad, gedit, etc)

5) Change the value maxVersion from 3.6.* to 4.*

6) Save the changes to install.rdf - ensure the changes are saved to the zip file.

7) Rename the .zip file back to .xpi

8) Install the modified .xpi file in Firefox 4 (drag and drop into firefox works)

* Note: My playlist's alternate lines are a light grey background with a light yellow font - almost unreadable - Any ideas what's causing this?

Fedora 14 - An attractive and functional desktop - mini setup guide

I'm a big fan of Fedora and It's been my desktop of choice since version 1 - but the default desktop for me is, well, boring...

I don't want lots of wizz-bang flying features all over my desktop, but I do want it to look attractive and must be functional.

I've settled on this rather basic setup - it works for me.

Ok, first install the Rpm Fusion repository - specifically the rpmfusion-free-release-stable.noarch.rpm
and then rpmfusion-nonfree-release-stable.noarch.rpm repository configurations.

This enables all the non licence happy things like propriety display drivers, media codecs, etc

To install the required applications use [System->Administration->Add/Remove Software]

Step 1 - Your video card - hardware accelerated 3D

If you've got an nVidia or Ati card you will most likely want to install the propriety drivers - these have always worked best for me.

To find out what video card you have, from a terminal, run: lspci | grep VGA

$ lspci | grep VGA
03:00.0 VGA compatible controller: nVidia Corporation G92 [GeForce 9800 GT]

On machines with nVidia or ati cards I install the propriety drivers as these have always worked best for me. If you have an nvidia or ati card install the correct kmod package. For nVidia, one of: kmod-nvidia, kmod-nvidia-173xx or kmod-nvidia-96xx and for Ati: kmod-catalyst

Note the different nvidia modules are for the different generations of nvidia cards - most new cards will just want the kmod-nvidia module.

Step 2 - Installing required packages

Once you have your video capable of 3D, install the following packages:

  • compiz-fusion
  • fusion-icon
  • emerald
  • emerald-themes
  • cairo-dock
  • avant-window-navigator
  • awn-extras-applets

Then reboot to make the new video driver take effect (for 3d acceleration).

Step 3 - Starting Compiz with the Fusion Icon

Start the fusion-icon [Applications->System Tools->Compiz Fusion Icon] - this will start compiz-fusion - The Fusion Icon will give you access to all the configuration you'll need for compiz.

If compiz starts fine, then add fusion-icon to your startup programs [System->Preferences->Statup Applications] then drag and drop the app from [Applications->System Tools->Compiz Fusion Icon] into the Startup Applications window to have them start at startup. This will start compiz at login.

Step 4 - Move the top gnome panel

I keep the top gnome panel collapsed sitting in the bottom right corner of my screen (just in case I need it).

Right click on the top panel and select Properties and change the Orientation to Right and select Show hide buttons. Click close.

Step 5 - Using Cairo for our Application launcher

Start cairo-dock [Application->System Tools->GLX Dock (Cairo dock with HW 3D)] and configure it to be at the top of the screen and turn off it's task bar behaviour. Position=Top, Taskbar=None.

You can drag and drop applications from the Applications menus onto the Cairo dock to add them to the launcher.

Add GLX Dock to your startup applications.

Step 6 - Using AWN Avant Window Navigator for the taskbar

Then start [Applications->Accessories->Avant Window Navigator] and configure it to be at the bottom of the screen, and remove the applets the are already added (this we use just as a taskbar) I change the look and theme to make it look sexy...

Right click on AWN and select Preferences

Either select "Start AWN Automatically" in the AWN preferences or Add AWN to your startup applications.

Now remove the bottom taskbar... (Right click and select "Delete this Panel")

Collapse the top sidebar that was moved to the right...

Then you can use the fusion-icon to configure compiz, select the window decorator (emerald is a good one) and choose window decorations.

No Application menu icons?

If you find you have no icons showing in your Application menus follow the steps in my post here: Fedora no icons in menus - fix to show menu icons

Suggested compiz plugins

Launch the compiz configuration tool (via fusion-icon) and turn on Cube, Cube rotate and wobbly windows (and any other crazy effects you want)..

Done!

Fedora/Gnome no icons in menus - fix to show menu icons

Each time I install Fedora (10, 11, 12, 13 & 14) I don't have any icons in my system menus.

My typical setup is, kmod-nvidia, compiz-fusion, cairo-dock (top as launcher), avant-window-navigator (at bottom as taskbar).

With this setup (and maybe the default 2d gnome setup(?)) I don't have any icons in the gnome system menus (Accessories, Preferences, Administration, etc) - Even the Applications Menu shown in Cairo are missing their icons.

To fix this install the package gconf-editor, run it and go to desktop->gnome->interface there you will find a setting menus_have_icons - check this setting and your menus will now have icons.

I have no idea why this is always unchecked after a fresh install with this setup..

Exporting Blender 3D models with textures for use with LibGDX on Android - Using Wavefront .OBJ

Blender is an amazing free open source 3D content creation suite. Blender allows exporting to Wavefront .OBJ format which is just a simple text file.

LibGDX is a game development library for Desktop and Android games.

The Problem

My models blender I import into libGDX do not map the textures correctly. I struggled to find what was wrong for some time until I noticed this bit of code in libGDX:

public class ObjLoader {
        /**
         * Loads a Wavefront OBJ file from the given input stream.
         *
         * @param in the InputStream
         *
         */

        public static Mesh loadObj(InputStream in) {
                return loadObj(in, false);
        }
       
        /**
         * Loads a Wavefront OBJ file from the given input stream.
         *
         * @param in the InputStream
         * @param flipV whether to flip the v texture coordinate or not
         *
         */

        public static Mesh loadObj (InputStream in, boolean flipV) {
                String line = "";

                try {
                        BufferedReader reader = new BufferedReader(new InputStreamReader(in));
                        StringBuffer b = new StringBuffer();
                        String l = reader.readLine();
                        while (l != null) {
                                b.append(l);
                                b.append("\n");
                                l = reader.readLine();
                        }

                        line = b.toString();
                        reader.close();
                } catch (Exception ex) {
                        return null;
                }
                return loadObjFromString(line, flipV);
        }
...
}

Notice the boolean flipV on the loadObj() overload. Now I was using the ModelLoader.loadObj(InputStream in) call to load my objects, this by default calls the ObjLoader.loadObj(InputStream in) method.

The Solution

Well it turns out that blender exported Wavefront .OBJ files require the flipV variant of the loadObj() method to correctly map the textures to your models.

The simple solution is to replace any calls in your code to ModelLoader.loadObj(in) with calls to ObjLoader.loadObj(in,true)

That's it, now your textures will map correctly to your models.

Logitech mouse scrolling up goes back in browser

I have some Logitech mice that on some instances of Windows XP have a bug where rolling the scroll wheel up causes the browser to go backwards through the history (as if you clicked the back button). Sometimes it seems to go a little crazy, going backwards and forwards and is useless for scrolling.

Today I finally found the solution - and it's simple.

The Solution

There is a setting called "Sample Rate" which seems to effect this behaviour, this setting is found under the mouse settings in the control panel: Start->Control Panel->Mouse->Hardware->Properties->Advanced Settings->Sample Rate (reports/second)

This has a default value of 100 changing this to 200 has solved this issue completely.

Firefox - Clearing erroneous saved form data

We've all done it - Put a typo in our login name, or typed our password in the name field of a site login - Now you want to remove that erroneous entry from your saved form data...

Well Firefox has a somewhat hidden, not well known, but simple method for removing single pieces of saved form data.

How: Go to the form/page with the data you want removed, highlight the form entry area and highlight the entry you want removed (using the cursor keys may make things easier), hit delete - voila gone!

Android Market - Stuck at "Starting Download..."

You've got your new Android phone and you're trying to use your wifi to download apps from the Android Market only to have all downloads stuck at "Starting Download" - Why can't I download app from the Android Market?

My Problem

My router's firewall is configured to deny all inbound and outbound connections on all but the excluded (opened) ports.

My Solution

Open port 5228 for inbound and outbound TCP/UDP connections.

I hope this solves your issues....

Handy Hint for developers

If your application requires a specific, non common port open and your app is unable to connect - perhaps give the user a hint "Oy buster, you got 5228 open?".

Shake n' Bake - Fixing a dead nVidia 7900gs from a Dell 9400 / e1705

The Dell 9400 / e1705 is a nice laptop - great combination of high res 1920x1200 17" display and a dual core 2 duo processor - the ideal desktop replacement. Except - There exists a serious design flaw with this laptop - the video card gets very hot causing it to eventually fail.

The symptoms (any of these):

  • Flickering video
  • Graphic corruption on your screen (lines, stripes, dots, squares, jumbled mess)
  • Laptop won't turn on - on light flickers for a second and off again.

Cracked Solder Joints

It is most likely that your nVidia 7800/7900gs is failing due to the stress of heating up and cooling down.

This usually leads to cracked solder joints.

Shake 'n Bake

Shake 'n bake is a term I use for this method - shake the bits off and bake it!

Yes, that's right- we're going to bake the card in our kitchen oven at 200c for 10min.

This will melt the solder and repair the card.

Important Notes

  • The GPU on the video card is connected to the heat sink and requires thorough cleaning and high quality thermal paste (or similar) - I have used Arctic Silver 5 thermal paste.
  • To improve the cooling and life of this fix, you can purchase special 'copper shims" from eBay that aid with better cooling.
  • Be sure not to let your oven rise above 200c and limit the exposure to 10min.
  • If the first attempt is not successful, repeat. I have read that other people have required two or three roastings to fully repair their cards.

    I have done two and each only required one bake for 10min.

The two laptops I fixed have been working great ever since.. I'll post if they ever fail again.

Here's a video of my repair effort.

Syndicate content