SHOUTcast Stream Ripper

• Download source file - 2.32 Kb

Introduction

This article is an extension of the article from Dani Forward. It implements the SHOUTcast protocol to get the metadata header from the SHOUTcast streams and read out the song titles. With this information, it is possible to automatically split the songs, store them as MP3 files on the hard disk and give them the correct song title that comes with the stream.

The source code

This is a 'one lazy night' project and doesn't pretend to be the best implementation of ripping the SHOUTcast stream. But it works simple and fine and may give you an idea, how to get the necessary information from the SHOUTcast stream and how to use them. So, any comments or improvements would be appreciated!

System libraries

First of all, we need the following Libraries:

using System;
using System.IO;
using System.Net;
using System.Text.RegularExpressions;

Establish the connection to the SHOUTcast Server

Then we establish a connection to the SHOUTcast Server. To get the song titles from the SHOUTcast stream, we need to alter the HttpWebRequest header and add "Icy-MetaData: 1". With that flag set, the SHOUTcast servers will send us the metadata with the song titles, if available.

The stream always begins with the ICY metadata header that comes with the HttpWebResponse object. This header contains some information about the SHOUTcast server and might look like this:

icy-notice1: This stream requires <A href="http://www.winamp.com/" target=_blank>Winamp</A> 
icy-notice2: SHOUTcast Distributed Network Audio Server/Linux v1.9.5
icy-name: RadioABF.net - Paris Electro Spirit Live From FRANCE
icy-genre: Techno House Electronic
icy-url: <A href="http://www.radioabf.net/" target=blank>http://www.radioabf.net/</A>
content-type: audio/mpeg
icy-pub: 1
icy-metaint: 32768
icy-br: 160

The icy-metaint: 32768 parameter is the most important value for us, because it tells us the blocksize of the MP3 data. In this example, the size of one MP3 block is 32768 bytes. After the connection has been established, the stream starts with an MP3 block of 32768 bytes. This block is followed by one single byte, that indicates the size of the following metadata block. This byte usually has the value 0, because there is no metadata block after each MP3 block. If there is a metadata block, the value of the single byte has to be multiplied by 16 to get the length of the following metadata block. The metadata block is followed by an MP3 block, the single metadata length byte, eventually a metadata block, an MP3 block, and so on (source: Shoutcast Metadata Protocol by Scott McIntyre).

  [STAThread]
  static void Main()
  {
   // examplestream: Radio ABF - http://www.radioabf.net
   // url: http://relay.pandora.radioabf.net:9000

   // station parameters
   String server = "http://relay.pandora.radioabf.net:9000";
   String serverPath = "/";

   String destPath = "C:\\"; // destination path for saved songs

   HttpWebRequest request = null; // web request
   HttpWebResponse response = null; // web response

   int metaInt = 0; // blocksize of mp3 data
   int count = 0; // byte counter
   int metadataLength = 0; // length of metadata header

   // metadata header that contains the actual songtitle
   string metadataHeader = ""; 
   // last metadata header, to compare with 
   // new header and find next song
   string oldMetadataHeader = null; 

   byte[] buffer = new byte[512]; // receive buffer

   // input stream on the webrequest
   Stream socketStream = null; 
   // output stream on the destination file
   Stream byteOut = null; 

   // create request
   request = (HttpWebRequest) WebRequest.Create(server);

   // clear old request header and build 
   // own header to activate Icy-metadata
   request.Headers.Clear();
   request.Headers.Add("GET", serverPath + " HTTP/1.0");
   // needed to receive metadata informations
   request.Headers.Add("Icy-MetaData", "1"); 
   request.UserAgent = "WinampMPEG/5.09";

   // execute request
   try
   {
    response = (HttpWebResponse) request.GetResponse();
   }
   catch (Exception ex)
   {
    Console.WriteLine(ex.Message);
    return;
   }

   // read blocksize to find metadata block
   metaInt = Convert.ToInt32(
             response.GetResponseHeader("icy-metaint"));

Receive bytes and separate the metadata from the MP3 data

After the connection to the SHOUTcast server has been established, byte blocks are read from the stream in an endless loop. Every single byte from the MP3 block will be counted and written to the output stream. As long as no metadata block with a song title information has been received, no output file will be created and no MP3 data will be written.

This is an example of a metadata block within the stream:

StreamTitle=' House Bulldogs - But your love (Radio Edit)';StreamUrl='';

With Regular Expressions it's quite simple to extract the song title from the metadata block. After 32768 bytes have been counted and written, the MP3 block is followed by the metadata length byte. This value is multiplied by 16 and stored in the headerLength integer. If this field is != 0, the metadata header will be written into the metadataHeader string and the headerLength is decremented by 1. When the headerLength field reaches 0, the complete header is written to the metadataHeader string. Now, the metadataHeader string will be compared with the oldMetadataHeader string that stores the last read metadata block. If the new block is not equal to the last block, that means the song has changed. Then, the song title will be extracted from the metadata block, a new file will be created with the extracted title and the output stream will be set to this file. Then, the writing process of MP3 data to the file starts again.

   try
   {
    // open stream on response
    socketStream = response.GetResponseStream();
    
    // rip stream in an endless loop
    while (true)
    {
     // read byteblock
     int bytes = socketStream.Read(buffer, 
                              0, buffer.Length);
     if (bytes < 0)
      return;

     for (int i=0 ; i < bytes ; i++)
     {
      // if there is a header, the 'metadataLength' 
      // would be set to a value != 0. Then we save 
      // the header to a string
      if (metadataLength != 0)
      {
       metadataHeader += Convert.ToChar(buffer[i]);
       metadataLength--;
       // all metadata informations were written 
       // to the 'metadataHeader' string
       if (metadataLength == 0) 
       {
        string fileName = "";

        // if songtitle changes, create a new file
        if (!metadataHeader.Equals(oldMetadataHeader))
        {
         // flush and close old byteOut stream
         if (byteOut != null)
         {
          byteOut.Flush();
          byteOut.Close();
         }
            
         // extract songtitle from metadata header. 
         // Trim was needed, because some stations 
         // don't trim the songtitle
         fileName = 
            Regex.Match(metadataHeader, 
             "(StreamTitle=')(.*)(';StreamUrl)").Groups[2].Value.Trim();
                                
         // write new songtitle to console for information
         Console.WriteLine(fileName);

         // create new file with the songtitle from 
         // header and set a stream on this file
         byteOut = createNewFile(destPath, fileName);

         // save new header to 'oldMetadataHeader' string, 
         // to compare if there's a new song starting
         oldMetadataHeader = metadataHeader;
        }
        metadataHeader = "";
       }
      }
      // write mp3 data to file or extract metadata headerlength
      else 
      {
       if (count++ < metaInt) // write bytes to filestream
       {
        // as long as we don't have a songtitle, 
        // we don't open a new file and don't write any bytes
        if (byteOut != null) 
        {
         byteOut.Write(buffer, i, 1);
         if (count%100 == 0)
          byteOut.Flush();
        }
       }
       // get headerlength from lengthbyte and 
       // multiply by 16 to get correct headerlength
       else 
       {
        metadataLength = Convert.ToInt32(buffer[i])*16;
        count = 0;
       }
      }
     }
    }
   }
   catch (Exception ex)
   {
    Console.WriteLine(ex.Message);
   }
   finally
   {
    if (byteOut != null)
     byteOut.Close();
    if (socketStream != null)
     socketStream.Close();
   }
  }

Method to create a new file and return the stream on this file

The method Main is used to create a new file and return an output stream onto this file. First, the method removes all the characters, that are not allowed in filenames. Then it checks, if the destination folder exists. If not, it will be created. After that, the method checks, if the filename already exists. If it exists, the file will not be overwritten. Instead, a new file with the filename <filename>(i).mp3 will be created.

  private static Stream createNewFile(String destPath, 
                                          String filename)
  {
   // remove characters, that are not allowed 
   // in filenames. (quick and dirrrrrty ;) )
   filename = filename.Replace(":", "");
   filename = filename.Replace("/", "");
   filename = filename.Replace("\\", "");
   filename = filename.Replace("<", "");
   filename = filename.Replace(">", "");
   filename = filename.Replace("|", "");
   filename = filename.Replace("?", "");
   filename = filename.Replace("*", "");
   filename = filename.Replace("\"", "");

   try
   {
    // create directory, if it doesn't exist
    if (!Directory.Exists(destPath))
     Directory.CreateDirectory(destPath);

    // create new file
    if (!File.Exists(destPath + filename + ".mp3"))
    {
     return File.Create(destPath + filename + ".mp3");
    }
    // if file already exists, don't overwrite it. Instead, 
    // create a new file named <filename>(i).mp3
    else 
    {
     for (int i=1;; i++)
     {
      if (!File.Exists(destPath + filename + 
                                "(" + i + ").mp3"))
      {
       return File.Create(destPath + filename + 
                                 "(" + i + ").mp3");
      }
     }
    }
   }
   catch (IOException)
   {
    return null;
   }
  }

Summary

This is a quick example, how to use the metadata in the SHOUTcast streams, to automatically name and split the stream into separate MP3 files. But it is not absolutely 100% safe for all possible streams. For example, if a station doesn't send any track information and the metadata block looks like this: StreamTitle='';StreamUrl='';, the program would split the songs correctly, but would name them ".mp3", "(1).mp3", "(2).mp3" and so on. But I kept it simple to show you just the basics of the protocol.

With some experience about using threads, it is no problem to use this code for downloading multiple streams at the same time. I have added some extra features, like different destination folders for the different streams, storing the stream information in an XML file, a view over all the streams with their status, downloaded bytes, bytes per second, etc. All this together with a GUI gives you quite a nice program to rip multiple streams at the same time. But as I mentioned at the beginning of this article, it was a 'quick and dirty' program that was developed during a lazy nightshift, and it's nothing I would let somebody see the spaghetti-source code of. ;-)

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here