|
Introduction
I wrote this tiny tool to compensate the limitations of the otherwise perfect iTunes player in reports generation. The program exports your iTunes library data to a Microsoft SQL Server (hence the name – sqlTunes) database which can then be queried using T-SQL. I am going to quickly review the underlying data structure of the iTunes library and then I will list some of the reports you can run on it.
Library Structure
iTunes stores the library data in two files – a proprietary binary file called iTunes Library.itl and its XML counterpart called iTunes Music Library.xml. Both files are located in the My Documents\My Music\iTunes\ folder.
The XML file is nothing more than just a representation of a generic dictionary. First it lists the library information, then the track data, and in the end the playlists. Here is a sample XML:
="1.0" ="UTF-8"
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Major Version</key><integer>1</integer>
<key>Minor Version</key><integer>1</integer>
<key>Application Version</key><string>6.0.1</string>
<key>Features</key><integer>1</integer>
<key>Music Folder</key>
<string>file://localhost/C:/Archive/Audio/iTunes/</string>
<key>Library Persistent ID</key>
<string>F8D57E57036B9A4E</string>
<key>Tracks</key>
<dict>
<key>36</key>
<dict>
<key>Track ID</key><integer>36</integer>
<key>Name</key><string>...</string>
<key>Artist</key><string>...</string>
<key>Album</key><string>...</string>
...
</dict>
...
</dict>
<key>Playlists</key>
<array>
...
</array>
</dict>
</plist>
As simple as that! sqlTunes ignores everything except the Tracks section which it reads recursively and fills in the Dictionary<string, object> object. It then builds INSERT statements and exports the data to the SQL Server.
SQL Structure
The structure is an exact copy of the library data. The database consists of one table called Track which is defined as:
CREATE TABLE [dbo].[Track] (
[Track ID] [int] NOT NULL,
[Name] [nvarchar] (200) NULL,
[Artist] [nvarchar] (100) NULL,
[Album] [nvarchar] (100) NULL,
[Grouping] [nvarchar] (100) NULL,
[Genre] [nvarchar] (100) NULL,
[Kind] [nvarchar] (100) NULL,
[Size] [int] NULL,
[Total Time] [int] NULL,
[Track Number] [int] NULL,
[Track Count] [int] NULL,
[Year] [int] NULL,
[Date Modified] [datetime] NULL,
[Date Added] [datetime] NULL,
[Bit Rate] [int] NULL,
[Sample Rate] [int] NULL,
[Comments] [nvarchar] (200) NULL,
[Play Count] [int] NULL,
[Play Date] [bigint] NULL,
[Play Date UTC] [datetime] NULL,
[Rating] [int] NULL,
[Track Type] [nvarchar] (100) NULL,
[Location] [nvarchar] (500) NULL,
[File Folder Count] [int] NULL,
[Library Folder Count] [int] NULL
) ON [PRIMARY]
This is probably not the best example to learn database normalisation but certainly enough to run our reports. sqlTunes will delete and re-create the table on each run. The database must exist, it will not create it.
Reports
And finally, here is the fun part. Let us start with something simple, say we want to know the average bit rate of the entire library:
SELECT CAST(ROUND(AVG(CAST([Bit Rate] AS float)),2) AS varchar)
AS [Average Bit Rate]
FROM Track
Now, let us get something more useful. This query lists your entire album collection:
SELECT DISTINCT Artist, Album, [Year], Genre
FROM Track
ORDER BY Artist, [Year], Album
This is a more correct version, it lists only the full albums. You will need to set the Track Count values to use it effectively:
SELECT Artist, Album, [Year], Genre
FROM Track
GROUP BY Artist, [Year], Album, Genre
HAVING COUNT(*) = MAX([Track Count])
ORDER BY Artist, [Year], Album
Likewise, this one lists the incomplete albums:
SELECT Artist, [Year], Album, Genre
FROM Track
GROUP BY Artist, [Year], Album, Genre
HAVING COUNT(*) < MAX([Track Count])
ORDER BY Artist, [Year], Album
Want to know the albums without the Track Count value? Here you go:
SELECT DISTINCT Artist, Album, [Year], Genre
FROM Track
WHERE ISNULL([Track Count],0)=0
ORDER BY Artist, [Year], Album
This one returns all the rated albums sorted by their rating. Handy if you have MP3s but want to update your CD collection:
SELECT Artist, Album, [Year], COUNT(*) AS [Songs Rated],
AVG(CAST(Rating AS float)) AS [Album Rating]
FROM Track
WHERE Rating IS NOT NULL
GROUP BY Artist, Album, [Year]
ORDER BY [Album Rating] DESC, [Songs Rated] DESC
This is the same for artists, can be useful when you are looking for new albums to buy:
SELECT Artist, COUNT(*) AS [Songs Rated], AVG(CAST(Rating AS float)) AS [Artist Rating]
FROM Track
WHERE Rating IS NOT NULL
GROUP BY Artist
ORDER BY [Artist Rating] DESC, [Songs Rated] DESC
This query reveals your genre preferences:
SELECT Genre, AVG(CAST(Rating AS float)) AS [Genre Rating], COUNT(*) AS [Songs Rated]
FROM Track
WHERE Rating IS NOT NULL
GROUP BY Genre
ORDER BY [Genre Rating] DESC, [Songs Rated] DESC
Further Developments
If this program proves to be useful, I will continue developing it in these three directions:
- Adding more reports - Please do post your requests, my imagination is limited but I can speak T-SQL :)
- Enhancing the interface - E.g. run reports directly from sqlTunes.
- Support more databases - Most iTunes users do not have Microsoft SQL Server. A file based database like SQLite or even Access will do the job as well.
License
History
- 2005-11-08: Initial version
- 2008-03-04: iTunes 7 compatibility
| You must Sign In to use this message board. |
|
| | Msgs 1 to 25 of 31 (Total in Forum: 31) (Refresh) | FirstPrevNext |
|
 |
|
|
Thanks for such a great tool.
I've been playing around with some reports, but got stuck on finding a suitable one for showing and summing track times I created a clumsy hack
--create time column ALTER TABLE track ADD tracktime datetime --convert total time to tracktime update track set tracktime = (convert(char(8), dateadd(ms,[Total time],0),108)) from track This does allow limited reporting on individual tracks, but not complete total times. Something for me to work on
I've also been playing around with some other simple reports
--genre album count select DISTINCT(genre), count(DISTINCT(album)) as 'Album count' from track GROUP BY genre order by 'Album count' DESC
--Smallest Album select Distinct Top 5 Artist, album, [year], Genre, [track count] FROM Track where [track count] is not null order by [track count] asc
--Largest Album select Distinct Top 5 Artist, album, [year], Genre, [track count] FROM Track order by [track count] desc
Has anyone tried to create reports using column [size] (i.e. see the largest album in physical size, or the biggest track)? Much like the [total time] field its a int column type, but I've not really started playing with that yet.
Thanks again David
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Is that imported into SQL? I can't seem to find it and I don't see it listed in the SQL code to create the table. That field exists in the iTunes Library XML file though.
Steve
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
 |
|
|
I've actually created something similar to this. To get past the database limitation, especially since you are recreating the single table every time, try using a DataSet. The application will no longer be dependent on any type of database and you can easily cache the dataset to a custom xml file that is far faster to read than the iTunes library. It still follows the same concept though, for up to date statistics, it needs to be regenerated every time. Let me know if you'd like to see the code that I've written. I'd be glad to share it but haven't had the time to go through and comment it or write up an article for clarity.
Life should NOT be a journey to the grave with the intention of arriving safely in an attractive and well preserved body, but rather to skid in sideways, burger in one hand, drink in the other, body thoroughly used up, totally worn out and screaming "WOO HOO......What a ride!"
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Hi,
Has anyone else had the error below when trying to run from source? I got it initially only with debug builds, but now also with release builds. I've even re-downloaded the source code (today) and rebuilt - this did not solve the problem.
I don't get this error in any other projects.
I am using Visual Studio 2005 Professional, version 8.0.50727.762. (This is SP1.) I do have the ReSharper addin, build # 259.
Regards, Peter
Error exception occurs on this line: Application.Run(new MainForm());
Exception text: System.ComponentModel.Win32Exception was unhandled Message="Window class name is not valid." Source="System.Windows.Forms" ErrorCode=-2147467259 NativeErrorCode=1400 StackTrace: at System.Windows.Forms.NativeWindow.WindowClass.RegisterClass() at System.Windows.Forms.NativeWindow.WindowClass.Create(String className, Int32 classStyle) at System.Windows.Forms.NativeWindow.CreateHandle(CreateParams cp) at System.Windows.Forms.Control.CreateHandle() at System.Windows.Forms.ProgressBar.CreateHandle() at System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible) at System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible) at System.Windows.Forms.Control.CreateControl() at System.Windows.Forms.Control.WmShowWindow(Message& m) at System.Windows.Forms.Control.WndProc(Message& m) at System.Windows.Forms.ScrollableControl.WndProc(Message& m) at System.Windows.Forms.ContainerControl.WndProc(Message& m) at System.Windows.Forms.Form.WmShowWindow(Message& m) at System.Windows.Forms.Form.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam) at System.Windows.Forms.SafeNativeMethods.ShowWindow(HandleRef hWnd, Int32 nCmdShow) at System.Windows.Forms.Control.SetVisibleCore(Boolean value) at System.Windows.Forms.Form.SetVisibleCore(Boolean value) at System.Windows.Forms.Control.set_Visible(Boolean value) at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context) at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context) at System.Windows.Forms.Application.Run(Form mainForm) at sqlTunes.Program.Main() in C:\src\csharp\sqlTunes\Win\Program.cs:line 36 at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args) at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args) at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly() at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart()
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
I downloaded the .exe version of the sqlTunes at the top of this page but it does not seem to have the update for iTunes 7 incorporated. Does anyone know if there is a place that I can obtain the most current version of this application in .exe form as I do not have Visual Studio? I am really interested to play around with it and see how it works. My E-Mail is jeff_shepler at hotmail dot com. Thank you very much. Jeff
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Thanks to Alexander Kojevnikov for the nice project, With it I could well get used to the work in C# and .Net. I have sqlTunes extended for database access with PostgreSQL. Selection of providers via a combobox. The access is via an abstract DB class, so it is also easy sqlTunes for access by mysql to expand. also there is a Button to create m3u files. download: http://www.lxsbbs.org/download/sqlTunes.zip
bye, jens.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Hi there!
Absolutely loving the idea of having my iTunes library in SQLServer, but having a major headache with it! Everytime I try and run the program, I get the error that it can't connect to the remote server? Im using SQL Server Express 2005 edition, and have tried lots of differenet things in the server box, but to no avail.
Does anyone have any suggestions? I saw an earlier post that suggested .\SQLExpress but this didn't work for me....
Any help would be most appreciated so that I can get on and use this cool tool!
Thanks in advance
Andy
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
i'd like to request a feature that would convert the xml file to .sql text file which could be imported to phpMyAdmin for mySQL users.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Instead of using a database at all, how about providing the option to import directly into a DataTable object? That way, anybody can use your program.
I'm currently loading all of the tracks into a DataTable object by getting them as objects through the iTunes COM interface. I've experimented with 13000 records. The fetch is slow (over a minute) but the DataTable and DataGridView seem to handle that number of records very well (on my relatively high end computer).
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
worked like a charm until I used an Itunes 7 doc. Any updates coming? This is a great easy to use tool.
WebzenNJ
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
I'm on holiday right now. I'll update the code when I get back to Sydney (end October).
Alexander Kojevnikov MCSD.NET Sydney, Australia
|
| Sign In·View Thread·PermaLink | 1.00/5 (1 vote) |
|
|
|
 |
|
|
Just a nudge to see if you can update for Itunes7. Seems to be the PlayDate field that has increased in size. However my C# is non existance. I am now getting a 404 Not Found when it trys to read the XML file. Could this also be the DTD??
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
 |
|
|
You can modify the code to work with iTunes 7 fairly easily. You need to modify the Helpers.cs and Definition.sql files. The problem appears to be with the size of a numeric field. I did not find out which one but I suspect it is one of the date fields so this may not be limited to iTunes 7.
In the Helpers.cs file, I modified two lines of code.
Line 270 changed to: val = long.Parse(s);
Line 406 added the following code. if (obj is long) { return ((long)obj).ToString(); }
Changed Definition.sql to: (Note added a few new columns and change int to numeric (18,0).)
if exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[Track]') and OBJECTPROPERTY(id, N'IsUserTable') = 1) drop table [dbo].[Track] GO
CREATE TABLE [dbo].[Track] ( [Track ID] [numeric] (18,0) NOT NULL , [Name] [nvarchar] (200) COLLATE SQL_Latin1_General_CP1_CI_AS NULL , [Artist] [nvarchar] (100) COLLATE SQL_Latin1_General_CP1_CI_AS NULL , [Composer] [nvarchar] (100) COLLATE SQL_Latin1_General_CP1_CI_AS NULL , [Album] [nvarchar] (100) COLLATE SQL_Latin1_General_CP1_CI_AS NULL , [Genre] [nvarchar] (100) COLLATE SQL_Latin1_General_CP1_CI_AS NULL , [Kind] [nvarchar] (100) COLLATE SQL_Latin1_General_CP1_CI_AS NULL , [Size] [numeric] (18,0) NULL , [Total Time] [numeric] (18,0) NULL , [Disc Number] [numeric] (18,0) NULL, [Disc Count] [numeric] (18,0) NULL, [Track Number] [numeric] (18,0) NULL , [Track Count] [numeric] (18,0) NULL , [Year] [numeric] (18,0) NULL , [Date Modified] [datetime] NULL , [Date Added] [datetime] NULL , [Bit Rate] [numeric] (18,0) NULL , [Sample Rate] [numeric] (18,0) NULL , [Comments] [nvarchar] (200) COLLATE SQL_Latin1_General_CP1_CI_AS NULL , [Play Count] [numeric] (18,0) NULL , [Play Date] [numeric] (18,0) NULL , [Play Date UTC] [datetime] NULL , [Rating] [numeric] (18,0) NULL , [Artwork Count] [numeric] (18,0) NULL, [Persistent ID] [nvarchar] (100) COLLATE SQL_Latin1_General_CP1_CI_AS NULL, [Track Type] [nvarchar] (100) COLLATE SQL_Latin1_General_CP1_CI_AS NULL , [Location] [nvarchar] (500) COLLATE SQL_Latin1_General_CP1_CI_AS NULL , [File Folder Count] [numeric] (18,0) NULL , [Library Folder Count] [numeric] (18,0) NULL ) ON [PRIMARY] GO
|
| Sign In·View Thread·PermaLink | 5.00/5 (1 vote) |
|
|
|
 |
|
|
Great idea but this is only half of the problem. You need to be able to make changes to the database, for instance:
UPDATE SOME_TABLE SET ARTIST_NAME = 'Dr. Dre' WHERE ARTIST_NAME = 'DR DRE'
And stuff like:
UPDATE SOME_TABLE SET GENRE = 'Alternative' WHERE ARTIST_NAME = 'Bush'
Then you upload your changes so you don't have to manually correct for all those retarded people in the world too slow to figure out how to use AUTOMATIC ID3 tagging.
Artist \ Album \ 00TrackNumber - TrackName It's that simple...
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
In my eyes it would be super a thing whom one also simply sql a file to provide could which dan into mysql the data base is imported... Best thanks David
Sorry but my english ist not good so i do this translation with google. I hope you understand wath i mean.
David from the country in that it milk of rivers gives and for chocolate beside the most exact clocks is appropriate. 
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Hi Alexander,
I am connecting to SQL Server on my own machine so I'm not sure why I would get this error. I am at work so can't spend too much time figuring out what is happening, but I have found that it occurs within the // Skip to block of code. Presumably the XMLTextReader is trying to retrieve the DTD and being blocked by our ISA Server. I wonder if there is a way to suppress the retrieval of the DTD as your code probably doesn't need it. Alternatively it must be possible to have sqlTunes use the ISA Client. I will look into this more when I have time, but would appreciate some comment from you on this. It seems that so many times when I download free/open source apps they don't work because of the ISA Server. While it may make our administrator's life easier, it isn't so much fun for users.
Cheers, Steve
|
| Sign In·View Thread·PermaLink | 2.00/5 (2 votes) |
|
|
|
 |
|
|
Alex, keen to use sqltunes, and have downloaded .net 2.0 and sql server for the purpose. However, depite the fact that the server seems to be running in config, I get this error:
under the default settings SQL server does not allow remote connections error: 40
Any clues?!
Cheers Scott
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
 |
|
|
Cheers Alexander- it turns out my problem was that I had (local) in the server field, when it should have been .\SQLExpress...!
Excellent app though- exactly what I was looking for.
Cheers Scott
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
I've been thinking for awhile that it would be great to have a SQL interface for for selecting/manipulating songs, e.g., create a playlist from the result of a query or set to 1 star everything that passes a certain where expression.
Do you have any plans to extend sqltunes to this sort of functionality?
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
You mean full synchronisation between iTunes and SQL Server? Interesting idea, I'll think about it...
Alexander Kojevnikov MCSD.NET Sydney, Australia
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Just what I was looking for. I am adding SQL - XML functionality and iTunesDB to SQL. Now I can finally restore iTunes and iPod libraries without losing any information.
Thanks, Patrick
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
| | |