Click here to Skip to main content
Click here to Skip to main content

Understanding the insides of the POP3 mail protocol: Part 2

, 9 Dec 2012 MIT
Rate this:
Please Sign up or sign in to vote.
This article describes the mail sending process for beginners of the mail protocol.

Introduction

Everyone who has a computer or mobile device has used mail. The mail system is an old, traditional simple protocol. The purpose of this article (part2) is to explore the insides of the POP3 protocol and show you how to implement it with C#.

Part 1 about SMTP protocol.
http://www.codeproject.com/Articles/399207/Understanding-the-insides-of-the-SMTP-mail-protoco
Part 3 about IMAP protocol.
http://www.codeproject.com/Articles/411018/Understanding-the-insides-of-the-IMAP-mail-protoco

You can get the complete library that includes Mail, Twitter, Facebook, Dropbox, Windows Live at http://higlabo.codeplex.com/.

POP3

You can select two protocols to receive mail from a mailbox. This article describes the POP3 protocol. Here is the basic flow to receive mail:

  • Open connection
  • Authenticate
  • List
  • Retr
  • Quit

Open connection

See Open connection section of this post.
http://www.codeproject.com/Articles/399207/Understanding-the-insides-of-the-SMTP-mail-protoco

At first, you must authenticate the mailbox with your username and password. Here is a pop authenticate flow diagram:

Pop3Client.cs
public Boolean AuthenticateByPop()
{
    String s = "";
    if (this.EnsureOpen() == Pop3ConnectionState.Connected)
    {
        s = this.Execute("user " + this.UserName, false);
        if (s.StartsWith("+OK") == true)
        {
            s = this.Execute("pass " + this.Password, false);
            if (s.StartsWith("+OK") == true)
            {
                this._State = Pop3ConnectionState.Authenticated;
            }
        }
    }
    return this._State == Pop3ConnectionState.Authenticated;
}

This is the response text of the POP3 protocol:

+OK send PASS please

This is the response text when failed:

-ERR [AUTH] Username and password not accepted.

Single line response format is here:

When success:

+OK[whitespace][message]

When failed:

-ERR[whitespace][message]

POP3 authentication is pretty easy, but low security. To enhance the security level, you can use A-POP authentication.

Pop3Client.cs
public Boolean AuthenticateByAPop()
{
    String s = "";
    String TimeStamp = "";
    Int32 StartIndex = 0;
    Int32 EndIndex = 0;

    if (this.EnsureOpen() == Pop3ConnectionState.Connected)
    {
        s = this.Execute("user " + this.UserName, false);
        if (s.StartsWith("+OK") == true)
        {
            if (s.IndexOf("<") > -1 &&
                s.IndexOf(">") > -1)
            {
                StartIndex = s.IndexOf("<");
                EndIndex = s.IndexOf(">");
                TimeStamp = s.Substring(StartIndex, EndIndex - StartIndex + 1);
                s = this.Execute("pass " + MailParser.ToMd5DigestString(TimeStamp + this.Password), false);
                if (s.StartsWith("+OK") == true)
                {
                    this._State = Pop3ConnectionState.Authenticated;
                }
            }
        }
    }
    return this._State == Pop3ConnectionState.Authenticated;
}

Password security

POP3 authentication is low-security since the username and password are send as raw text into the network. Malicious software or people can sniff your password and use it maliciously. A-POP is different from POP3 since the sent password is encoded by MD5Digest with the challenge text received from the server. The server compares the saved challenge text and text received from the client. There is no way to receive the actual password text. So A-POP is secure than POP authentication.

Get mail list

After authentication, you can get the mail list using the LIST command. The response text of the LIST command is as below.

+OK 250 messages (338834 bytes)
1 2762
2 1704
3 2259
4 1748
5 1799
6 1802
7 1109
8 1701
9 1075
10 1130
...
245 1310
246 1725
247 1092
248 1237
249 1161
250 1099
.

The format is like below:

+OK [message]
[mail index][white space][mail size]
[mail index][white space][mail size]
...repeat...
[mail index][white space][mail size]
[period]

You can get it by calling the ExecuteList method of the Pop3Client class:

public List<ListCommandResult> ExecuteList()
{
    ListCommand cm = new ListCommand();
    List<ListCommandResult> l = new List<ListCommandResult>();
    StringReader sr = null;
    String s = "";
    String line = "";

    this.CheckAuthenticate();
    s = this.Execute(cm);
    this.CheckResponseError(s);
            
    sr = new StringReader(s);
    while (sr.Peek() > -1)
    {
        line = sr.ReadLine();
        if (line == ".")
        { break; }
        if (line.StartsWith("+OK", StringComparison.OrdinalIgnoreCase) == true)
        { continue; }

        l.Add(new ListCommandResult(line));
    }
    return l;
}

This result is represented by the ListCommandResult class.

Get message

You can get the actual mail message data by calling the GetMessage method of the Pop3Client class. Here is some sample code to receive a mail message:

private static void Pop3MailReceive()
{
    MailMessage mg = null;

    using (Pop3Client cl = new Pop3Client("pop.gmail.com"))
    {
        cl.Port = 995;
        cl.UserName = "xxxxx";
        cl.Password = "yyyyy";
        cl.Ssl = true;
        cl.AuthenticateMode = Pop3AuthenticateMode.Auto;
        var bl = cl.Authenticate();
        if (bl == true)
        {
            var l = cl.ExecuteList();
            for (int i = 0; i < l.Count; i++)
            {
                mg = cl.GetMessage(l[i].MailIndex);
            }
        }
    }
}

You must pass an index to the GetMessage method larger than 1. Please take care of that the first value is 1, not zero.

The MailMessage class schema is shown here:

The MailMessage class inherits from InternetTextMessage.

You can get subject, body text using this sample code:

String subject = mg.Subject;
String body = mg.BodyText;

You can get raw text using the Data property that is defined on the InternetTextMessage class.

String rawText = mg.Data;

Attached file

Mail sometimes includes an attached file. You can get the attached files using the Contents property of the MailMessage class.

var mg = cl.GetMessage(1);
foreach (var ct in mg.Contents)
{
    ct.DecodeData("C:\\" + ct.FileName);
}

You can save attached files by calling the DecodeData method of the MailContent object.

In rare cases, MailMessage itself is an attached file. You can confirm whether MailMessage is an attached file or not by checking the IsAttachment property of the MailMessage object.

var mg = cl.GetMessage(1);
if (mg.IsAttachment == true)
{
    mg.DecodeData("C:\\MyFile.png");
}

HTML mail

HTML mail is a text mail. You can get HTML text using the BodyText property of the MailMessage or MailContent object.

String htmlText = mg.BodyText;
foreach (var ct in mg.Contents)
{
    if (ct.IsHtml == false) { continue; }
    String htmlText1 = ct.BodyText;
}

To show HTML to your UI component, you must set this htmlText to your control. In a Windows Forms or WPF application, use the browser control to show it. In a web application, you can select some control, HtmlGenericControl, Label control, ...etc. Here is some sample code for a web application:

protected void Page_Load(object sender, EventArgs e)
{
    MailMessage mg = null;
    String htmlText = "";

    using (Pop3Client cl = new Pop3Client("pop.gmail.com"))
    {
        cl.Port = 995;
        cl.UserName = "xxxxx";
        cl.Password = "yyyyy";
        cl.Ssl = true;
        cl.AuthenticateMode = Pop3AuthenticateMode.Auto;
        if (cl.Authenticate() == false) { return; }
        if (mg.IsHtml == true)
        {
            htmlText = mg.BodyText;
        }
        else 
        {
            foreach (var ct in mg.Contents)
            {
                if (ct.IsHtml == true) 
                {
                    htmlText = ct.BodyText;
                    break; 
                }
            }
        }
    }
    this.Label1.Text = htmlText;
}

.eml file

Outlook and other mail client programs would create a .eml file. Actually, these files are simple text files, like below:

Return-Path: <xxxxx@gmail.com>
Received: from  (bf240.xxx.xxxx.com. [61.197.23.240])
        by mx.google.com with ESMTPS id pb4sm20464671pbc.55.2012.05.13.17.56.12
        (version=SSLv3 cipher=OTHER);
        Sun, 13 May 2012 17:56:13 -0700 (PDT)
Message-ID: <4fc05e2d.e4a9440a.302b.ffffe399@mx.google.com>
Date: Mon, 14 May 2012 00:56:10 +0900
From: xxxxx@gmail.com
Subject: Test
Content-Transfer-Encoding: Quoted-Printable
Content-Disposition: inline
Mime-Version: 1.0
Reply-To: xxxxx@hotmail.com
X-Priority: 3
To: yyyyy@gmail.com
Content-Type: text/plain; charset="iso-8859-1"

=E4=C4=F6=D6=FC=DC=DF


.

You can create a MailMessage object from text data.

String text = File.ReadAllText("C:\\MyMail.eml");
var mg = new MailMessage(text);

Delete mail

Here is the delete process in POP3:

You can see that the response text of the DELE command is like below.

+OK marked for deletion

The Dele command marks the mail that you indicate as candidate for deletion. These marked mails will be deleted when you send the quit command. You can delete a mail by using the DeleteMail method of the Pop3Client object.

protected void Button1_Click(object sender, EventArgs e)
{
    MailMessage mg = null;
    String htmlText = "";

    using (Pop3Client cl = new Pop3Client("pop.gmail.com"))
    {
        cl.Port = 995;
        cl.UserName = "xxxxx";
        cl.Password = "yyyyy";
        cl.Ssl = true;
        cl.AuthenticateMode = Pop3AuthenticateMode.Auto;
        
        cl.DeleteMail(1, 2, 3);
    }
}

There is no authenticate required since all processes (open connection, authenticate, dele, quit) will automatically execute inside the DeleteMail method. Here is an implementation of the DeleteMail method:

public Boolean DeleteMail(params Int64[] mailIndex)
{
    DeleCommand cm = null;
    String s = "";

    if (this.EnsureOpen() == Pop3ConnectionState.Disconnected) { return false; }
    if (this.Authenticate() == false) { return false; }
    for (int i = 0; i < mailIndex.Length; i++)
    {
        cm = new DeleCommand(mailIndex[i]);
        s = this.Execute(cm);
        if (MailParser.IsResponseOk(s) == false) { return false; }
    }
    this.ExecuteQuit();
    return true;
}

You can see the open connection, authenticate, send dele command, and finally send quit command inside of the DeleteMail method.

Mail format

Here is the actual raw text data of a POP3 message:

Return-Path: <xxxxx@gmail.com>
Received: from  (bf240.xxx.xxxx.com. [61.197.23.240])
        by mx.google.com with ESMTPS id pb4sm20464671pbc.55.2012.05.13.17.56.12
        (version=SSLv3 cipher=OTHER);
        Sun, 13 May 2012 17:56:13 -0700 (PDT)
Message-ID: <4fc05e2d.e4a9440a.302b.ffffe399@mx.google.com>
Date: Mon, 14 May 2012 00:56:10 +0900
From: xxxxx@gmail.com
Subject: Test
Content-Transfer-Encoding: Quoted-Printable
Content-Disposition: inline
Mime-Version: 1.0
Reply-To: xxxxx@hotmail.com
X-Priority: 3
To: yyyyy@gmail.com
Content-Type: text/plain; charset="iso-8859-1"

=E4=C4=F6=D6=FC=DC=DF


.

Message has two parts: header and body. Red is header and blue is body.

Header and body are separated by a blank line. Header has a list of values with a key value pair.

[key][colon][value]

The multi-line format is like below:

[key][colon][value]
[tab or white space][value]
...repeat...
[tab or white space][value]

The characters you can use in message body is 7bit char only. If you want to send "ありがとうございます" (it is a Japanese character that means thank you), you must encode it using Quoted-Printable or Base64 encoding. The mail format has been very complex since its history.

Important RCFs are 822, 2045-2049, 2231, 2282, 2407. Please check more details if you want to know.

Manage read or unread with the UIDL command

POP3 does not have functionality to manage read state. You must use your own implementation to manage read or not. You can use the UIDL command to achieve read state management. Here is the response text of a UIDL command:

+OK
1 GmailId136106770caf88db
2 GmailId136106acda5c2ff1
3 GmailId136106ae7f1b631b
4 GmailId136106b691ad4a79
5 GmailId136106ba4df09d7f
6 GmailId136106bdfbd92cdb
7 GmailId136106c5168c87fe
...
245 GmailId13610723f4c51313
246 GmailId13610724acde6b5b
247 GmailId13610724dc2d28b0
248 GmailId136107261a4d3b0c
249 GmailId1361072652837e0f
250 GmailId136107269afc66ae
.

This format is like below:

+OK
[mail index][white space][unique id]
...repeat
[mail index][white space][unique id]
[period]

The UidlCommandResult class represents this response text:

To manage read state, you save this UID list for the next mail. I'll show you the concept code here:

private static void Pop3MailReceiveUnRead()
{
    MailMessage mg = null;

    using (Pop3Client cl = new Pop3Client("pop.gmail.com"))
    {
        cl.Port = 995;
        cl.UserName = "xxxxx";
        cl.Password = "yyyyy";
        cl.Ssl = true;
        cl.AuthenticateMode = Pop3AuthenticateMode.Auto;
        var bl = cl.Authenticate();
        if (bl == true)
        {
            var ul = cl.ExecuteUidl();
            String path = "C:\\MailUidl.txt";
            List<String> l = new List<String>();
            //Get read mail index list from file
            if (File.Exists(path) == true)
            {
                l.AddRange(File.ReadAllLines(path));
            }
            List<UidlCommandResult> unreadList = new List<UidlCommandResult>();
            for (int i = 0; i < ul.Count; i++)
            {
                if (l.Contains(ul[i].Uid) == true) { continue; }
                unreadList.Add(ul[i]);
            }
            //Save UIDL data to text file
            StringBuilder sb = new StringBuilder(ul.Count * 10);
            for (int i = 0; i < ul.Count; i++)
            {
                sb.AppendLine(ul[i].Uid);
            }
            File.WriteAllText(path, sb.ToString());
        }
    }
}

You can save UIDL data to a database or XML file depending on your requirements.

Forward Mail by SMTP

If you want to transfer your received mail, you can use CreateSmtpMessage method of MailMessage object.
Here is a sample code to do it

private static void Pop3MailReceiveUnRead()
{
    MailMessage mg = null;

    using (Pop3Client cl = new Pop3Client("pop.gmail.com"))
    {
        cl.Port = 995;
        cl.UserName = "xxxxx";
        cl.Password = "yyyyy";
        cl.Ssl = true;
        cl.AuthenticateMode = Pop3AuthenticateMode.Auto;
        var bl = cl.Authenticate();
        if (bl == true)
        {
            mg = cl.GetMessage(1);
            SmtpMessage smg = mg.CreateSmtpMessage();
            // you can send smg by SmtpClient object
        }
    }
}

And you can send SmtpMessage class by SmtpClient class.
You can learn how to send a mail by SmtpClient class by this article: Understanding the insides of the SMTP Mail protocol: Part 1.



Next article for IMAP
Understanding the insides of the IMAP Mail protocol: Part 3.

History

  • 2012/06/14: First post.
  • 2012/07/06: Modify source code and article
  • 2012/07/24: Modify article. Add CreateSmtpMessage section.
  • 2012/12/10: Modify article.

License

This article, along with any associated source code and files, is licensed under The MIT License

Share

About the Author

Higty
Web Developer
Japan Japan
I'm Working at Software Company in Tokyo.

Comments and Discussions

 
QuestionProblem POP3 saving email attachment PinmemberMember 305716310-Sep-14 23:17 
Questionsmtp [modified] PinmemberHAMZEI2-Nov-13 20:25 
AnswerRe: smtp PinmemberHigty5-Nov-13 14:35 
QuestionUnsure of how to resolve this.EnsureOpen() [modified] PinmemberTony Girgenti28-Sep-13 9:59 
AnswerRe: Unsure of how to resolve this.EnsureOpen() PinmemberHigty4-Oct-13 2:05 
GeneralRe: Unsure of how to resolve this.EnsureOpen() PinmemberTony Girgenti4-Oct-13 13:06 
GeneralRe: Unsure of how to resolve this.EnsureOpen() PinmemberHigty8-Oct-13 13:49 
GeneralRe: Unsure of how to resolve this.EnsureOpen() PinmemberTony Girgenti9-Oct-13 0:57 
GeneralRe: Unsure of how to resolve this.EnsureOpen() PinmemberHigty15-Oct-13 21:34 
GeneralRe: Unsure of how to resolve this.EnsureOpen() PinmemberTony Girgenti16-Oct-13 5:07 
GeneralRe: Unsure of how to resolve this.EnsureOpen() PinmemberHigty16-Oct-13 14:03 
QuestionHelp Pinmemberevry1falls28-Jan-13 5:44 
AnswerRe: Help PinmemberHigty7-May-13 0:01 
GeneralRe: Help Pinmemberevry1falls17-May-13 4:55 
GeneralRe: Help PinmemberHigty20-May-13 14:06 
GeneralRe: Help Pinmemberevry1falls16-Jun-13 14:56 
QuestionQuestion PinmemberSolon Hinson19-Nov-12 11:50 
AnswerRe: Question PinmemberHigty21-Nov-12 21:20 
QuestionCan i remove attachments from email Pinmemberofiroth7-Aug-12 20:25 
AnswerRe: Can i remove attachments from email PinmemberHigty21-Nov-12 21:14 
QuestionVote of 5 PinmemberGanesanSenthilvel5-Jul-12 18:48 
AnswerRe: Vote of 5 PinmemberHigty6-Jul-12 0:20 
GeneralMy vote of 5 PinmemberRahul Rajat Singh27-Jun-12 22:43 
GeneralRe: My vote of 5 PinmemberHigty4-Jul-12 2:27 
GeneralMy vote of 5 Pinmemberacarpio197526-Jun-12 22:05 
GeneralRe: My vote of 5 PinmemberHigty4-Jul-12 2:26 
GeneralMy vote of 5 PinmemberNicolas Dorier22-Jun-12 22:09 
Thanks for the lib Smile | :)
GeneralRe: My vote of 5 PinmemberHigty24-Jun-12 13:16 
Questionmail receive hung up. Any idea why ? Pinmemberyasuo yamamoto21-Jun-12 17:42 
AnswerRe: mail receive hung up. Any idea why ? PinmemberHigty21-Jun-12 18:46 
GeneralRe: mail receive hung up. Any idea why ? Pinmemberyasuo yamamoto24-Jun-12 22:56 
GeneralRe: mail receive hung up. Any idea why ? PinmemberHigty25-Jun-12 15:11 
QuestionNice job on this one and the SMTP protocals... One question.. PinmemberKinStephen21-Jun-12 9:28 
AnswerRe: Nice job on this one and the SMTP protocals... One question.. PinmemberHigty21-Jun-12 18:39 
GeneralThis is great. Pinmemberyasuo yamamoto21-Jun-12 2:48 
GeneralRe: This is great. PinmemberHigty21-Jun-12 3:51 
GeneralMy vote of 5 Pinmemberburak29914-Jun-12 12:18 
GeneralRe: My vote of 5 PinmemberHigty17-Jun-12 0:16 
GeneralMy vote of 5 PinmemberSergio Andrés Gutiérrez Rojas14-Jun-12 11:30 
GeneralRe: My vote of 5 PinmemberHigty17-Jun-12 0:14 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web01 | 2.8.141029.1 | Last Updated 10 Dec 2012
Article Copyright 2012 by Higty
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid