Protobuf-net: the unofficial manual

It should be noted that some techniques that protobuf-net supports are only available when using the full .NET framework in full-trust mode. As mentioned here, for example, if you're using the precompiler then you won't be able to deserialize into private readonly fields. Personally I am using the full .NET framework, and I don't know what gotchas lie in wait in other environments.

At least when using the standard serialization mode (I've no idea about the others), you can write special parameterless methods marked with [ProtoBeforeSerialization], [ProtoAfterSerialization],[ProtoBeforeDeserialization], or [ProtoAfterDeserialization]. The most important one is probably [ProtoAfterDeserialization], which lets you make sure the object is valid or initialize any fields that weren't part of the data stream. For example:

[ProtoContract]
struct LatLonPoint
{
    [ProtoMember(1)] double X;
    [ProtoMember(2)] double Y;
    [ProtoAfterDeserialization] 
    void Validate() { 
        if (!(Math.Abs(X) <= 180 && Math.Abs(Y) <= 90)) 
            throw new FormatException("Invalid latitude or longitude enountered");
    }
}

• Protobuf-net serializes a collection using a "repeated" field (in protocol buffer lingo). Therefore, you should be able to safely change collection types between versions. For example, you can serialize a Foo[] and later deserialize it into a List.
• I don't know exactly how protobuf-net decides whether a given type is a collection. Wild guess: it might look for the IEnumerable<T> interface and an Add method?
• If you serialize a class that contains an empty but non-null collection, protobuf-net does not seem to distinguish an empty collection from null. When you deserialize the object, protobuf-net will leave the collection equal to null (or if your constructor created a collection, it will remain created).
• By default, protobuf-net "appends" to a collection rather than replacing one. So if you write a default constructor (without suppressing it) that initializes an int[] array to 10 items, and then deserialize a buffer with 10 items, you'll end up with an array of 20 items. Oops. Use the [ProtoMember(N, OverwriteList=true)] option to replace the existing list (if any) instead (or use SkipConstructor).
• protobuf-net automatically supports fields of type IEnumerable<T>, ICollection<T> or IList<T>. The list data type is not recorded, and during deserialization it always loads the data into a new List<T>.
• It also supports fields of type IDictionary<K,V>, and deserializes it as Dictionary<K,V>.

My impression is that less-than-standard collection interfaces such as IReadOnlyList<T> (or the custom read-only interface I use personally) are not supported. You can't define a surrogate for IReadOnlyList<T>, because the conversion to and from a surrogate requires a C# conversion operator, and the C# compiler prohibits you from defining a conversion to or from an interface.

Now, if you can modify the class that contains the IReadOnlyList<T> property, then you can create a private dummy property that exists only to help protobuf-net. The dummy property will be an IList<T> and have a [ProtoMember(N)] attribute while the original property does not. The dummy setter will then have to create a read-only wrapper around the List<T> it is given by protobuf-net. Here is an example:

[ProtoContract]
public class MyClass
{
    ...
    public IReadOnlyList<int> MyList;
        
    [ProtoMember(1, OverwriteList=true)]
    private IEnumerable<int> PB_MyList { 
        get { return MyList; } 
        set { MyList = ((IList<int>)value).AsReadOnly(); }
    }
    ...
}

Here, AsReadOnly() should create a read-only wrapper around the list (I leave this as an "exercise for the reader"). You may find that the OverwriteList option is necessary, otherwise protobuf-net apparently tries to call the PB_MyList getter and treat it as a IList<T> if possible.

If you can't modify the class that uses IReadOnlyList<T>, a last-resort approach is to create a surrogate type for the entire class.

• Protobuf.net's precompiler let's you use protobuf-net on platforms where run-time code generation or reflection are not available (e.g. .NET CF, iOS, Silverlight, and even .NET 1.1). It can also be used on the full .NET framework to avoid some run-time work.
• By default, protobuf-net will accept [XmlType] and [XmlElement(Order = N)] in place of [ProtoContract] and [ProtoMember(N)], which is nice if you're already using XML serialization or if you want to avoid explicitly depending on protobuf-net. Similarly, it accepts the WCF attributes [DataContract] and [DataMember(Order = N)]. The Order option is required for protobuf support.
• [XmlInclude] and [KnownType] cannot be used in place of [ProtoInclude] because they do not have an integer parameter to use as the tag number.
• Tag values must be positive. [ProtoMember(0)] is illegal.
• Very useful: print the result of RuntimeTypeModel.GetSchema(typeof(Root)) to find out what protocol buffers will be used to represent your data (e.g. RuntimeTypeModel.Default.GetSchema). Note: I assume this will be the actual schema used for serialization, but the documentation says merely that it will "Suggest a .proto definition".
• A constructor with a default argument like Constr(int x = 0) is not recognized as a parameterless constructor.
• The SkipConstructor and ImplicitFields options are not inherited, and probably other options are not inherited either. So, for example, if you use SkipConstructor on a base class, the constructor of the derived class is still called (and, by implication, the base class constructor).
• You may have noticed the "Visual Studio 2008 / 2010 support" download package on the download page, but what the heck is it for? I haven't tried it myself, but based on this blog entry, I suspect it's a tool for generating C# code from a .proto file automatically.
• Sometimes protobuf-net can serialize something but not deserialize it; be sure to test both directions.
• RuntimeTypeModel.DeepClone() is a handy way to test whether serialization and deserialization both work. This method will typically serialize the object and immediately deserialize it again.
• I suspect you can use the standard [NonSerialized] attribute on a field or property, as an alternative to [ProtoIgnore].
• When serializing a sub-object, protobuf-net can either write a length-prefixed buffer, or if the field that contains the sub-object has the [ProtoMember(N, DataFormat = DataFormat.Group)] option, it can use what Marc Gravell calls a "group-delimited" record, which avoids the overhead of measuring the record size in advance. Google calls this feature "groups", but it has deprecated the feature and AFAICT, removed any documentation about it that might have existed in the past.
• In [ProtoMember], the default value of IsRequired is false. I can only assume it means the same as required in a .proto file. I'm guessing that a required field is always written and that there will be some sort of exception if a required field is missing during deserialization.
• Protobuf-net supports Nullable<T>. I heard that a value of type int? or any other nullable type will simply not be written to the stream if it is null (therefore, I have no idea what happens if you use IsRequired=true on a nullable field--and that includes a reference-typed field.)
• Protobuf-net will (reasonably) refuse to serialize a property that does not have a setter, saying "Unable to apply changes to property". It will, however, serialize a property whose setter is private, and call that setter during deserialization.
• If you download the source code of protobuf-net via subversion, you'll have access to the Examples project, which demonstrates various ways to use the library.

• Call model.Add(typeof(C), false).SetSurrogate(typeof(S)) to establish S as a substitute for C during serialization. If a surrogate is used, all other options for C are ignored (the options for S are used instead). If not using a surrogate, you should probably use model.Add(typeof(C), true) instead although I'm uncertain what the true flag actually does, whether it's equivalent to [ProtoContract] or does something else.
• model[type].Add(7, "Foo").Add(5, "Bar") is equivalent to the attributes [ProtoMember(7)] on the field/property Foo, and [ProtoMember(5)] on the field/property Bar.
• model[type].Add("Fizz", "Buzz", ...) assigns tag numbers sequentially starting from 1 or, if some tag numbers exist already, from the highest existing tag number plus one. So if the type has no fields defined yet, Fizz will be #1 and Buzz will be #2.
• model[type].AddSubType(100, typeof(Derived)) is equivalent to [ProtoInclude(100, typeof(Derived))]. Example here.

• You can safely remove a serialized field (or a derived class) between versions of your software. Protobuf-net simply silently drops fields that are in the data stream but not in the class. Just be careful not to use the removed tag number again.
• You must not change the value of AsReference (or AsReferenceDefault) between versions.
• It's handy to save the results of GetSchema (mentioned above) to keep track of your old versions. You can then "diff" two schema versions to spot potential incompatibilities.
• Don't change the storage representation of an integer between versions; your numbers will be silently messed up if you migrate between TwosComplement and ZigZag. In theory I think protobuf-net could handle a change from FixedSize to TwosComplement or vice-versa, but I don't know if it actually can. Similarly, a change from FixedSize int to FixedSize long could work in theory, but I don't know about practice.
• On the other hand, you can increase the size of a non-FixedSize integer, e.g. byte to short or int to long, since the wire format is unchanged.
• The Google docs have more information relevant to versioning.

message NetObjectProxy {
  // for a tracked object, the key of the **first** 
  // time this object was seen
  optional int32 existingObjectKey = 1; 
  // for a tracked object, a **new** key, the first 
  // time this object is seen
  optional int32 newObjectKey = 2;
  // for dynamic typing, the key of the **first** time 
  // this type was seen
  optional int32 existingTypeKey = 3; 
  // for dynamic typing, a **new** key, the first time 
  // this type is seen
  optional int32 newTypeKey = 4;
  // for dynamic typing, the name of the type (only 
  // present along with newTypeKey) 
  optional string typeName = 8;
  // the new string/value (only present along with 
  // newObjectKey)
  optional bytes payload = 10;
}