You want to help? Excellent! ----------------------------- Barry currently only supports about 3 database record formats. A database dump reveals over 50 different databases types, and you can help code these. Use the following command to dump the Database Database to stdout: ./btool -t Pick a database you are interested in, and run the following command to dump the protocol to stdout: ./btool -v -d "Memos" Protocol Documentation: ----------------------- Once you have the raw protocol dump, you need to reverse engineer it. There are a number of places you can find documentation on this. The original source that I started with was the Cassis project docs. You can find them here: http://off.net/cassis/protocol-description.html While these docs are quite helpful, they only cover the serial protocol. The USB protocol is more data-block based. There are no checksums, since USB provides a reliable data transfer. The common protocol headers are struct based, until you get to the variable sized data. (This is also struct based, but requires pointer arithmetic.) See the following Barry source file for protocol structure information: src/protostructs.h Protostructs.h -------------- These structures are all low level in nature, and MUST be packed. You will notice that every struct in this header is flagged with the gcc extension "__attribute__ ((packed))", which makes it useful to apply structure to raw protocol data. You can build / deconstruct a protocol packet in this manner: Imagine you have a packet of raw data in the following buffer: char buff[4096]; You can apply the Packet struct to it directly: struct Packet *packet = (struct Packet *) buff; Looking at the structure, this gives you a lot of fixed-position data: /////////////////////////////////////////////////////////////////////////////// // Main packet struct struct Packet { uint16_t socket; // socket ID... 0 is always there uint16_t size; // total size of data packet uint8_t command; union PacketData { SocketCommand socket; SequenceCommand sequence; ModeSelectCommand mode; DBAccess db; uint8_t raw[1]; } __attribute__ ((packed)) u; } __attribute__ ((packed)); In the case of database records, you are interested in the DBAccess union member: /////////////////////////////////////////////////////////////////////////////// // Database access command structure // even fragmented packets have a tableCmd struct DBAccess { uint8_t tableCmd; union DBData { DBCommand command; DBResponse response; CommandTableField table[1]; uint8_t return_code; uint8_t fragment[1]; } __attribute__ ((packed)) u; } __attribute__ ((packed)); When downloading database records, they all arrive in a DBResponse packet. struct DBResponse { uint8_t operation; union Parameters { DBR_OldTaggedRecord tagged; DBR_OldDBDBRecord old_dbdb; DBR_DBDBRecord dbdb; } __attribute__ ((packed)) u; } __attribute__ ((packed)); Depending on the command you use, the format of the record will change. Also, newer versions of Blackberries have new "GET" commands that return a slightly different record header. Barry currenly uses the "old" command, which is supported by all USB Blackberries tested so far, so it has the greatest compatibility. Some database records are tagged with a unique ID. These databases include the Contact records, Service Book records, and Calendar records. This allows for intelligent syncing, as Barry can reference, update, and delete records individually instead of a group. struct DBR_OldTaggedRecord { uint8_t unknown; uint16_t index; uint32_t uniqueId; uint8_t unknown2; union TaggedData { CommonField field[1]; } __attribute__ ((packed)) u; } __attribute__ ((packed)); This is enough information to get you to the data you need to reverse engineer. That data will be located at: packet->u.db.u.response.u.tagged.u.field[0] So, taking an Address Book record as an example: 00000000: 06 00 37 00 40 03 44 00 08 00 80 82 b1 22 00 06 ..7.@.D......".. 00000010: 00 20 43 68 72 69 73 00 05 00 20 46 72 65 79 00 . Chris... Frey. 00000020: 14 00 01 63 64 66 72 65 79 40 6e 65 74 64 69 72 ...cdfrey@netdir 00000030: 65 63 74 2e 63 61 00 ect.ca. We now know: 06 00 - packet->socket 37 00 - packet->size 40 - packet->command 03 - packet->u.db.tableCmd 44 - packet->u.db.u.response.operation 00 - packet->u.db.u.response.u.tagged.unknown 08 00 - packet->u.db.u.response.u.tagged.index 80 82 b1 22 - packet->u.db.u.response.u.tagged.uniqueId 00 - packet->u.db.u.response.u.tagged.unknown2 06 00 - packet->u.db.u.response.u.tagged.u.field[0].size 20 - packet->u.db.u.response.u.tagged.u.field[0].type ... - continued data from sequential CommonField packets Note that CommonField packets are variable size, so you can only use field[0] the first time. You'll need to do pointer arithmetic to calculate the start of the next CommonField. This is a common theme in the database records. There is a static header (eg. DBR_OldTaggedRecord) followed by a number of fields, which each contains a common field header (eg. CommonField), followed by a variable sized block of data (CommonField's u.raw field). The next field starts size bytes later, and so on. This should give you enough information to begin hacking on additional database records. Once you have a "Record" header struct, you are ready to begin parsing into an API class. Record API Classes ------------------ Record API Classes are intended to be consistent, C++-friendly structures for storing the parsed data. You can find them in: src/record.h These classes may be broken out into multiple files someday. The requirements of these classes are as follows: - no virtual functions - must be valid STL container items - generally contain all public data - provide member functions for: - dumping to ostream - parsing downloaded data blocks from the BlackBerry - building data blocks for the BlackBerry for uploads - must have absolutely no low level protocol contamination The final point deserves some commentary. When using gcc-specific extensions like __attribute__, and when using pointer arithmetic and pointer casts across types, as is necessary in some of the low level parsing code, special compiler flags are required. Specifically, g++ requires -fno-strict-aliasing to stop it from optimizing away some of the pointer casts and data accesses. This is not desireable in an end-user application, so care must be taken to hide all low level parsing in the library, while exposing the record class structures and member function APIs to the application. Therefore src/record.h (or any new record header you add) must not include src/protostructs.h. Your .cc code may include protostructs.h, but not the header, which will be used by the application. Good luck, and happy hacking! October 2006