[ANN] HID report descriptor library and tool

[Nikolai Kondrashov]

Hello everyone,

I would like to announce a HID report descriptor library and tool I'm making
in my spare time, as a part of the DIGImend project [1]. It is called
simply "hidrd".

It is aimed to replace the HID Descriptor Tool [2], produced by the USB
Implementers Forum, and address its shortcomings. To me these are mainly the
inability to read native descriptor format, GUI-only interface, and
instability, plus it is Windows-only (although it runs under WINE).

The ultimate goal is to make a command-line tool which could be used to
conveniently make your own and modify existing descriptors. So the idea is
to implement conversion to/from some format more suitable for human
consumption and editing. XML was chosen for a start. Yeah, I know, some
people consider it ugly and not that convenient, but it was the easiest way
to me.

The tool is called "hidrd-convert" and currently can only convert from the
native format to XML. However it could already be useful for a developer
wishing to better understand a particular descriptor, or HID report
descriptors as a whole. The next major release (0.2) will include conversion
from XML to the native format.

Here is how the output looks for a Microsoft's Natural Ergonomic Keyboard:

<?xml version="1.0"?>
<descriptor xmlns="http://digimend.sourceforge.net"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://digimend.sourceforge.net hidrd.xsd">
<usage_page>desktop<!-- Generic desktop controls --></usage_page>
<usage>desktop_keyboard<!-- Keyboard (06, top-level) --></usage>
<COLLECTION type="application">
<usage_page>led<!-- LEDs --></usage_page>
<usage_minimum>01<!-- Top-level --></usage_minimum>
<usage_maximum>03<!-- Top-level --></usage_maximum>
<logical_minimum>0</logical_minimum>
<logical_maximum>1</logical_maximum>
<report_size>1</report_size>
<report_count>3</report_count>
<output>
<variable/>
</output>
<usage>4B</usage>
<report_count>1</report_count>
<output>
<variable/>
</output>
<report_count>4</report_count>
<output>
<constant/>
</output>
<usage_page>keyboard<!-- Keyboard/keypad --></usage_page>
<usage_minimum>keyboard_kb_leftcontrol<!-- Keyboard left control (E0) --></usage_minimum>
<usage_maximum>keyboard_kb_right_gui<!-- Keyboard right GUI (E7) --></usage_maximum>
<report_count>8</report_count>
<input>
<variable/>
</input>
<report_size>8</report_size>
<report_count>1</report_count>
<input>
<constant/>
</input>
<usage_minimum>keyboard_none<!-- No event (00) --></usage_minimum>
<usage_maximum>keyboard_kb_lang2<!-- Keyboard LANG2 (91) --></usage_maximum>
<logical_maximum>255</logical_maximum>
<report_count>6</report_count>
<input/>
</COLLECTION>
</descriptor>

This is produced by running the hidrd-convert tool like this:
hidrd-convert -o xml descriptor.bin descriptor.xml

I can show you some a bit more interesting examples (including a Genius
tablet and the PS3 gamepad) on request (since they are bigger).

The hidrd release 0.1.1 could be downloaded here:
https://sourceforge.net/projects/digimend/files/hidrd/0.1/hidrd-0.1.1.tar.gz/download
You can also visit the hidrd section on the DIGImend project page [3].

Any comments and feedback are welcome, of course :)

Thank you for you attention.

Sincerely,
Nick

[1] http://digimend.sourceforge.net/
[2] http://www.usb.org/developers/hidpage/#Descriptor_Tool
[3] http://digimend.sourceforge.net/#proj-hidrd

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Sarah Sharp]

Post by Nikolai Kondrashov
Hello everyone,
I would like to announce a HID report descriptor library and tool I'm making
in my spare time, as a part of the DIGImend project [1]. It is called
simply "hidrd".
It is aimed to replace the HID Descriptor Tool [2], produced by the USB
Implementers Forum, and address its shortcomings. To me these are mainly the
inability to read native descriptor format, GUI-only interface, and
instability, plus it is Windows-only (although it runs under WINE).
The ultimate goal is to make a command-line tool which could be used to
conveniently make your own and modify existing descriptors. So the idea is
to implement conversion to/from some format more suitable for human
consumption and editing. XML was chosen for a start. Yeah, I know, some
people consider it ugly and not that convenient, but it was the easiest way
to me.
The tool is called "hidrd-convert" and currently can only convert from the
native format to XML. However it could already be useful for a developer
wishing to better understand a particular descriptor, or HID report
descriptors as a whole. The next major release (0.2) will include conversion
from XML to the native format.
<?xml version="1.0"?>
<descriptor xmlns="http://digimend.sourceforge.net"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://digimend.sourceforge.net hidrd.xsd">
<usage_page>desktop<!-- Generic desktop controls --></usage_page>
<usage>desktop_keyboard<!-- Keyboard (06, top-level) --></usage>
<COLLECTION type="application">
<usage_page>led<!-- LEDs --></usage_page>
<usage_minimum>01<!-- Top-level --></usage_minimum>
<usage_maximum>03<!-- Top-level --></usage_maximum>
<logical_minimum>0</logical_minimum>
<logical_maximum>1</logical_maximum>
<report_size>1</report_size>
<report_count>3</report_count>
<output>
<variable/>
</output>
<usage>4B</usage>
<report_count>1</report_count>
<output>
<variable/>
</output>
<report_count>4</report_count>
<output>
<constant/>
</output>
<usage_page>keyboard<!-- Keyboard/keypad --></usage_page>
<usage_minimum>keyboard_kb_leftcontrol<!-- Keyboard left control (E0) --></usage_minimum>
<usage_maximum>keyboard_kb_right_gui<!-- Keyboard right GUI (E7) --></usage_maximum>
<report_count>8</report_count>
<input>
<variable/>
</input>
<report_size>8</report_size>
<report_count>1</report_count>
<input>
<constant/>
</input>
<usage_minimum>keyboard_none<!-- No event (00) --></usage_minimum>
<usage_maximum>keyboard_kb_lang2<!-- Keyboard LANG2 (91) --></usage_maximum>
<logical_maximum>255</logical_maximum>
<report_count>6</report_count>
<input/>
</COLLECTION>
</descriptor>
hidrd-convert -o xml descriptor.bin descriptor.xml
I can show you some a bit more interesting examples (including a Genius
tablet and the PS3 gamepad) on request (since they are bigger).
https://sourceforge.net/projects/digimend/files/hidrd/0.1/hidrd-0.1.1.tar.gz/download
You can also visit the hidrd section on the DIGImend project page [3].
Any comments and feedback are welcome, of course :)

Can you make a plain text output, like what `lsusb -v` produces?
XML gives me the heebie jeebies.

Sarah Sharp
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Nikolai Kondrashov]

Post by Sarah Sharp
Can you make a plain text output, like what `lsusb -v` produces?
XML gives me the heebie jeebies.

Yeah, sure. Do you need precisely the lsusb output? I have the specification
example output in the plan. Like this one:

Usage Page (Generic Desktop),
Usage (Mouse),
Collection (Application),
Usage (Pointer),
Collection (Physical),
Report Count (3),
Report Size (1),
Usage Page (Buttons),
Usage Minimum (1),
Usage Maximum (3),
Logical Minimum (0),
Logical Maximum (1),
Input (Data, Variable, Absolute),
End Collection,
End Collection

However, I'm not sure if it will be possible to convert such format back,
maybe with some corrections. XML was chosen because it is easy to convert
back and forth (parser is ready and it has a validator) and that's the main
purpose of the utility.

Does anybody else need the text output more than the XML output?

Sincerely,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Xiaofan Chen]

Post by Nikolai Kondrashov
I would like to announce a HID report descriptor library and tool I'm making
in my spare time, as a part of the DIGImend project [1]. It is called
simply "hidrd".

I see that you also has the hidrd-dump developed along with hidrd-convert.
Currently hidrd-dump (report descriptor dump utility) does not work too
well. If possible, combining these two so that we can get the report
descriptor of an existing device and convert it to the XML or other
format.

[Nikolai Kondrashov]

Post by Xiaofan Chen
I see that you also has the hidrd-dump developed along with hidrd-convert.
Currently hidrd-dump (report descriptor dump utility) does not work too
well.

Could you please explain what's wrong with it? Thanks :) Or do you mean the
problem is that it doesn't dump the device descriptors itself?

Post by Xiaofan Chen
If possible, combining these two so that we can get the report
descriptor of an existing device and convert it to the XML or other
format.

Well, I don't think a separate program is really needed for this.
You could write a simple script to do it, like this one:

#!/bin/sh
hidrd-dump "$@" | tr -d ' \n' | xxd -r -p | hidrd-convert -o xml

xxd is packaged with vim, if you don't have it.

Sincerely,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Xiaofan Chen]

Post by Nikolai Kondrashov

Post by Xiaofan Chen
I see that you also has the hidrd-dump developed along with hidrd-convert.
Currently hidrd-dump (report descriptor dump utility) does not work too
well.

Could you please explain what's wrong with it? Thanks :) Or do you mean the
problem is that it doesn't dump the device descriptors itself?

It is not working well basically for some device.
I think it is based on libhid which does not detach the 2nd interface
if the device is a USB composite device.

Eg: I have a few HID device now.

***@ubuntu64:~/Desktop$ lsusb
Bus 001 Device 011: ID 04d8:0033 Microchip Technology, Inc. (PICkit 2)
Bus 001 Device 008: ID 046d:c216 Logitech, Inc. Dual Action Gamepad
Bus 002 Device 007: ID 046d:c054 Logitech, Inc. (Mouse)
Bus 002 Device 003: ID 04f2:0760 Chicony Electronics Co., Ltd (Acer Keyboard).

The first three are good. The last one is a USB composite
device and only one interface is dumped.

***@ubuntu64:~/Desktop$ hidrd-dump 046d c216
05 01 09 04 A1 01 A1 02 15 00 26 FF 00 35 00 46
FF 00 75 08 95 04 09 30 09 31 09 32 09 35 81 02
25 07 46 3B 01 75 04 95 01 65 14 09 39 81 42 65
00 25 01 45 01 75 01 95 0C 05 09 19 01 29 0C 81
02 06 00 FF 75 01 95 10 25 01 45 01 09 01 81 02
C0 A1 02 26 FF 00 46 FF 00 75 08 95 07 09 02 91
02 C0 A1 02 26 FF 00 46 FF 00 75 08 95 05 06 00
FF 09 01 B1 02 C0 C0

***@ubuntu64:~/Desktop$ hidrd-dump 04d8 0033
06 00 FF 09 01 A1 01 19 01 29 40 15 00 26 FF 00
75 08 95 40 81 02 19 01 29 40 91 02 C0

***@ubuntu64:~/Desktop$ hidrd-dump 046d c054
05 01 09 02 A1 01 09 01 A1 00 05 09 19 01 29 08
15 00 25 01 75 01 95 08 81 02 05 01 16 01 F8 26
FF 07 75 0C 95 02 09 30 09 31 81 06 15 81 25 7F
75 08 95 01 09 38 81 06 05 0C 0A 38 02 95 01 81
06 C0 C0

***@ubuntu64:~/Desktop$ hidrd-dump 04f2 0760
05 01 09 06 A1 01 05 07 19 E0 29 E7 15 00 25 01
75 01 95 08 81 02 95 01 75 08 81 01 95 03 75 01
05 08 19 01 29 03 91 02 95 05 75 01 91 01 95 06
75 08 15 00 26 FF 00 05 07 19 00 2A FF 00 81 00
C0

The last one should be something like this.
http://www.microchip.com/forums/fb.aspx?m=377479

Post by Nikolai Kondrashov

Post by Xiaofan Chen
If possible, combining these two so that we can get the report
descriptor of an existing device and convert it to the XML or other
format.

Well, I don't think a separate program is really needed for this.
#!/bin/sh
xxd is packaged with vim, if you don't have it.

Thanks. This seems to work.

The following is for PICkit 2, an HID programmer from Microchip
(http://www.microchip.com/pickit2).
***@ubuntu64:~/Desktop$ dumphidrd.sh 04d8 0033
<?xml version="1.0"?>
<descriptor xmlns="http://digimend.sourceforge.net"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://digimend.sourceforge.net hidrd.xsd">
<usage_page>FF00<!-- Vendor-defined --></usage_page>
<usage>01<!-- Top-level --></usage>
<COLLECTION type="application">
<usage_minimum>01<!-- Top-level --></usage_minimum>
<usage_maximum>40</usage_maximum>
<logical_minimum>0</logical_minimum>
<logical_maximum>255</logical_maximum>
<report_size>8</report_size>
<report_count>64</report_count>
<input>
<variable/>
</input>
<usage_minimum>01<!-- Top-level --></usage_minimum>
<usage_maximum>40</usage_maximum>
<output>
<variable/>
</output>
</COLLECTION>
</descriptor>

***@ubuntu64:~/Desktop$ dumphidrd.sh 04f2 0760
<?xml version="1.0"?>
<descriptor xmlns="http://digimend.sourceforge.net"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://digimend.sourceforge.net hidrd.xsd">
<usage_page>desktop<!-- Generic desktop controls --></usage_page>
<usage>desktop_keyboard<!-- Keyboard (06, top-level) --></usage>
<COLLECTION type="application">
<usage_page>keyboard<!-- Keyboard/keypad --></usage_page>
<usage_minimum>keyboard_kb_leftcontrol<!-- Keyboard left control
(E0) --></usage_minimum>
<usage_maximum>keyboard_kb_right_gui<!-- Keyboard right GUI (E7)
--></usage_maximum>
<logical_minimum>0</logical_minimum>
<logical_maximum>1</logical_maximum>
<report_size>1</report_size>
<report_count>8</report_count>
<input>
<variable/>
</input>
<report_count>1</report_count>
<report_size>8</report_size>
<input>
<constant/>
</input>
<report_count>3</report_count>
<report_size>1</report_size>
<usage_page>led<!-- LEDs --></usage_page>
<usage_minimum>01<!-- Top-level --></usage_minimum>
<usage_maximum>03<!-- Top-level --></usage_maximum>
<output>
<variable/>
</output>
<report_count>5</report_count>
<report_size>1</report_size>
<output>
<constant/>
</output>
<report_count>6</report_count>
<report_size>8</report_size>
<logical_minimum>0</logical_minimum>
<logical_maximum>255</logical_maximum>
<usage_page>keyboard<!-- Keyboard/keypad --></usage_page>
<usage_minimum>keyboard_none<!-- No event (00) --></usage_minimum>
<usage_maximum>FF</usage_maximum>
<input/>
</COLLECTION>
</descriptor>

The real report descriptor should be something like this.
http://www.microchip.com/forums/fb.aspx?m=377479 (lsusb, Linux)
http://www.microchip.com/forums/fb.aspx?m=367960 (HHD, Windows)

Somehow I prefer the plain text version like the one from lsusb.

[Xiaofan Chen]

Post by Xiaofan Chen

Post by Nikolai Kondrashov

Post by Xiaofan Chen
I see that you also has the hidrd-dump developed along with hidrd-convert.
Currently hidrd-dump (report descriptor dump utility) does not work too
well.

Could you please explain what's wrong with it? Thanks :) Or do you mean the
problem is that it doesn't dump the device descriptors itself?

It is not working well basically for some device.
I think it is based on libhid which does not detach the 2nd interface
if the device is a USB composite device.

I forgot one important thing. Your program (or libhid) detaches
the kernel hid driver, but it is better to re-attach the driver back
one the program exits. By doing this, you will lose your USB
HID mouse or keyboard (or need to unplug and plug) if using
your program.

I am not so sure about libhid. But libusb-1.0 has such function.
http://libusb.sourceforge.net/api-1.0/group__dev.html
int libusb_detach_kernel_driver (libusb_device_handle *dev, int interface)
int libusb_attach_kernel_driver (libusb_device_handle *dev, int interface)

[Nikolai Kondrashov]

Post by Xiaofan Chen
I forgot one important thing. Your program (or libhid) detaches
the kernel hid driver, but it is better to re-attach the driver back
one the program exits. By doing this, you will lose your USB
HID mouse or keyboard (or need to unplug and plug) if using
your program.

Thanks. Yeah, I know about it, of course.

Post by Xiaofan Chen
I am not so sure about libhid. But libusb-1.0 has such function.
http://libusb.sourceforge.net/api-1.0/group__dev.html
int libusb_detach_kernel_driver (libusb_device_handle *dev, int interface)
int libusb_attach_kernel_driver (libusb_device_handle *dev, int interface)

Thanks, I was just recently reading the libusb 1.0 documentation and
enjoying the appearance of the attach function. The libusb 0.1 didn't
have one and so did libhid.

The hid-dump, I was talking about before, will be based on libusb 1.0 and
will address this issue.

The only problem is that I'm doing it in my spare time and there isn't much
of it, so it wouldn't be fast. Sorry.

Sincerely,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Xiaofan Chen]

Post by Nikolai Kondrashov
Thanks, I was just recently reading the libusb 1.0 documentation and
enjoying the appearance of the attach function. The libusb 0.1 didn't
have one and so did libhid.
The hid-dump, I was talking about before, will be based on libusb 1.0 and
will address this issue.

That would be nice.

Post by Nikolai Kondrashov
The only problem is that I'm doing it in my spare time and there isn't much
of it, so it wouldn't be fast. Sorry.

No problem, take your time.

[Nikolai Kondrashov]

Post by Xiaofan Chen
The first three are good. The last one is a USB composite
device and only one interface is dumped.
The last one should be something like this.
http://www.microchip.com/forums/fb.aspx?m=377479

Thank you for descriptors and the detailed description of the problem :) I
was just wondering yesterday why my Keyboard/Mouse PS/2 adapter had only
keyboard in the descriptor. Now I know I have a device to test the fix on :)

I'm going to implement a hidrd-dump replacement, which will be called
"hid-dump" and will be able to address interfaces and dump reports too, not
only descriptors.

Post by Xiaofan Chen
Somehow I prefer the plain text version like the one from lsusb.

Yes, I understand that, I would prefer the plain text too, or at least some
terser language. However, currently, the XML is a real booster for
development. I'm placing the plain text output right after or even beside
the XML reading in my priorities.

Sincerely,
Nick
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html