Discussion:
Using UVC webcam gadget with a real v4l2 device
Bhupesh SHARMA
2012-04-22 18:24:53 UTC
Permalink
Hi Laurent,

I have been doing some experimentation with the UVC webcam gadget along with the UVC user-space
application which you have written.

The UVC webcam gadget works fine with the user space application handling the CONTROL events and
providing DATA events. Now, I wish to interface a real v4l2 device, for e.g. VIVI or more particularly
a soc_camera based host and subdev pair.

Now, I see that I can achieve this by opening the UVC and V4L2 devices and doing MMAP -> REQBUF ->
QBUF -> DQBUF calls on both the devices per the UVC control event received. But this will involve
copying the video buffer in the user-space application from v4l2 (_CAPTURE) to uvc (_OUTPUT) domains,
which will significantly reduce the video capture performance.

Is there a better solution to this issue? Maybe doing something like a RNDIS gadget does with
the help of u_ether.c like helper routines. But if I remember well it also requires the BRCTL (Bridge
Control Utility) in userspace to route data arriving on usb0 to eth0 and vice-versa. Not sure though,
if it does copying of a skb buffer from ethernet to usb domain and vice-versa.

Any idea is much appreciated.

Thanks for your help,
Bhupesh
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Laurent Pinchart
2012-04-23 14:17:15 UTC
Permalink
Hi Bhupesh,
Post by Bhupesh SHARMA
Hi Laurent,
I have been doing some experimentation with the UVC webcam gadget along with
the UVC user-space application which you have written.
The UVC webcam gadget works fine with the user space application handling
the CONTROL events and providing DATA events. Now, I wish to interface a
real v4l2 device, for e.g. VIVI or more particularly a soc_camera based
host and subdev pair.
Now, I see that I can achieve this by opening the UVC and V4L2 devices and
doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the devices per the UVC
control event received. But this will involve copying the video buffer in
the user-space application from v4l2 (_CAPTURE) to uvc (_OUTPUT) domains,
which will significantly reduce the video capture performance.
Is there a better solution to this issue? Maybe doing something like a RNDIS
gadget does with the help of u_ether.c like helper routines. But if I
remember well it also requires the BRCTL (Bridge Control Utility) in
userspace to route data arriving on usb0 to eth0 and vice-versa. Not sure
though, if it does copying of a skb buffer from ethernet to usb domain and
vice-versa.
To avoid copying data between the two devices you should use USERPTR instead
of MMAP on at least one of the two V4L2 devices. The UVC gadget driver doesn't
support USERPTR yet though. This shouldn't be too difficult to fix, we need to
replace the custom buffers queue implementation with videobuf2, as has been
done in the uvcvideo driver.

I'll try to implement this. Would you then be able to test patches ?
--
Regards,

Laurent Pinchart
Bhupesh SHARMA
2012-04-23 18:46:22 UTC
Permalink
Hi Laurent,
-----Original Message-----
Sent: Monday, April 23, 2012 7:47 PM
To: Bhupesh SHARMA
Subject: Re: Using UVC webcam gadget with a real v4l2 device
Hi Bhupesh,
Post by Bhupesh SHARMA
Hi Laurent,
I have been doing some experimentation with the UVC webcam gadget
along with
Post by Bhupesh SHARMA
the UVC user-space application which you have written.
The UVC webcam gadget works fine with the user space application
handling
Post by Bhupesh SHARMA
the CONTROL events and providing DATA events. Now, I wish to
interface a
Post by Bhupesh SHARMA
real v4l2 device, for e.g. VIVI or more particularly a soc_camera
based
Post by Bhupesh SHARMA
host and subdev pair.
Now, I see that I can achieve this by opening the UVC and V4L2
devices and
Post by Bhupesh SHARMA
doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the devices per
the UVC
Post by Bhupesh SHARMA
control event received. But this will involve copying the video
buffer in
Post by Bhupesh SHARMA
the user-space application from v4l2 (_CAPTURE) to uvc (_OUTPUT)
domains,
Post by Bhupesh SHARMA
which will significantly reduce the video capture performance.
Is there a better solution to this issue? Maybe doing something like
a RNDIS
Post by Bhupesh SHARMA
gadget does with the help of u_ether.c like helper routines. But if I
remember well it also requires the BRCTL (Bridge Control Utility) in
userspace to route data arriving on usb0 to eth0 and vice-versa. Not
sure
Post by Bhupesh SHARMA
though, if it does copying of a skb buffer from ethernet to usb
domain and
Post by Bhupesh SHARMA
vice-versa.
To avoid copying data between the two devices you should use USERPTR instead
of MMAP on at least one of the two V4L2 devices. The UVC gadget driver doesn't
support USERPTR yet though. This shouldn't be too difficult to fix, we need to
replace the custom buffers queue implementation with videobuf2, as has been
done in the uvcvideo driver.
I was thinking of using the USERPTR method too, but I realized that currently
neither UVC webcam gadget nor soc-camera subsystem supports this IO method.
They support only MMAP IO as of now :(
I'll try to implement this. Would you then be able to test patches ?
For sure, I can test your patches on my setup.

BTW, I was exploring GSTREAMER to use the data arriving from soc-camera (v4l2)
capture device '/dev/video1' via 'v4l2src' plugin and routing the same to
the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.

Don't know if this can work cleanly in my setup and whether GSTREAMER actually
performs a buffer copy internally. But I will at-least give it a try :)

Regards,
Bhupesh
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Laurent Pinchart
2012-04-23 20:55:57 UTC
Permalink
Hi Bhupesh,
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
Hi Laurent,
I have been doing some experimentation with the UVC webcam gadget along
with the UVC user-space application which you have written.
The UVC webcam gadget works fine with the user space application
handling the CONTROL events and providing DATA events. Now, I wish to
interface a real v4l2 device, for e.g. VIVI or more particularly a
soc_camera based host and subdev pair.
Now, I see that I can achieve this by opening the UVC and V4L2 devices
and doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the devices per
the UVC control event received. But this will involve copying the video
buffer in the user-space application from v4l2 (_CAPTURE) to uvc
(_OUTPUT) domains, which will significantly reduce the video capture
performance.
Is there a better solution to this issue? Maybe doing something like a
RNDIS gadget does with the help of u_ether.c like helper routines. But
if I remember well it also requires the BRCTL (Bridge Control Utility)
in userspace to route data arriving on usb0 to eth0 and vice-versa. Not
sure though, if it does copying of a skb buffer from ethernet to usb
domain and vice-versa.
To avoid copying data between the two devices you should use USERPTR
instead of MMAP on at least one of the two V4L2 devices. The UVC gadget
driver doesn't support USERPTR yet though. This shouldn't be too difficult
to fix, we need toreplace the custom buffers queue implementation with
videobuf2, as has been done in the uvcvideo driver.
I was thinking of using the USERPTR method too, but I realized that
currently neither UVC webcam gadget nor soc-camera subsystem supports this
IO method. They support only MMAP IO as of now :(
Both soc-camera and the UVC gadget driver should be ported to videobuf2 to fix
the problem.
Post by Bhupesh SHARMA
Post by Laurent Pinchart
I'll try to implement this. Would you then be able to test patches ?
For sure, I can test your patches on my setup.
I had a quick look, but there's a bit more work than expected. The UVC gadget
driver locking scheme needs to be revisited. I unfortunately won't have time
to work on that in the next couple of weeks, and very probably not before end
of June. Sorry.

If you want to give it a try, I can provide you with some pointers.
Post by Bhupesh SHARMA
BTW, I was exploring GSTREAMER to use the data arriving from soc-camera
(v4l2) capture device '/dev/video1' via 'v4l2src' plugin and routing the
same to the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.
Don't know if this can work cleanly in my setup and whether GSTREAMER
actually performs a buffer copy internally. But I will at-least give it a
try :)
There will definitely be a buffer copy (and actually two copies, as the UVC
gadget driver performs a second copy internally) if you don't use USERPTR.
--
Regards,

Laurent Pinchart

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Bhupesh SHARMA
2012-04-25 15:06:47 UTC
Permalink
Hi Laurent,
-----Original Message-----
Sent: Tuesday, April 24, 2012 2:26 AM
To: Bhupesh SHARMA
Subject: Re: Using UVC webcam gadget with a real v4l2 device
Hi Bhupesh,
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
Hi Laurent,
I have been doing some experimentation with the UVC webcam gadget
along
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
with the UVC user-space application which you have written.
The UVC webcam gadget works fine with the user space application
handling the CONTROL events and providing DATA events. Now, I
wish to
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
interface a real v4l2 device, for e.g. VIVI or more particularly
a
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
soc_camera based host and subdev pair.
Now, I see that I can achieve this by opening the UVC and V4L2
devices
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
and doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the
devices per
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
the UVC control event received. But this will involve copying the
video
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
buffer in the user-space application from v4l2 (_CAPTURE) to uvc
(_OUTPUT) domains, which will significantly reduce the video
capture
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
performance.
Is there a better solution to this issue? Maybe doing something
like a
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
RNDIS gadget does with the help of u_ether.c like helper
routines. But
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
if I remember well it also requires the BRCTL (Bridge Control
Utility)
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
in userspace to route data arriving on usb0 to eth0 and vice-
versa. Not
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
sure though, if it does copying of a skb buffer from ethernet to
usb
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
domain and vice-versa.
To avoid copying data between the two devices you should use
USERPTR
Post by Bhupesh SHARMA
Post by Laurent Pinchart
instead of MMAP on at least one of the two V4L2 devices. The UVC
gadget
Post by Bhupesh SHARMA
Post by Laurent Pinchart
driver doesn't support USERPTR yet though. This shouldn't be too
difficult
Post by Bhupesh SHARMA
Post by Laurent Pinchart
to fix, we need toreplace the custom buffers queue implementation
with
Post by Bhupesh SHARMA
Post by Laurent Pinchart
videobuf2, as has been done in the uvcvideo driver.
I was thinking of using the USERPTR method too, but I realized that
currently neither UVC webcam gadget nor soc-camera subsystem supports
this
Post by Bhupesh SHARMA
IO method. They support only MMAP IO as of now :(
Both soc-camera and the UVC gadget driver should be ported to videobuf2 to fix
the problem.
Post by Bhupesh SHARMA
Post by Laurent Pinchart
I'll try to implement this. Would you then be able to test patches
?
Post by Bhupesh SHARMA
For sure, I can test your patches on my setup.
I had a quick look, but there's a bit more work than expected. The UVC gadget
driver locking scheme needs to be revisited. I unfortunately won't have time
to work on that in the next couple of weeks, and very probably not before end
of June. Sorry.
If you want to give it a try, I can provide you with some pointers.
It's a pity. You are the best person to do it as you have in-depth know
-how of both v4l2 and UVC webcam gadget. But I can give it a try if you
can provide me some pointers..
Post by Bhupesh SHARMA
BTW, I was exploring GSTREAMER to use the data arriving from soc-
camera
Post by Bhupesh SHARMA
(v4l2) capture device '/dev/video1' via 'v4l2src' plugin and routing
the
Post by Bhupesh SHARMA
same to the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.
Don't know if this can work cleanly in my setup and whether GSTREAMER
actually performs a buffer copy internally. But I will at-least give
it a
Post by Bhupesh SHARMA
try :)
There will definitely be a buffer copy (and actually two copies, as the UVC
gadget driver performs a second copy internally) if you don't use USERPTR.
That's what I was afraid of. But can you let me know where the gadget driver
performs a second copy internally, so that I can also start exploring the
USERPTR method using the pointers provided by you..

Regards,
Bhupesh
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Bhupesh SHARMA
2012-04-26 05:23:59 UTC
Permalink
Hi Laurent,

Sorry to jump-in before your reply on my previous mail,
but as I was studying the USERPTR stuff in more detail, I have a few more
queries which I believe you can include in your reply as well..
-----Original Message-----
From: Bhupesh SHARMA
Sent: Wednesday, April 25, 2012 8:37 PM
To: 'Laurent Pinchart'
Subject: RE: Using UVC webcam gadget with a real v4l2 device
Hi Laurent,
-----Original Message-----
Sent: Tuesday, April 24, 2012 2:26 AM
To: Bhupesh SHARMA
Subject: Re: Using UVC webcam gadget with a real v4l2 device
Hi Bhupesh,
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
Hi Laurent,
I have been doing some experimentation with the UVC webcam
gadget
along
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
with the UVC user-space application which you have written.
The UVC webcam gadget works fine with the user space
application
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
handling the CONTROL events and providing DATA events. Now, I
wish to
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
interface a real v4l2 device, for e.g. VIVI or more
particularly
a
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
soc_camera based host and subdev pair.
Now, I see that I can achieve this by opening the UVC and V4L2
devices
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
and doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the
devices per
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
the UVC control event received. But this will involve copying
the
video
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
buffer in the user-space application from v4l2 (_CAPTURE) to
uvc
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
(_OUTPUT) domains, which will significantly reduce the video
capture
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
performance.
Is there a better solution to this issue? Maybe doing something
like a
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
RNDIS gadget does with the help of u_ether.c like helper
routines. But
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
if I remember well it also requires the BRCTL (Bridge Control
Utility)
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
in userspace to route data arriving on usb0 to eth0 and vice-
versa. Not
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
sure though, if it does copying of a skb buffer from ethernet
to
usb
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
domain and vice-versa.
To avoid copying data between the two devices you should use
USERPTR
Post by Bhupesh SHARMA
Post by Laurent Pinchart
instead of MMAP on at least one of the two V4L2 devices. The UVC
gadget
Post by Bhupesh SHARMA
Post by Laurent Pinchart
driver doesn't support USERPTR yet though. This shouldn't be too
difficult
Post by Bhupesh SHARMA
Post by Laurent Pinchart
to fix, we need toreplace the custom buffers queue implementation
with
Post by Bhupesh SHARMA
Post by Laurent Pinchart
videobuf2, as has been done in the uvcvideo driver.
I was thinking of using the USERPTR method too, but I realized that
currently neither UVC webcam gadget nor soc-camera subsystem
supports
this
Post by Bhupesh SHARMA
IO method. They support only MMAP IO as of now :(
Both soc-camera and the UVC gadget driver should be ported to
videobuf2
to fix
the problem.
I am now a bit confused on how the entire system will work now:
- Does USERPTR method needs to be supported both in UVC gadget and soc-camera side,
or one can still support the MMAP method and the other can now be changed to support USERPTR method
and we can achieve a ZERO buffer copy operation using this method?

- More specifically, I would like to keep the soc-camera still using MMAP (and hence still using video-buf)
and make changes at the UVC gadget side to support USERPTR and videobuf2. Will this work?

- At the application side how should we design the flow in case both support USERPTR, i.e. the buffer needs
to be protected from simultaneous access from the UVC gadget driver and soc-camera driver (to ensure that
a single buffer can be shared across them). Also in case we keep soc-camera still using MMAP and UVC gadget
side supporting USERPTR, how can we share a common buffer across the UVC gadget and soc-camera driver.

- In case of USERPTR method the camera capture hardware should be able to DMA the received data to the user
space buffers. Are there any specific requirements on the DMA capability of these use-space buffers
(scatter-gather or contiguous?).

Regards,
Bhupesh
Post by Bhupesh SHARMA
Post by Laurent Pinchart
I'll try to implement this. Would you then be able to test
patches
?
Post by Bhupesh SHARMA
For sure, I can test your patches on my setup.
I had a quick look, but there's a bit more work than expected. The
UVC
gadget
driver locking scheme needs to be revisited. I unfortunately won't
have
time
to work on that in the next couple of weeks, and very probably not before end
of June. Sorry.
If you want to give it a try, I can provide you with some pointers.
It's a pity. You are the best person to do it as you have in-depth
know
-how of both v4l2 and UVC webcam gadget. But I can give it a try if you
can provide me some pointers..
Post by Bhupesh SHARMA
BTW, I was exploring GSTREAMER to use the data arriving from soc-
camera
Post by Bhupesh SHARMA
(v4l2) capture device '/dev/video1' via 'v4l2src' plugin and
routing
the
Post by Bhupesh SHARMA
same to the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.
Don't know if this can work cleanly in my setup and whether
GSTREAMER
Post by Bhupesh SHARMA
actually performs a buffer copy internally. But I will at-least
give
it a
Post by Bhupesh SHARMA
try :)
There will definitely be a buffer copy (and actually two copies, as
the
UVC
gadget driver performs a second copy internally) if you don't use USERPTR.
That's what I was afraid of. But can you let me know where the gadget
driver
performs a second copy internally, so that I can also start exploring
the
USERPTR method using the pointers provided by you..
Regards,
Bhupesh
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Laurent Pinchart
2012-04-30 10:20:47 UTC
Permalink
Hi Bhupesh,
Post by Bhupesh SHARMA
Hi Laurent,
Sorry to jump-in before your reply on my previous mail,
but as I was studying the USERPTR stuff in more detail, I have a few more
queries which I believe you can include in your reply as well..
[snip]
Post by Bhupesh SHARMA
- Does USERPTR method needs to be supported both in UVC gadget and
soc-camera side, or one can still support the MMAP method and the other can
now be changed to support USERPTR method and we can achieve a ZERO buffer
copy operation using this method?
You need USERPTR support on one side only. In practice many (all?) soc-camera
drivers require physically contiguous memory, so you will need to use MMAP on
the soc-camera side and USERPTR on the UVC gadget side. DMABUF, when merged in
the kernel, will be a better solution (but will require all drivers to use
vb2).
Post by Bhupesh SHARMA
- More specifically, I would like to keep the soc-camera still using MMAP
(and hence still using video-buf) and make changes at the UVC gadget side
to support USERPTR and videobuf2. Will this work?
Please see above :-)
Post by Bhupesh SHARMA
- At the application side how should we design the flow in case both
support USERPTR, i.e. the buffer needs to be protected from simultaneous
access from the UVC gadget driver and soc-camera driver (to ensure that a
single buffer can be shared across them). Also in case we keep soc-camera
still using MMAP and UVC gadget side supporting USERPTR, how can we share a
common buffer across the UVC gadget and soc-camera driver.
That's easy. Request the same number of buffers on both sides with REQBUFS,
mmap() them to userspace on the soc-camera side, and then use the user pointer
to queue them with QBUF on the UVC side. You just need to ensure that a buffer
is never enqueued to two drivers at the same time. Wait for buffers to be
ready on both sides with select(), and when a buffer is ready dequeue it and
requeue it on the other side.
Post by Bhupesh SHARMA
- In case of USERPTR method the camera capture hardware should be able to
DMA the received data to the user space buffers. Are there any specific
requirements on the DMA capability of these use-space buffers
(scatter-gather or contiguous?).
DMA to userspace is quite hackish. You should use the MMAP method on the soc-
camera side.
--
Regards,

Laurent Pinchart

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Bhupesh SHARMA
2012-04-30 10:47:24 UTC
Permalink
Hi Laurent,
-----Original Message-----
Sent: Monday, April 30, 2012 3:51 PM
To: Bhupesh SHARMA
Subject: Re: Using UVC webcam gadget with a real v4l2 device
Hi Bhupesh,
Post by Bhupesh SHARMA
Hi Laurent,
Sorry to jump-in before your reply on my previous mail,
but as I was studying the USERPTR stuff in more detail, I have a few
more
Post by Bhupesh SHARMA
queries which I believe you can include in your reply as well..
[snip]
Post by Bhupesh SHARMA
- Does USERPTR method needs to be supported both in UVC gadget
and
Post by Bhupesh SHARMA
soc-camera side, or one can still support the MMAP method and the
other can
Post by Bhupesh SHARMA
now be changed to support USERPTR method and we can achieve a ZERO
buffer
Post by Bhupesh SHARMA
copy operation using this method?
You need USERPTR support on one side only. In practice many (all?) soc-
camera
drivers require physically contiguous memory, so you will need to use MMAP on
the soc-camera side and USERPTR on the UVC gadget side. DMABUF, when merged in
the kernel, will be a better solution (but will require all drivers to use
vb2).
Perfect. So, I plan now to add vb2 support for uvc-gadget and leave soc-camera
side to use the mmap stuff.

Now, waiting for your pointers for managing the race-conditions in the UVC gadget
and also avoiding the memcpy that is happening in the QBUF call on the UVC gadget,
before I start the actual work.

Thanks for your help.

Regards,
Bhupesh
Post by Bhupesh SHARMA
- More specifically, I would like to keep the soc-camera still
using MMAP
Post by Bhupesh SHARMA
(and hence still using video-buf) and make changes at the UVC gadget
side
Post by Bhupesh SHARMA
to support USERPTR and videobuf2. Will this work?
Please see above :-)
Post by Bhupesh SHARMA
- At the application side how should we design the flow in case
both
Post by Bhupesh SHARMA
support USERPTR, i.e. the buffer needs to be protected from
simultaneous
Post by Bhupesh SHARMA
access from the UVC gadget driver and soc-camera driver (to ensure
that a
Post by Bhupesh SHARMA
single buffer can be shared across them). Also in case we keep soc-
camera
Post by Bhupesh SHARMA
still using MMAP and UVC gadget side supporting USERPTR, how can we
share a
Post by Bhupesh SHARMA
common buffer across the UVC gadget and soc-camera driver.
That's easy. Request the same number of buffers on both sides with REQBUFS,
mmap() them to userspace on the soc-camera side, and then use the user pointer
to queue them with QBUF on the UVC side. You just need to ensure that a buffer
is never enqueued to two drivers at the same time. Wait for buffers to be
ready on both sides with select(), and when a buffer is ready dequeue it and
requeue it on the other side.
Post by Bhupesh SHARMA
- In case of USERPTR method the camera capture hardware should be
able to
Post by Bhupesh SHARMA
DMA the received data to the user space buffers. Are there any
specific
Post by Bhupesh SHARMA
requirements on the DMA capability of these use-space buffers
(scatter-gather or contiguous?).
DMA to userspace is quite hackish. You should use the MMAP method on the soc-
camera side.
--
Regards,
Laurent Pinchart
Laurent Pinchart
2012-05-02 11:55:38 UTC
Permalink
Hi Bhupesh,
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
Hi Laurent,
Sorry to jump-in before your reply on my previous mail, but as I was
studying the USERPTR stuff in more detail, I have a few more queries
which I believe you can include in your reply as well..
[snip]
Post by Bhupesh SHARMA
- Does USERPTR method needs to be supported both in UVC gadget and
soc-camera side, or one can still support the MMAP method and the other
can now be changed to support USERPTR method and we can achieve a ZERO
buffer copy operation using this method?
You need USERPTR support on one side only. In practice many (all?) soc-
camera drivers require physically contiguous memory, so you will need to
use MMAP on the soc-camera side and USERPTR on the UVC gadget side.
DMABUF, when merged in the kernel, will be a better solution (but will
require all drivers to use vb2).
Perfect. So, I plan now to add vb2 support for uvc-gadget and leave soc-
camera side to use the mmap stuff.
Now, waiting for your pointers for managing the race-conditions in the UVC
gadget and also avoiding the memcpy that is happening in the QBUF call on
the UVC gadget, before I start the actual work.
The memcpy doesn't happen at QBUF time, but when filling the URBs. Avoiding it
will be pretty difficult, as the driver needs to add packet headers. I would
leave that out for now.

Regarding videobuf2 support, the main issue comes from race conditions between
stream start, buffer queueing and URB completion. Unlike the UVC host driver
where URBs can be resubmitted immediately, the gadget driver can only resubmit
URBs (in uvc_video_complete()) when there is data to be sent. Otherwise the
URB is put on a free URBs list (video->req_free) and enqueued in
uvc_video_pump() the next time a buffer is queued. This requires taking
various locks and must thus be considered with care. I'm pretty unhappy with
calling video->encode with the queue irqlock held, I would like to change
that, but I don't expect to to be an easy task.
--
Regards,

Laurent Pinchart

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-***@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Bhupesh SHARMA
2012-04-30 03:34:09 UTC
Permalink
Hi Laurent,
-----Original Message-----
From: Bhupesh SHARMA
Sent: Thursday, April 26, 2012 10:54 AM
To: 'Laurent Pinchart'
Subject: RE: Using UVC webcam gadget with a real v4l2 device
Hi Laurent,
Sorry to jump-in before your reply on my previous mail,
but as I was studying the USERPTR stuff in more detail, I have a few
more
queries which I believe you can include in your reply as well..
-----Original Message-----
From: Bhupesh SHARMA
Sent: Wednesday, April 25, 2012 8:37 PM
To: 'Laurent Pinchart'
Subject: RE: Using UVC webcam gadget with a real v4l2 device
Hi Laurent,
-----Original Message-----
Sent: Tuesday, April 24, 2012 2:26 AM
To: Bhupesh SHARMA
Subject: Re: Using UVC webcam gadget with a real v4l2 device
Hi Bhupesh,
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
Hi Laurent,
I have been doing some experimentation with the UVC webcam
gadget
along
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
with the UVC user-space application which you have written.
The UVC webcam gadget works fine with the user space
application
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
handling the CONTROL events and providing DATA events. Now, I
wish to
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
interface a real v4l2 device, for e.g. VIVI or more
particularly
a
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
soc_camera based host and subdev pair.
Now, I see that I can achieve this by opening the UVC and
V4L2
devices
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
and doing MMAP -> REQBUF -> QBUF -> DQBUF calls on both the
devices per
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
the UVC control event received. But this will involve copying
the
video
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
buffer in the user-space application from v4l2 (_CAPTURE) to
uvc
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
(_OUTPUT) domains, which will significantly reduce the video
capture
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
performance.
Is there a better solution to this issue? Maybe doing
something
like a
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
RNDIS gadget does with the help of u_ether.c like helper
routines. But
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
if I remember well it also requires the BRCTL (Bridge Control
Utility)
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
in userspace to route data arriving on usb0 to eth0 and vice-
versa. Not
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
sure though, if it does copying of a skb buffer from ethernet
to
usb
Post by Bhupesh SHARMA
Post by Laurent Pinchart
Post by Bhupesh SHARMA
domain and vice-versa.
To avoid copying data between the two devices you should use
USERPTR
Post by Bhupesh SHARMA
Post by Laurent Pinchart
instead of MMAP on at least one of the two V4L2 devices. The
UVC
gadget
Post by Bhupesh SHARMA
Post by Laurent Pinchart
driver doesn't support USERPTR yet though. This shouldn't be
too
difficult
Post by Bhupesh SHARMA
Post by Laurent Pinchart
to fix, we need toreplace the custom buffers queue
implementation
with
Post by Bhupesh SHARMA
Post by Laurent Pinchart
videobuf2, as has been done in the uvcvideo driver.
I was thinking of using the USERPTR method too, but I realized
that
Post by Bhupesh SHARMA
currently neither UVC webcam gadget nor soc-camera subsystem
supports
this
Post by Bhupesh SHARMA
IO method. They support only MMAP IO as of now :(
Both soc-camera and the UVC gadget driver should be ported to
videobuf2
to fix
the problem.
- Does USERPTR method needs to be supported both in UVC gadget
and soc-camera side,
or one can still support the MMAP method and the other can now
be changed to support USERPTR method
and we can achieve a ZERO buffer copy operation using this
method?
- More specifically, I would like to keep the soc-camera still
using MMAP (and hence still using video-buf)
and make changes at the UVC gadget side to support USERPTR and
videobuf2. Will this work?
- At the application side how should we design the flow in case
both support USERPTR, i.e. the buffer needs
to be protected from simultaneous access from the UVC gadget
driver and soc-camera driver (to ensure that
a single buffer can be shared across them). Also in case we
keep soc-camera still using MMAP and UVC gadget
side supporting USERPTR, how can we share a common buffer
across the UVC gadget and soc-camera driver.
- In case of USERPTR method the camera capture hardware should be
able to DMA the received data to the user
space buffers. Are there any specific requirements on the DMA
capability of these use-space buffers
(scatter-gather or contiguous?).
Can you help me with these queries.

Thanks,
Bhupesh
Regards,
Bhupesh
Post by Bhupesh SHARMA
Post by Laurent Pinchart
I'll try to implement this. Would you then be able to test
patches
?
Post by Bhupesh SHARMA
For sure, I can test your patches on my setup.
I had a quick look, but there's a bit more work than expected. The
UVC
gadget
driver locking scheme needs to be revisited. I unfortunately won't
have
time
to work on that in the next couple of weeks, and very probably not before end
of June. Sorry.
If you want to give it a try, I can provide you with some pointers.
It's a pity. You are the best person to do it as you have in-depth
know
-how of both v4l2 and UVC webcam gadget. But I can give it a try if
you
can provide me some pointers..
Post by Bhupesh SHARMA
BTW, I was exploring GSTREAMER to use the data arriving from soc-
camera
Post by Bhupesh SHARMA
(v4l2) capture device '/dev/video1' via 'v4l2src' plugin and
routing
the
Post by Bhupesh SHARMA
same to the UVC gadget '/dev/video0' via the 'v4l2sink' plugin.
Don't know if this can work cleanly in my setup and whether
GSTREAMER
Post by Bhupesh SHARMA
actually performs a buffer copy internally. But I will at-least
give
it a
Post by Bhupesh SHARMA
try :)
There will definitely be a buffer copy (and actually two copies, as
the
UVC
gadget driver performs a second copy internally) if you don't use USERPTR.
That's what I was afraid of. But can you let me know where the gadget
driver
performs a second copy internally, so that I can also start exploring
the
USERPTR method using the pointers provided by you..
Regards,
Bhupesh
Peter Chen
2012-04-24 00:36:34 UTC
Permalink
Post by Bhupesh SHARMA
Hi Laurent,
I have been doing some experimentation with the UVC webcam gadget along with the UVC user-space
application which you have written.
I have tried UVC webcam gadget at Freescale i.mx platform,
unfortunately, It can't work
properly with my demo application. Would you tell me where I can get
Laurent's user-space
application? Thanks.
--
BR,
Peter Chen
Laurent Pinchart
2012-04-24 09:04:40 UTC
Permalink
Post by Bhupesh SHARMA
Hi Laurent,
I have been doing some experimentation with the UVC webcam gadget along
with the UVC user-space application which you have written.
I have tried UVC webcam gadget at Freescale i.mx platform, unfortunately,
It can't work properly with my demo application. Would you tell me where I
can get Laurent's user-space application? Thanks.
Here you go: http://git.ideasonboard.org/uvc-gadget.git
--
Regards,

Laurent Pinchart
Loading...