Page 1 of 1

Ring buffers in userland

Posted: Mon May 04, 2020 6:44 pm
by hidnplayr
Hello fellow kernel-developers,

Does anyone know the steps required to implement ring-buffers for the application level?
I studied the kernel code a bit, but am afraid I don't quite understood the inner working of user heap.

Re: Ring buffers in userland

Posted: Mon May 04, 2020 7:50 pm
by dunkaist
Hi hidnplayr,
hidnplayr wrote:Does anyone know
Not me :D

I'm not sure I understood your question. Do you want something like io_uring?

Re: Ring buffers in userland

Posted: Mon May 04, 2020 8:05 pm
by hidnplayr
https://en.wikipedia.org/wiki/Circular_ ... timization

Analog to create_ring_buffer in kernel, but accessible from application/libraries.

Re: Ring buffers in userland

Posted: Mon May 04, 2020 11:38 pm
by dunkaist
After reading this post I managed to get two adjacent virtual pages mapped to the same physical page, RW accessible from the application:

Code: Select all

stdcall user_alloc, 0x2000
<-lin

call alloc_page
<-phys

stdcall map_page, lin, phys, PG_UWR
stdcall map_page, lin+0x1000, phys, PG_UWR
May be it makes sense to create create_ring_buffer_user and create_ring_buffer_common and move most part of create_ring_buffer to the latter one.

Re: Ring buffers in userland

Posted: Tue May 05, 2020 8:20 pm
by hidnplayr
dunkaist wrote:After reading this post I managed to get two adjacent virtual pages mapped to the same physical page, RW accessible from the application:

Code: Select all

stdcall user_alloc, 0x2000
<-lin

call alloc_page
<-phys

stdcall map_page, lin, phys, PG_UWR
stdcall map_page, lin+0x1000, phys, PG_UWR
May be it makes sense to create create_ring_buffer_user and create_ring_buffer_common and move most part of create_ring_buffer to the latter one.
Problem I see is that "stdcall user_alloc" gives us virtual memory that is already mapped to physical memory.
(And didn't find yet how/where it is mapped.)

Also, how to undo everything when we're done?

Re: Ring buffers in userland

Posted: Wed May 06, 2020 5:44 am
by dunkaist
hidnplayr wrote:Problem I see is that "stdcall user_alloc" gives us virtual memory that is already mapped to physical memory.
(And didn't find yet how/where it is mapped.)
Doesn't user_alloc allocates memory lazily, i.e. only virtual pages immediately, physical pages in PF handler?
If this is the case, then it is clear why you didn't find where user_alloc works with physical pages.
hidnplayr wrote:Also, how to undo everything when we're done?
Again, I'm just interpreting this post.
I see that user_free calls free_page too. Isn't it enough?

Re: Ring buffers in userland

Posted: Wed May 06, 2020 7:10 pm
by hidnplayr
dunkaist wrote:
hidnplayr wrote:Problem I see is that "stdcall user_alloc" gives us virtual memory that is already mapped to physical memory.
(And didn't find yet how/where it is mapped.)
Doesn't user_alloc allocates memory lazily, i.e. only virtual pages immediately, physical pages in PF handler?
If this is the case, then it is clear why you didn't find where user_alloc works with physical pages.
It seems you are right. Allocation is done in "page_fault_handler" in memory.inc
It explains the magic bit flag '2' sometimes referred to in the code as 'reserved'.

Thanks!