Understanding flow code

IkerZubi · September 16, 2016, 8:33am

Hi!

I have actually bought a PX4FLOW. It is the first time I work with image processing and I am really interested of knowing about how it works. I have been trying to understand the code but I have some questions.
My problem is not the sintaxis of the code (I can find it on the internet), what I don’t understand is the use/ utility of some functions. I think that with a short definition of some functions should be enough to start understanding everything.

I know that it is based on SAD algorithm which takes the sum of the absolute diferences on each pixel. I have seen that the lower is the sum, the more similar are the 2 images. The functions I don’t understand are the next ones:

1.-

github.com

PX4/PX4-Flow/blob/4a314cfdb099aed9795b825e7518203771207fbb/src/modules/flow/flow.c#L62

    
      
          #define FRAME_SIZE	global_data.param[PARAM_IMAGE_WIDTH]
          #define SEARCH_SIZE	global_data.param[PARAM_MAX_FLOW_PIXEL] // maximum offset to search: 4 + 1/2 pixels
          #define TILE_SIZE	8               						// x & y tile size
          #define NUM_BLOCKS	5 // x & y number of tiles to check
          
          
#define sign(x) (( x > 0 ) - ( x < 0 ))
          
          
uint8_t compute_flow(uint8_t *image1, uint8_t *image2, float x_rate, float y_rate, float z_rate, float *pixel_flow_x, float *pixel_flow_y);
          
          
// compliments of Adam Williams
          #define ABSDIFF(frame1, frame2) \
          ({ \
           int result = 0; \
           asm volatile( \
            "mov %[result], #0\n"           /* accumulator */ \
           \
            "ldr r4, [%[src], #0]\n"        /* read data from address + offset*/ \
            "ldr r5, [%[dst], #0]\n" \
            "usada8 %[result], r4, r5, %[result]\n"      /* difference */ \
            "ldr r4, [%[src], #4]\n"        /* read data from address + offset */ \
            "ldr r5, [%[dst], #4]\n" \

I think it is doing the SAD algorithm taking the difference on the images?

2.-

github.com

PX4/PX4-Flow/blob/4a314cfdb099aed9795b825e7518203771207fbb/src/modules/flow/flow.c#L62

    
      
          #define FRAME_SIZE	global_data.param[PARAM_IMAGE_WIDTH]
          #define SEARCH_SIZE	global_data.param[PARAM_MAX_FLOW_PIXEL] // maximum offset to search: 4 + 1/2 pixels
          #define TILE_SIZE	8               						// x & y tile size
          #define NUM_BLOCKS	5 // x & y number of tiles to check
          
          
#define sign(x) (( x > 0 ) - ( x < 0 ))
          
          
uint8_t compute_flow(uint8_t *image1, uint8_t *image2, float x_rate, float y_rate, float z_rate, float *pixel_flow_x, float *pixel_flow_y);
          
          
// compliments of Adam Williams
          #define ABSDIFF(frame1, frame2) \
          ({ \
           int result = 0; \
           asm volatile( \
            "mov %[result], #0\n"           /* accumulator */ \
           \
            "ldr r4, [%[src], #0]\n"        /* read data from address + offset*/ \
            "ldr r5, [%[dst], #0]\n" \
            "usada8 %[result], r4, r5, %[result]\n"      /* difference */ \
            "ldr r4, [%[src], #4]\n"        /* read data from address + offset */ \
            "ldr r5, [%[dst], #4]\n" \

Why it is a 4x4 pattern taken?
I have been reading about why is interesting to take the average pixel gradient but I continue without understanding why is so interesting and why it is used for knowing if the pixel is suitable for flow tracking:

github.com

PX4/PX4-Flow/blob/4a314cfdb099aed9795b825e7518203771207fbb/src/modules/flow/flow.c#L433

    
      
          	/* initialize with 0 */
          	for (j = 0; j < hist_size; j++) { histx[j] = 0; histy[j] = 0; }
          
          
	/* iterate over all patterns
          	 */
          	for (j = pixLo; j < pixHi; j += pixStep)
          	{
          		for (i = pixLo; i < pixHi; i += pixStep)
          		{
          			/* test pixel if it is suitable for flow tracking */
          			uint32_t diff = compute_diff(image1, i, j, (uint16_t) global_data.param[PARAM_IMAGE_WIDTH]);
          			if (diff < global_data.param[PARAM_BOTTOM_FLOW_FEATURE_THRESHOLD])
          			{
          				continue;
          			}
          
          
			uint32_t dist = 0xFFFFFFFF; // set initial distance to "infinity"
          			int8_t sumx = 0;
          			int8_t sumy = 0;
          			int8_t ii, jj;

3.-PX4-Flow/flow.c at 4a314cfdb099aed9795b825e7518203771207fbb · PX4/PX4-Flow · GitHub

I think I understand this function : It does the SAD algorithm for a 8x8 pattern taken from the two photos of (188*120). It only takes a 8x8 pattern to make the process quicklier and computable, isn’t it?. For analyzing the pixels, it analyze each bit of each pixel and takes de diference with the other image, so the ABSDIFF of EACH BIT of EACH PIXEL should be 1 or 0? It takes a lot of different 8x8 patterns of each photo?

4.- PX4-Flow/flow.c at 4a314cfdb099aed9795b825e7518203771207fbb · PX4/PX4-Flow · GitHub

I can see the difference from the above function. Isn’t the SAD computation done in the previous step function? Why it is doing again the SAD compute?

That’s all, I think if I understand what this functions are used for (Not how are done, why are used) I will be able to understand everything. I have surfing on the internet and I think I understand SAD algorithm, but I don know how it is implemented.

Thank you for your help