matcher.subpixel — Performing subpixel registration between images

The matcher.subpixel module

New in version 0.1.0.

autocnet.matcher.subpixel.check_image_size(imagesize)

Given an x,y tuple, ensure that the values are odd. Used by the subpixel template to also ensure that the template size is the one requested and not 2x the template size.

Parameters

imagesize (tuple) – in the form (size_x, size_y)

autocnet.matcher.subpixel.clip_roi(img, center_x, center_y, size_x=200, size_y=200, dtype='uint64')

Given an input image, clip a square region of interest centered on some pixel at some size.

Parameters

• img (ndarray or object) – The input image to be clipped or an object with a read_array method that takes a pixels argument in the form [xstart, ystart, xstop, ystop]

• center_x (Numeric) – The x coordinate to the center of the roi

• center_y (Numeric) – The y coordinate to the center of the roi

• img_size (int) – 1/2 of the total image size. This value is the number of pixels grabbed from each side of the center

Returns

clipped_img – The clipped image

Return type

ndarray

autocnet.matcher.subpixel.estimate_affine_transformation(destination_coordinates, source_coordinates)

Given a set of destination control points compute the affine transformation required to project the source control points into the destination.

Parameters

• destination_coordinates (array-like) – An n,2 data structure containing the destination control points

• source_coordinates (array-like) – An n,2 data structure containing the source control points

Returns

An skimage affine transform object

Return type

object

autocnet.matcher.subpixel.estimate_logpolar_transform(img1, img2, low_sigma=0.5, high_sigma=30, verbose=False)

Estimates the rotation and scale difference for img1 that maps to img2 using phase cross correlation on a logscale projection.

Scale and angular changes in cartesian space become translation in log-polar space. Translation from subpixel registration in log-polar space can then be translated into scale/rotation change in the original cartesian images. This scale + roation change estimation is then returned as an affine transform object. This can then be used before other subpixel registration methods to enable scale+rotation invariance.

See also

skimage.filters.difference_of_gaussians

Bandpass filtering using a difference of gaussians

skimage.filters.window

Simple wondowing function to remove spectral leakage along the axes in the fourier transform

References

1

Rittavee Matungka. 2009. Studies on log-polar transform for image registration and improvements using adaptive sampling and logarithmic spiral. Ph.D. Dissertation. Ohio State University, USA. Advisor(s) Yuan F. Zheng. Order Number: AAI3376091.

2

https://github.com/polakluk/fourier-mellin

3

https://scikit-image.org/docs/stable/auto_examples/registration/plot_register_rotation.html

Parameters

• img1 (np.ndarray) – The source image, this is the image that is used as a base as img2 is registered to the center on img1

• img2 (np.ndarray) – The image that will be moved to match img1

• low_sigma (float, list, np.array) – The low standard deviation for the Gaussian kernel used in the difference of gaussians filter. This reccomended to remove high frequency noise from the image before the log-polar projection as high frequency noise negatively impact registration in log-polar space. The lower the sigma, the sharper the resulting image will be. Use a small low_sigma with a large high_sigma to remove high frequency noise. Default is 0.5.

• high_sigma (float, list, np.array) – Standard deviation for the Gaussian kernel with the larger sigmas across all axes used in the difference of gaussians filter. This reccomended to remove high frequency noise from the image before the log-polar projection as high frequency noise negatively impact registration in log-polar space. The higher this sigma compared to the low_sigma, the more detail will be preserved. Use a small low_sigma with a large high_sigma to remove high frequency noise. A high sigma equal to ~1.6x the low sigma is reccomended for edge detection, so consider high_sigmas >= low_sigma*1.6. Default is 30.

• verbose (bool) – If true, prints out information detailing the registration process

Returns

Scikit-image affine transformation object containing rotation and scale information to warp img1 to img2

Return type

skimage.transform.SimilarityTansform

autocnet.matcher.subpixel.fourier_mellen(img1, img2, verbose=False, phase_kwargs={})

Iterative phase registration using a log-polar projection to estimate an affine for scale and roation invariance.

Parameters

• img1 (np.ndarray) – The source image, this is the image that is used as a base as img2 is registered to the center on img1

• img2 (np.ndarray) – The image that will be moved to match img1

• verbose (bool) – If true, prints out information detailing the registration process

• phase_kwargs (dict) – Parameters to be passed into the iterative_phase matcher

Returns

• float – The new x coordinate for img2 that registers to the center of img1

• float – The new y coordinate for img2 that registers to the center of img1

• float – Error returned by the iterative phase matcher

autocnet.matcher.subpixel.geom_match(destination_cube, source_cube, bcenter_x, bcenter_y, template_kwargs={'image_size': (59, 59), 'template_size': (31, 31)}, verbose=True)

Propagates a source measure into destination images and then perfroms subpixel registration. Measure creation is done by projecting the (lon, lat) associated with the source measure into the destination image. The created measure is then matched to the source measure using a quick projection of the destination image into source image space (using an affine transformation) and a naive template match with optional phase template match.

Parameters

• destination_cube (plio.io.io_gdal.GeoDataset) – The image to be matched to

• source_cube (plio.io.io_gdal.GeoDataset) – The image that is transformed and matched into the destination_cube

• bcenter_x (int) – sample location of source measure in base_cube

• bcenter_y (int) – line location of source measure in base_cube

• size_x (int) – half-height of the subimage used in the affine transformation

• size_y (int) – half-width of the subimage used in affine transformation

• template_kwargs (dict) – contains keywords necessary for autocnet.matcher.subpixel.subpixel_template

• verbose (boolean) – indicates level of print out desired. If True, two subplots are output; the first subplot contains the source subimage and projected destination subimage, the second subplot contains the registered measure’s location in the base subimage and the unprojected destination subimage with the corresponding template metric correlation map.

Returns

• sample (int) – sample of new measure in destination image space

• line (int) – line of new measures in destination image space

• dist (np.float or tuple of np.float) – distance matching algorithm moved measure if template matcher only (default): returns dist_template if template and phase matcher: returns (dist_template, dist_phase)

• metric (np.float or tuple of np.float) – matching metric output by the matcher if template matcher only (default): returns maxcorr if template and phase matcher: returns (maxcorr, perror, pdiff)

• temp_corrmap (np.ndarray) – correlation map of the naive template matcher

See also

autocnet.matcher.subpixel.subpixel_template

for list of kwargs that can be passed to the matcher

autocnet.matcher.subpixel.subpixel_phase

for list of kwargs that can be passed to the matcher

autocnet.matcher.subpixel.geom_match_classic(base_cube, input_cube, bcenter_x, bcenter_y, size_x=60, size_y=60, template_kwargs={'image_size': (59, 59), 'template_size': (31, 31)}, phase_kwargs=None, verbose=True)

Propagates a source measure into destination images and then perfroms subpixel registration. Measure creation is done by projecting the (lon, lat) associated with the source measure into the destination image. The created measure is then matched to the source measure using a quick projection of the destination image into source image space (using an affine transformation) and a naive template match with optional phase template match. :param base_cube: source image :type base_cube: plio.io.io_gdal.GeoDataset :param input_cube: destination image; gets matched to the source image :type input_cube: plio.io.io_gdal.GeoDataset :param bcenter_x: sample location of source measure in base_cube :type bcenter_x: int :param bcenter_y: line location of source measure in base_cube :type bcenter_y: int :param size_x: half-height of the subimage used in the affine transformation :type size_x: int :param size_y: half-width of the subimage used in affine transformation :type size_y: int :param template_kwargs: contains keywords necessary for autocnet.matcher.subpixel.subpixel_template :type template_kwargs: dict :param phase_kwargs: contains kwargs for autocnet.matcher.subpixel.subpixel_phase :type phase_kwargs: dict :param verbose: indicates level of print out desired. If True, two subplots are output; the first subplot contains

the source subimage and projected destination subimage, the second subplot contains the registered measure’s location in the base subimage and the unprojected destination subimage with the corresponding template metric correlation map.

Returns

• sample (int) – sample of new measure in destination image space

• line (int) – line of new measures in destination image space

• dist (np.float or tuple of np.float) – distance matching algorithm moved measure if template matcher only (default): returns dist_template if template and phase matcher: returns (dist_template, dist_phase)

• metric (np.float or tuple of np.float) – matching metric output by the matcher if template matcher only (default): returns maxcorr if template and phase matcher: returns (maxcorr, perror, pdiff)

• temp_corrmap (np.ndarray) – correlation map of the naive template matcher

See also

autocnet.matcher.subpixel.subpixel_template

for list of kwargs that can be passed to the matcher

autocnet.matcher.subpixel.subpixel_phase

for list of kwargs that can be passed to the matcher

autocnet.matcher.subpixel.geom_match_simple(base_cube, input_cube, bcenter_x, bcenter_y, size_x=60, size_y=60, match_func='classic', match_kwargs={'image_size': (101, 101), 'template_size': (31, 31)}, phase_kwargs=None, verbose=True)

Propagates a source measure into destination images and then perfroms subpixel registration. Measure creation is done by projecting the (lon, lat) associated with the source measure into the destination image. The created measure is then matched to the source measure using a quick projection of the destination image into source image space (using an affine transformation) and a naive template match with optional phase template match.

This version projects the entirity of the input cube onto the base

Parameters

• base_cube (plio.io.io_gdal.GeoDataset) – source image

• input_cube (plio.io.io_gdal.GeoDataset) – destination image; gets matched to the source image

• bcenter_x (int) – sample location of source measure in base_cube

• bcenter_y (int) – line location of source measure in base_cube

• size_x (int) – half-height of the subimage used in the affine transformation

• size_y (int) – half-width of the subimage used in affine transformation

• template_kwargs (dict) – contains keywords necessary for autocnet.matcher.subpixel.subpixel_template

• phase_kwargs (dict) – contains kwargs for autocnet.matcher.subpixel.subpixel_phase

• verbose (boolean) – indicates level of print out desired. If True, two subplots are output; the first subplot contains the source subimage and projected destination subimage, the second subplot contains the registered measure’s location in the base subimage and the unprojected destination subimage with the corresponding template metric correlation map.

Returns

• sample (int) – sample of new measure in destination image space

• line (int) – line of new measures in destination image space

• dist (np.float or tuple of np.float) – distance matching algorithm moved measure if template matcher only (default): returns dist_template if template and phase matcher: returns (dist_template, dist_phase)

• metric (np.float or tuple of np.float) – matching metric output by the matcher if template matcher only (default): returns maxcorr if template and phase matcher: returns (maxcorr, perror, pdiff)

• temp_corrmap (np.ndarray) – correlation map of the naive template matcher

See also

autocnet.matcher.subpixel.subpixel_template

for list of kwargs that can be passed to the matcher

autocnet.matcher.subpixel.subpixel_phase

for list of kwargs that can be passed to the matcher

autocnet.matcher.subpixel.iterative_phase(sx, sy, dx, dy, s_img, d_img, size=(51, 51), reduction=11, convergence_threshold=1.0, max_dist=50, **kwargs)

Iteratively apply a subpixel phase matcher to source (s_img) and destination (d_img) images. The size parameter is used to set the initial search space. The algorithm is recursively applied to reduce the total search space by reduction until the convergence criteria are met. Convergence is defined as the point at which the computed shifts (x_shift,y_shift) are less than the convergence_threshold. In instances where the size is reducted to 1 pixel the algorithm terminates and returns None.

Parameters

• sx (numeric) – The x position of the center of the template to be matched to

• sy (numeric) – The y position of the center of the template to be matched to

• dx (numeric) – The x position of the center of the search to be matched from

• dy (numeric) – The y position of the center of the search to be matched to

• s_img (object) – A plio geodata object from which the template is extracted

• d_img (object) – A plio geodata object from which the search is extracted

• size (tuple) – Size of the template in the form (x,y)

• reduction (int) – With each recursive call to this func, the size is reduced by this amount

• convergence_threshold (float) – The value under which the result can shift in the x and y directions to force a break

Returns

• dx (float) – The new x value for the match in the destination (d) image

• dy (float) – The new y value for the match in the destination (d) image

• metrics (tuple) – A tuple of metrics. In the case of the phase matcher this are difference and RMSE in the phase dimension.

See also

subpixel_phase

the function that applies a single iteration of the phase matcher

autocnet.matcher.subpixel.register_to_base(pointid, base_image, cost_func=<function <lambda>>, ncg=None, geom_func='simple', geom_kwargs={'size_x': 16, 'size_y': 16}, match_func='classic', match_kwargs={}, verbose=False, **kwargs)

autocnet.matcher.subpixel.subpixel_ciratefi(sx, sy, dx, dy, s_img, d_img, search_size=251, template_size=51, **kwargs)

Uses a pattern-matcher on subsets of two images determined from the passed-in keypoints and optional sizes to compute an x and y offset from the search keypoint to the template keypoint and an associated strength.

Parameters

• sx (numeric) – The x position of the center of the template to be matched to

• sy (numeric) – The y position of the center of the template to be matched to

• dx (numeric) – The x position of the center of the search to be matched from

• dy (numeric) – The y position of the center of the search to be matched to

• s_img (object) – A plio geodata object from which the template is extracted

• d_img (object) – A plio geodata object from which the search is extracted

• search_size (int) – An odd integer for the size of the search image

• template_size (int) – A odd integer for the size of the template that is iterated over the search images

Returns

• x_offset (float) – Shift in the x-dimension

• y_offset (float) – Shift in the y-dimension

• strength (float) – Strength of the correspondence in the range [-1, 1]

autocnet.matcher.subpixel.subpixel_phase(sx, sy, dx, dy, s_img, d_img, image_size=(51, 51), **kwargs)

Apply the spectral domain matcher to a search and template image. To shift the images, the x_shift and y_shift, need to be subtracted from the center of the search image. It may also be necessary to apply the fractional pixel adjustment as well (if for example the center of the search is not an integer); this function do not manage shifting.

Parameters

• template (ndarray) – The template used to search

• search (ndarray) – The search image

Returns

• x_offset (float) – Shift in the x-dimension

• y_offset (float) – Shift in the y-dimension

• strength (tuple) – With the RMSE error and absolute difference in phase

autocnet.matcher.subpixel.subpixel_register_measure(measureid, subpixel_template_kwargs={}, size_x=100, size_y=100, cost_func=<function <lambda>>, threshold=0.005, ncg=None, **kwargs)

Given a measure, subpixel register to the reference measure of its associated point.

Parameters

• ncg (obj) – the network candidate graph that the point is associated with; used for the DB session that is able to access the point.

• measureid (int or obj) – The identifier of the measure in the DB or a Measures object

• subpixel_template_kwargs (dict) – Any keyword arguments passed to the template matcher

• cost (func) – A generic cost function accepting two arguments (x,y), where x is the distance that a point has shifted from the original, sensor identified intersection, and y is the correlation coefficient coming out of the template matcher.

• threshold (numeric) – measures with a cost <= the threshold are marked as ignore=True in the database.

autocnet.matcher.subpixel.subpixel_register_point(pointid, cost_func=<function <lambda>>, threshold=0.005, ncg=None, geom_func='simple', match_func='classic', match_kwargs={}, use_cache=False, verbose=False, **kwargs)

Given some point, subpixel register all of the measures in the point to the first measure.

Parameters

• pointid (int or obj) – The identifier of the point in the DB or a Points object

• cost_func (func) – A generic cost function accepting two arguments (x,y), where x is the distance that a point has shifted from the original, sensor identified intersection, and y is the correlation coefficient coming out of the template matcher.

• threshold (numeric) – measures with a cost <= the threshold are marked as ignore=True in the database.

• ncg (obj) – the network candidate graph that the point is associated with; used for the DB session that is able to access the point.

• geom_func (callable) – function used to tranform the source and/or destination image before running a matcher.

• match_func (callable) – subpixel matching function to use registering measures

• use_cache (bool) – If False (default) this func opens a database session and writes points and measures directly to the respective tables. If True, this method writes messages to the point_insert (defined in ncg.config) redis queue for asynchronous (higher performance) inserts.

autocnet.matcher.subpixel.subpixel_register_points(subpixel_template_kwargs={'image_size': (251, 251)}, cost_kwargs={}, threshold=0.005, Session=None)

Serial subpixel registration of all of the points in a given DB table.

Parameters

• Session (obj) – A SQLAlchemy Session factory.

• pointid (int) – The identifier of the point in the DB

• subpixel_template_kwargs (dict) – Ay keyword arguments passed to the template matcher

• cost (func) – A generic cost function accepting two arguments (x,y), where x is the distance that a point has shifted from the original, sensor identified intersection, and y is the correlation coefficient coming out of the template matcher.

• threshold (numeric) – measures with a cost <= the threshold are marked as ignore=True in the database.

autocnet.matcher.subpixel.subpixel_template(sx, sy, dx, dy, s_img, d_img, image_size=(251, 251), template_size=(51, 51), func=<function pattern_match>, verbose=False, **kwargs)

Uses a pattern-matcher on subsets of two images determined from the passed-in keypoints and optional sizes to compute an x and y offset from the search keypoint to the template keypoint and an associated strength.

Parameters

• sx (Numeric) – Source X coordinate

• sy (Numeric) – Source y coordinate

• dx (Numeric) – The desintation x coordinate

• dy (Numeric) – The destination y coordinate

• s_img (GeoDataset) – The source image GeoDataset

• d_img (GeoDataset) – The destination image GeoDataset

• image_size (tuple) – (xsize, ysize) of the image that is searched within (this should be larger than the template size)

• template_size (tuple) – (xsize, ysize) of the template to iterate over the image in order to identify the area(s) of highest correlation.

• func (callable) – The function used to pattern match

• verbose (bool) – If true, generate plots of the matches

Returns

• x_shift (float) – Shift in the x-dimension

• y_shift (float) – Shift in the y-dimension

• strength (float) – Strength of the correspondence in the range [-1, 1]

• corrmap (ndarray) – An n,m array of correlation coefficients

See also

autocnet.matcher.naive_template.pattern_match

for the kwargs that can be passed to the matcher

autocnet.matcher.naive_template.pattern_match_autoreg

for the jwargs that can be passed to the autoreg style matcher

autocnet.matcher.subpixel.subpixel_template_classic(sx, sy, dx, dy, s_img, d_img, image_size=(251, 251), template_size=(51, 51), func=<function pattern_match>, **kwargs)

Uses a pattern-matcher on subsets of two images determined from the passed-in keypoints and optional sizes to compute an x and y offset from the search keypoint to the template keypoint and an associated strength. :param sx: Source X coordinate :type sx: Numeric :param sy: Source y coordinate :type sy: Numeric :param dx: The desintation x coordinate :type dx: Numeric :param dy: The destination y coordinate :type dy: Numeric :param s_img: The source image GeoDataset :type s_img: GeoDataset :param d_img: The destination image GeoDataset :type d_img: GeoDataset :param image_size: (xsize, ysize) of the image that is searched within (this should be larger

than the template size)

Parameters

template_size (tuple) – (xsize, ysize) of the template to iterate over the image in order to identify the area(s) of highest correlation.

Returns

• x_shift (float) – Shift in the x-dimension

• y_shift (float) – Shift in the y-dimension

• strength (float) – Strength of the correspondence in the range [-1, 1]

See also

autocnet.matcher.naive_template.pattern_match

for the kwargs that can be passed to the matcher

autocnet.matcher.naive_template.pattern_match_autoreg

for the jwargs that can be passed to the autoreg style matcher

autocnet.matcher.subpixel.subpixel_transformed_template(sx, sy, dx, dy, s_img, d_img, transform, image_size=(251, 251), template_size=(51, 51), template_buffer=5, func=<function pattern_match>, verbose=False, **kwargs)

Uses a pattern-matcher on subsets of two images determined from the passed-in keypoints and optional sizes to compute an x and y offset from the search keypoint to the template keypoint and an associated strength.

Parameters

• sx (Numeric) – Source X coordinate

• sy (Numeric) – Source y coordinate

• dx (Numeric) – The desintation x coordinate

• dy (Numeric) – The destination y coordinate

• s_img (GeoDataset) – The source image GeoDataset

• d_img (GeoDataset) – The destination image GeoDataset

• transform (object) – A skiage transform object that has scale. The transform object is used to project the template into the image.

• image_size (tuple) – (xsize, ysize) of the image that is searched within (this should be larger than the template size)

• template_size (tuple) – (xsize, ysize) of the template to iterate over the image in order to identify the area(s) of highest correlation.

• template_buffer (int) – The inverse buffer applied to the transformed template image. When the warp is applied to project from the template into the image, some amount of no data exists around the edges. This variable is used to clip some number of pixels off the edges of the template. The higher the rotation the higher this value should be.

• func (callable) – The function used to pattern match

• verbose (bool) – If true, generate plots of the matches

Returns

• x_shift (float) – Shift in the x-dimension

• y_shift (float) – Shift in the y-dimension

• strength (float) – Strength of the correspondence in the range [-1, 1]

• corrmap (ndarray) – An n,m array of correlation coefficients

See also

autocnet.matcher.naive_template.pattern_match

for the kwargs that can be passed to the matcher

autocnet.matcher.naive_template.pattern_match_autoreg

for the jwargs that can be passed to the autoreg style matcher