我正在尝试使用重叠很少的图像创建全景图，但我知道相机的角度，因此我确切地知道有多少重叠，并且我知道图像的顺序，因此我知道每个图像在全景图中的位置。 作为第一步，我只是将图像连接在一起，但结果不够好。 有没有办法将位图裁剪为梯形以消除（大部分）重叠，然后在串联之前将位图拉伸回矩形？ 我知道这会在拉伸过程中产生失真，并且梯形只是位图实际需要裁剪的近似值，但我希望这足够好。
您正在寻找的技术称为图像配准 http://en.wikipedia.org/wiki/Image_registration使用仿射变换。这可以在软件中通过计算图像 B 的矩阵变换将其映射到图像 A 来实现。我假设您正在尝试使用 Windows 窗体和 GDI+ 在软件中执行此操作？那么您可用的矩阵是 3x3 矩阵，能够缩放、平移、旋转和倾斜。这通常足以创建简单的图像配准，并且我在商业软件包中成功地使用了该技术（但是 WPF）。
要使用仿射变换实现图像配准，首先需要配准一对图像中的控制点集合。由此我们可以计算 2D 变换来配准图像？我已经在 WPF 中完成了此操作，它可以使用 System.Windows.Media.Matrix 类定义 3x3 矩阵，该类具有以下构造函数：
Matrix(double m11, double m12, double m21, double m22,
double offsetX, double offsetY)
注意：GDI+有一个Matrix类，其构造函数可能不同，但原理是相同的
构造函数参数形成矩阵，如下所示：
M11 M12 0
M21 M22 0
OffsetX OffsetY 1
现在，如果输入点称为 X、Y，输出点称为 U、V，则将 X、Y 映射到 U、V 的仿射矩阵变换 T 可以计算如下：
U = X * T
[U1 V1 1] = [X1 Y1 1] [A B 0]
[U2 V2 1] = [X2 Y2 1] * [C D 0]
[U3 V3 1] = [X3 Y3 1] [Tx Ty 1]
这也可以简化如下：
U = X * T
[U1 V1] = [X1 Y1 1] [A B ]
[U2 V2] = [X2 Y2 1] * [C D ]
[U3 V3] = [X3 Y3 1] [Tx Ty]
or
X^-1 * U = T
[X1 Y1 1]^-1 [U1 V1] [A B ]
[X2 Y2 1] * [U2 V2] = [C D ]
[X3 Y3 1] [U3 V3] [Tx Ty]
用英语来说这意味着，给定图像 1 中对应于图像 2 的点 X、Y 的列表，包含 XY 点的矩阵 X 的逆矩阵乘以图像 2 中对应点的矩阵即可得出从图像 1 到图像 2 的矩阵变换
输出变换 T 包含 A、B、C、D 和 Tx、Ty，它们对应于 3x3 仿射矩阵类（上面的构造函数）中的 M11、M12、M21、M22、OffsetX、OffsetY。但是，如果 X 矩阵和 U 矩阵的点超过 3 个，则解是超定的，必须找到最小二乘拟合。这是使用 Moores-Penrose 伪逆求 X^-1 来实现的。
这在代码中意味着什么？我编写了自己的 Matrix3x3、Matrix3x2 类和控制点（x，y 点）来处理转换，然后将其应用于元素上的 WPF MatrixTransform。在 GDI+ 中，您可以通过在调用 Graphics.DrawImage 之前将 Matrix 应用到图形管道来执行相同的操作。让我们看看如何计算变换矩阵。
我们需要的第一个类是 Matrix3x3 类：
public class Matrix3x3 : ICloneable
{
#region Local Variables
private double [] coeffs;
private const int _M11 = 0;
private const int _M12 = 1;
private const int _M13 = 2;
private const int _M21 = 3;
private const int _M22 = 4;
private const int _M23 = 5;
private const int _M31 = 6;
private const int _M32 = 7;
private const int _M33 = 8;
#endregion
#region Construction
/// <summary>
/// Initializes a new instance of the <see cref="Matrix3x3"/> class.
/// </summary>
public Matrix3x3()
{
coeffs = new double[9];
}
/// <summary>
/// Initializes a new instance of the <see cref="Matrix3x3"/> class.
/// </summary>
/// <param name="coefficients">The coefficients to initialise. The number of elements of the array should
/// be equal to 9, else an exception will be thrown</param>
public Matrix3x3(double[] coefficients)
{
if (coefficients.GetLength(0) != 9)
throw new Exception("Matrix3x3.Matrix3x3()", "The number of coefficients passed in to the constructor must be 9");
coeffs = coefficients;
}
/// <summary>
/// Initializes a new instance of the <see cref="Matrix3x3"/> class.
/// </summary>
/// <param name="m11">The M11 coefficient</param>
/// <param name="m12">The M12 coefficien</param>
/// <param name="m13">The M13 coefficien</param>
/// <param name="m21">The M21 coefficien</param>
/// <param name="m22">The M22 coefficien</param>
/// <param name="m23">The M23 coefficien</param>
/// <param name="m31">The M31 coefficien</param>
/// <param name="m32">The M32 coefficien</param>
/// <param name="m33">The M33 coefficien</param>
public Matrix3x3(double m11, double m12, double m13, double m21, double m22, double m23, double m31, double m32, double m33)
{
// The 3x3 matrix is constructed as follows
//
// | M11 M12 M13 |
// | M21 M22 M23 |
// | M31 M32 M33 |
coeffs = new double[] { m11, m12, m13, m21, m22, m23, m31, m32, m33 };
}
/// <summary>
/// Initializes a new instance of the <see cref="Matrix3x3"/> class. The IAffineTransformCoefficients
/// passed in is used to populate coefficients M11, M12, M21, M22, M31, M32. The remaining column (M13, M23, M33)
/// is populated with homogenous values 0 0 1.
/// </summary>
/// <param name="affineMatrix">The IAffineTransformCoefficients used to populate M11, M12, M21, M22, M31, M32</param>
public Matrix3x3(IAffineTransformCoefficients affineTransform)
{
coeffs = new double[] { affineTransform.M11, affineTransform.M12, 0,
affineTransform.M21, affineTransform.M22, 0,
affineTransform.OffsetX, affineTransform.OffsetY, 1};
}
#endregion
#region Public Properties
/// <summary>
/// Gets or sets the M11 coefficient
/// </summary>
/// <value>The M11</value>
public double M11
{
get
{
return coeffs[_M11];
}
set
{
coeffs[_M11] = value;
}
}
/// <summary>
/// Gets or sets the M12 coefficient
/// </summary>
/// <value>The M12</value>
public double M12
{
get
{
return coeffs[_M12];
}
set
{
coeffs[_M12] = value;
}
}
/// <summary>
/// Gets or sets the M13 coefficient
/// </summary>
/// <value>The M13</value>
public double M13
{
get
{
return coeffs[_M13];
}
set
{
coeffs[_M13] = value;
}
}
/// <summary>
/// Gets or sets the M21 coefficient
/// </summary>
/// <value>The M21</value>
public double M21
{
get
{
return coeffs[_M21];
}
set
{
coeffs[_M21] = value;
}
}
/// <summary>
/// Gets or sets the M22 coefficient
/// </summary>
/// <value>The M22</value>
public double M22
{
get
{
return coeffs[_M22];
}
set
{
coeffs[_M22] = value;
}
}
/// <summary>
/// Gets or sets the M23 coefficient
/// </summary>
/// <value>The M23</value>
public double M23
{
get
{
return coeffs[_M23];
}
set
{
coeffs[_M23] = value;
}
}
/// <summary>
/// Gets or sets the M31 coefficient
/// </summary>
/// <value>The M31</value>
public double M31
{
get
{
return coeffs[_M31];
}
set
{
coeffs[_M31] = value;
}
}
/// <summary>
/// Gets or sets the M32 coefficient
/// </summary>
/// <value>The M32</value>
public double M32
{
get
{
return coeffs[_M32];
}
set
{
coeffs[_M32] = value;
}
}
/// <summary>
/// Gets or sets the M33 coefficient
/// </summary>
/// <value>The M33</value>
public double M33
{
get
{
return coeffs[_M33];
}
set
{
coeffs[_M33] = value;
}
}
/// <summary>
/// Gets the determinant of the matrix
/// </summary>
/// <value>The determinant</value>
public double Determinant
{
get
{
// |a b c|
// In general, for a 3X3 matrix |d e f|
// |g h i|
//
// The determinant can be found as follows:
// a(ei-fh) - b(di-fg) + c(dh-eg)
// Get coeffs
double a = coeffs[_M11];
double b = coeffs[_M12];
double c = coeffs[_M13];
double d = coeffs[_M21];
double e = coeffs[_M22];
double f = coeffs[_M23];
double g = coeffs[_M31];
double h = coeffs[_M32];
double i = coeffs[_M33];
double ei = e * i;
double fh = f * h;
double di = d * i;
double fg = f * g;
double dh = d * h;
double eg = e * g;
// Compute the determinant
return (a * (ei - fh)) - (b * (di - fg)) + (c * (dh - eg));
}
}
/// <summary>
/// Gets a value indicating whether this matrix is singular. If it is singular, it cannot be inverted
/// </summary>
/// <value>
/// <c>true</c> if this instance is singular; otherwise, <c>false</c>.
/// </value>
public bool IsSingular
{
get
{
return Determinant == 0;
}
}
/// <summary>
/// Gets the inverse of this matrix. If the matrix is singular, this method will throw an exception
/// </summary>
/// <value>The inverse</value>
public Matrix3x3 Inverse
{
get
{
// Taken from http://everything2.com/index.pl?node_id=1271704
// a b c
//In general, the inverse matrix of a 3X3 matrix d e f
// g h i
//is
// 1 (ei-fh) (bi-ch) (bf-ce)
// ----------------------------- x (fg-di) (ai-cg) (cd-af)
// a(ei-fh) - b(di-fg) + c(dh-eg) (dh-eg) (bg-ah) (ae-bd)
// Get coeffs
double a = coeffs[_M11];
double b = coeffs[_M12];
double c = coeffs[_M13];
double d = coeffs[_M21];
double e = coeffs[_M22];
double f = coeffs[_M23];
double g = coeffs[_M31];
double h = coeffs[_M32];
double i = coeffs[_M33];
//// Compute often used components
double ei = e * i;
double fh = f * h;
double di = d * i;
double fg = f * g;
double dh = d * h;
double eg = e * g;
double bi = b * i;
double ch = c * h;
double ai = a * i;
double cg = c * g;
double cd = c * d;
double bg = b * g;
double ah = a * h;
double ae = a * e;
double bd = b * d;
double bf = b * f;
double ce = c * e;
double cf = c * d;
double af = a * f;
// Construct the matrix using these components
Matrix3x3 tempMat = new Matrix3x3(ei - fh, ch - bi, bf - ce, fg - di, ai - cg, cd - af, dh - eg, bg - ah, ae - bd);
// Compute the determinant
double det = Determinant;
if (det == 0.0)
{
throw new Exception("Matrix3x3.Inverse", "Unable to invert the matrix as it is singular");
}
// Scale the matrix by 1/determinant
tempMat.Scale(1.0 / det);
return tempMat;
}
}
/// <summary>
/// Gets a value indicating whether this matrix is affine. This will be true if the right column
/// (M13, M23, M33) is 0 0 1
/// </summary>
/// <value><c>true</c> if this instance is affine; otherwise, <c>false</c>.</value>
public bool IsAffine
{
get
{
return (coeffs[_M13] == 0 && coeffs[_M23] == 0 && coeffs[_M33] == 1);
}
}
#endregion
#region Public Methods
/// <summary>
/// Multiplies the current matrix by the 3x3 matrix passed in
/// </summary>
/// <param name="rhs"></param>
public void Multiply(Matrix3x3 rhs)
{
// Get coeffs
double a = coeffs[_M11];
double b = coeffs[_M12];
double c = coeffs[_M13];
double d = coeffs[_M21];
double e = coeffs[_M22];
double f = coeffs[_M23];
double g = coeffs[_M31];
double h = coeffs[_M32];
double i = coeffs[_M33];
double j = rhs.M11;
double k = rhs.M12;
double l = rhs.M13;
double m = rhs.M21;
double n = rhs.M22;
double o = rhs.M23;
double p = rhs.M31;
double q = rhs.M32;
double r = rhs.M33;
// Perform multiplication. Formula taken from
// http://www.maths.surrey.ac.uk/explore/emmaspages/option1.html
coeffs[_M11] = a * j + b * m + c * p;
coeffs[_M12] = a * k + b * n + c * q;
coeffs[_M13] = a * l + b * o + c * r;
coeffs[_M21] = d * j + e * m + f * p;
coeffs[_M22] = d * k + e * n + f * q;
coeffs[_M23] = d * l + e * o + f * r;
coeffs[_M31] = g * j + h * m + i * p;
coeffs[_M32] = g * k + h * n + i * q;
coeffs[_M33] = g * l + h * o + i * r;
}
/// <summary>
/// Scales the matrix by the specified scalar value
/// </summary>
/// <param name="scalar">The scalar.</param>
public void Scale(double scalar)
{
coeffs[0] *= scalar;
coeffs[1] *= scalar;
coeffs[2] *= scalar;
coeffs[3] *= scalar;
coeffs[4] *= scalar;
coeffs[5] *= scalar;
coeffs[6] *= scalar;
coeffs[7] *= scalar;
coeffs[8] *= scalar;
}
/// <summary>
/// Makes the matrix an affine matrix by setting the right column (M13, M23, M33) to 0 0 1
/// </summary>
public void MakeAffine()
{
coeffs[_M13] = 0;
coeffs[_M23] = 0;
coeffs[_M33] = 1;
}
#endregion
#region ICloneable Members
/// <summary>
/// Creates a new object that is a copy of the current instance.
/// </summary>
/// <returns>
/// A new object that is a copy of this instance.
/// </returns>
public object Clone()
{
double[] coeffCopy = (double[])coeffs.Clone();
return new Matrix3x3(coeffCopy);
}
#endregion
#region IAffineTransformCoefficients Members
//
// NB: M11, M12, M21, M22 members of IAffineTransformCoefficients are implemented within the
// #region Public Properties directive
//
/// <summary>
/// Gets or sets the Translation Offset in the X Direction
/// </summary>
/// <value>The M31</value>
public double OffsetX
{
get
{
return coeffs[_M31];
}
set
{
coeffs[_M31] = value;
}
}
/// <summary>
/// Gets or sets the Translation Offset in the Y Direction
/// </summary>
/// <value>The M32</value>
public double OffsetY
{
get
{
return coeffs[_M32];
}
set
{
coeffs[_M32] = value;
}
}
#endregion
}
还有一个 Matrix3x2 类
public class Matrix3x2 : ICloneable
{
#region Local Variables
private double[] coeffs;
private const int _M11 = 0;
private const int _M12 = 1;
private const int _M21 = 2;
private const int _M22 = 3;
private const int _M31 = 4;
private const int _M32 = 5;
#endregion
#region Construction
/// <summary>
/// Initializes a new instance of the <see cref="Matrix3x2"/> class.
/// </summary>
public Matrix3x2()
{
coeffs = new double[6];
}
/// <summary>
/// Initializes a new instance of the <see cref="Matrix3x2"/> class.
/// </summary>
/// <param name="coefficients">The coefficients to initialise. The number of elements of the array should
/// be equal to 6, else an exception will be thrown</param>
public Matrix3x2(double[] coefficients)
{
if (coefficients.GetLength(0) != 6)
throw new Exception("Matrix3x2.Matrix3x2()",
"The number of coefficients passed in to the constructor must be 6");
coeffs = coefficients;
}
public Matrix3x2(double m11, double m12, double m21, double m22, double m31, double m32)
{
coeffs = new double[] { m11, m12, m21, m22, m31, m32 };
}
/// <summary>
/// Initializes a new instance of the <see cref="Matrix3x2"/> class. The IAffineTransformCoefficients
/// passed in is used to populate coefficients M11, M12, M21, M22, M31, M32.
/// </summary>
/// <param name="affineMatrix">The IAffineTransformCoefficients used to populate M11, M12, M21, M22, M31, M32</param>
public Matrix3x2(IAffineTransformCoefficients affineTransform)
{
coeffs = new double[] { affineTransform.M11, affineTransform.M12,
affineTransform.M21, affineTransform.M22,
affineTransform.OffsetX, affineTransform.OffsetY};
}
#endregion
#region Public Properties
/// <summary>
/// Gets or sets the M11 coefficient
/// </summary>
/// <value>The M11</value>
public double M11
{
get
{
return coeffs[_M11];
}
set
{
coeffs[_M11] = value;
}
}
/// <summary>
/// Gets or sets the M12 coefficient
/// </summary>
/// <value>The M12</value>
public double M12
{
get
{
return coeffs[_M12];
}
set
{
coeffs[_M12] = value;
}
}
/// <summary>
/// Gets or sets the M21 coefficient
/// </summary>
/// <value>The M21</value>
public double M21
{
get
{
return coeffs[_M21];
}
set
{
coeffs[_M21] = value;
}
}
/// <summary>
/// Gets or sets the M22 coefficient
/// </summary>
/// <value>The M22</value>
public double M22
{
get
{
return coeffs[_M22];
}
set
{
coeffs[_M22] = value;
}
}
/// <summary>
/// Gets or sets the M31 coefficient
/// </summary>
/// <value>The M31</value>
public double M31
{
get
{
return coeffs[_M31];
}
set
{
coeffs[_M31] = value;
}
}
/// <summary>
/// Gets or sets the M32 coefficient
/// </summary>
/// <value>The M32</value>
public double M32
{
get
{
return coeffs[_M32];
}
set
{
coeffs[_M32] = value;
}
}
#endregion
#region Public Methods
/// <summary>
/// Transforms the the ILocation passed in and returns the result in a new ILocation
/// </summary>
/// <param name="location">The location to transform</param>
/// <returns>The transformed location</returns>
public ILocation Transform(ILocation location)
{
// Perform the following equation:
//
// | x y 1 | | M11 M12 | |(xM11 + yM21 + M31) (xM12 + yM22 + M32)|
// * | M21 M22 | =
// | M31 M32 |
double x = location.X * coeffs[_M11] + location.Y * coeffs[_M21] + coeffs[_M31];
double y = location.X * coeffs[_M12] + location.Y * coeffs[_M22] + coeffs[_M32];
return new Location(x, y);
}
/// <summary>
/// Multiplies the 3x3 matrix passed in with the current 3x2 matrix
/// </summary>
/// <param name="x">The 3x3 Matrix X</param>
public void Multiply(Matrix3x3 lhs)
{
// Multiply the 3x3 matrix with the 3x2 matrix and store inside the current 2x3 matrix
//
// [a b c] [j k] [(aj + bl + cn) (ak + bm + co)]
// [d e f] * [l m] = [(dj + el + fn) (dk + em + fo)]
// [g h i] [n o] [(gj + hl + in) (gk + hm + io)]
// Get coeffs
double a = lhs.M11;
double b = lhs.M12;
double c = lhs.M13;
double d = lhs.M21;
double e = lhs.M22;
double f = lhs.M23;
double g = lhs.M31;
double h = lhs.M32;
double i = lhs.M33;
double j = coeffs[_M11];
double k = coeffs[_M12];
double l = coeffs[_M21];
double m = coeffs[_M22];
double n = coeffs[_M31];
double o = coeffs[_M32];
coeffs[_M11] = a * j + b * l + c * n;
coeffs[_M12] = a * k + b * m + c * o;
coeffs[_M21] = d * j + e * l + f * n;
coeffs[_M22] = d * k + e * m + f * o;
coeffs[_M31] = g * j + h * l + i * n;
coeffs[_M32] = g * k + h * m + i * o;
}
#endregion
#region ICloneable Members
/// <summary>
/// Creates a new object that is a copy of the current instance.
/// </summary>
/// <returns>
/// A new object that is a copy of this instance.
/// </returns>
public object Clone()
{
double[] coeffCopy = (double[])coeffs.Clone();
return new Matrix3x2(coeffCopy);
}
#endregion
#region IAffineTransformCoefficients Members
//
// NB: M11, M12, M21, M22 members of IAffineTransformCoefficients are implemented within the
// #region Public Properties directive
//
/// <summary>
/// Gets or sets the Translation Offset in the X Direction
/// </summary>
/// <value>The M31</value>
public double OffsetX
{
get
{
return coeffs[_M31];
}
set
{
coeffs[_M31] = value;
}
}
/// <summary>
/// Gets or sets the Translation Offset in the Y Direction
/// </summary>
/// <value>The M32</value>
public double OffsetY
{
get
{
return coeffs[_M32];
}
set
{
coeffs[_M32] = value;
}
}
#endregion
}
由此，我们可以使用与两个图像相对应的点列表来执行图像配准。为了澄清这意味着什么，假设您的全景照片具有某些相同的功能。两者都有大教堂尖顶，都有一棵树。配准图像 A 到 B 的点将是每个图像中对应的 X、Y 位置，即：两个图像中尖塔的 XY 位置将是一对点。
现在有了这个点列表，我们可以计算我们的变换：
public Matrix3x2 ComputeForwardTransform(IList<Point> baselineLocations, IList<Point> registerLocations)
{
if (baselineLocations.Count < 3 || registerLocations.Count < 3)
{
throw new Exception("ComputeForwardTransform()",
"Unable to compute the forward transform. A minimum of 3 control point pairs are required");
}
if (baselineLocations.Count != registerLocations.Count)
{
throw new Exception("ComputeForwardTransform()",
"Unable to compute the forward transform. The number of control point pairs in baseline and registration results must be equal");
}
// To compute
// Transform = ((X^T * X)^-1 * X^T)U = (X^T * X)^-1 (X^T * U)
// X^T * X =
// [ Sum(x_i^2) Sum(x_i*y_i) Sum(x_i) ]
// [ Sum(x_i*y_i) Sum(y_i^2) Sum(y_i) ]
// [ Sum(x_i) Sum(y_i) Sum(1)=n ]
// X^T * U =
// [ Sum(x_i*u_i) Sum(x_i*v_i) ]
// [ Sum(y_i*u_i) Sum(y_i*v_i) ]
// [ Sum(u_i) Sum(v_i) ]
IList<Point> xy = baselineLocations;
IList<Point> uv = registerLocations;
double a = 0, b = 0, c = 0, d = 0, e = 0, f = 0, g = 0, h = 0, n = xy.Count;
double p = 0, q = 0, r = 0, s = 0, t = 0, u = 0;
for (int i = 0; i < n; i++)
{
// Compute sum of squares for X^T * X
a += xy[i].X * xy[i].X;
b += xy[i].X * xy[i].Y;
c += xy[i].X;
d += xy[i].X * xy[i].Y;
e += xy[i].Y * xy[i].Y;
f += xy[i].Y;
g += xy[i].X;
h += xy[i].Y;
// Compute sum of squares for X^T * U
p += xy[i].X * uv[i].X;
q += xy[i].X * uv[i].Y;
r += xy[i].Y * uv[i].X;
s += xy[i].Y * uv[i].Y;
t += uv[i].X;
u += uv[i].Y;
}
// Create matrices from the coefficients
Matrix3x2 uMat = new Matrix3x2(p, q, r, s, t, u);
Matrix3x3 xMat = new Matrix3x3(a, b, c, d, e, f, g, h, n);
// Invert X
Matrix3x3 xInv = xMat.Inverse;
// Perform the multiplication to get the transform
uMat.Multiply(xInv);
// Matrix uMat now holds the image registration transform to go from the current result to baseline
return uMat;
}
最后，上面的内容可以这样调用：
// 其中 xy1、xy2、xy3 是第一张图像中的控制点，uv1、uv2、uv3 是 // 第二张图片中的对应对 Matrix3x2 结果 = ComputeForwardTransform(new [] {xy1, xy2, xy3}.new [] {uv1, uv2, uv3});
无论如何，我希望这对你有帮助。我意识到它不是 GDI+ 特定的，但确实详细讨论了如何使用 3x3 变换来注册图像，这可以在 GDI+ 和 WPF 中使用。实际上，我的硬盘驱动器深处有一个代码示例，如果您需要对上述内容进行澄清，我很乐意与您进行更多讨论。
Below: Demo showing stiched images
