Switch 8x16 and 16x32 filters to the type-3 versions

Multimedia / Daala - Gregory Maxwell [xiph.org] - 23 May 2013 15:28 UTC

The type-3 matrixes allow filters with somewhat higher coding gains and are also agreeable to some minor reductions in the number of operations.

The coefficients in this commit were selected using the simplex search in trans2d running on the subset3 data. The optimum selected on subset1 data is very close, no further than +/- 1 in any dimension.

This also rips out a bunch of duplicated integer prefilter code in the tools directory.

273271a Switch 8x16 and 16x32 filters to the type-3 versions.
src/filter.c | 83 +++++++-------
tools/od_defs.h | 2 +-
tools/od_filter.c | 314 +----------------------------------------------------
3 files changed, 52 insertions(+), 347 deletions(-)

Upstream: git.xiph.org

  • Share