예전에 하던 플젝 중에 40×30 데이터를 가지고 와서 640×480 해상도로 확대해서 화면에 뿌려줘야 하는 것이 있었음.
이미지 데이터는 아니고 double형 데이터인데 이걸 단순 확대 하면 구멍 혹은 격자가 생기니 부드럽게 늘려야 하는것이였음.
선형보간하면 된다고 생각했는데 뭔가 잘못짠거겠지만 좀 이미지가 틀어지는거 같아서 걍 인터넷 검색해서 소스를 찾았음
/**
* Bilinear resize ARGB image.
* pixels is an array of size w * h.
* Target dimension is w2 * h2.
* w2 * h2 cannot be zero.
*
* @param pixels Image pixels.
* @param w Image width.
* @param h Image height.
* @param w2 New width.
* @param h2 New height.
* @return New array with size w2 * h2.
*/
public int[] resizeBilinear(int[] pixels, int w, int h, int w2, int h2) {
int[] temp = new int[w2*h2] ;
int a, b, c, d, x, y, index ;
float x_ratio = ((float)(w-1))/w2 ;
float y_ratio = ((float)(h-1))/h2 ;
float x_diff, y_diff, blue, red, green ;
int offset = 0 ;
for (int i=0;i<h2;i++) {
for (int j=0;j<w2;j++) {
x = (int)(x_ratio * j) ;
y = (int)(y_ratio * i) ;
x_diff = (x_ratio * j) - x ;
y_diff = (y_ratio * i) - y ;
index = (y*w+x) ;
a = pixels[index] ;
b = pixels[index+1] ;
c = pixels[index+w] ;
d = pixels[index+w+1] ;
// blue element
// Yb = Ab(1-w)(1-h) + Bb(w)(1-h) + Cb(h)(1-w) + Db(wh)
blue = (a&0xff)*(1-x_diff)*(1-y_diff) + (b&0xff)*(x_diff)*(1-y_diff) +
(c&0xff)*(y_diff)*(1-x_diff) + (d&0xff)*(x_diff*y_diff);
// green element
// Yg = Ag(1-w)(1-h) + Bg(w)(1-h) + Cg(h)(1-w) + Dg(wh)
green = ((a>>8)&0xff)*(1-x_diff)*(1-y_diff) + ((b>>8)&0xff)*(x_diff)*(1-y_diff) +
((c>>8)&0xff)*(y_diff)*(1-x_diff) + ((d>>8)&0xff)*(x_diff*y_diff);
// red element
// Yr = Ar(1-w)(1-h) + Br(w)(1-h) + Cr(h)(1-w) + Dr(wh)
red = ((a>>16)&0xff)*(1-x_diff)*(1-y_diff) + ((b>>16)&0xff)*(x_diff)*(1-y_diff) +
((c>>16)&0xff)*(y_diff)*(1-x_diff) + ((d>>16)&0xff)*(x_diff*y_diff);
temp[offset++] =
0xff000000 | // hardcode alpha
((((int)red)<<16)&0xff0000) |
((((int)green)<<8)&0xff00) |
((int)blue) ;
}
}
return temp ;
}
출처: http://tech-algorithm.com/articles/bilinear-image-scaling
뭐 이걸 그대로 가져다 쓴건 아니고 저것처럼 RGB값이 아니라 데이터값이라 좀 변형하긴 했음.
근데 이게 은근히 느림. 당시 갑 님이 원했던게 25fps 고정인데 영상데이터랑 측정데이터랑 같이 묶음으로 오는게 아니고 별도의 스트림으로 따로오니 오는 족족 실시간 처리를 하지 않으면 싱크가 틀어지는 문제가 생김.
한 프레임 데이터 처리하는데 40ms 가 넘으면 안되는 상황이 되버렸음.
당시 NDK 를 쓰면 빠르다 라는 말이 있어서 시도는 해봤는데 그때 뭘 잘못한건진 몰라도 NDK 함수로 데이터 넘기는것만 해도 40ms가 넘는걸로 나와서 아예 전부 C 내부에서 데이터 받아서 처리하고 화면 출력하는거 아니면 기간내에 못만들겠다 라고 판단해서 걍 자바로 다 짬.
결과는 갤럭시 S6에서 돌려도 프레임이 5~8프레임 나옴.
그때부터 프레임 확보를 위해 이런저런 삽질을 했는데 뭐 그건 여기 쓸 내용은 아니니 생략하고 아무튼 그거 생각나서 NDK로 다시 도전해봄.
(다들 NDK가 빠르다는데 나만 안되는건 내가 뭔가 잘못했을거야 라는 생각하에)
결론
Moto G 1세대인 내 폰에서 40×30 ==> 640×480으로 변환하는데 걸린 시간은 평균 50ms (100번 변환해서 평균값) (실제 플젝은 갤럭시 S6d에서 40×30을 280×210으로 처리했었음)
오호 이거해보니 당시 플젝 할때 좀 더 알아보고 NDK를 썼으면 좀 더 선명한 화면에 좀더 안정적인 프레임을 유지할수 있지 않았을까 생각
package kr.co.linsoo.ndktest1;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.TextView;
import java.util.Arrays;
public class MainActivity extends AppCompatActivity {
static {
System.loadLibrary("resizeBilinearNDK");
}
public native void resizeBilinearNDK(int[] inPixels, int[] outPixels, int w, int h, int w2, int h2);
Bitmap mTestBitmap;
Button m_ButtonJAVA, m_ButtonNDK, m_ButtonClear;
TextView mTextView01, m_TextViewJAVA, m_TextViewNDK;
Boolean m_bTesting = false;
ImageView m_ImgViewSRC = null, m_ImgViewOutput = null;
int[] m_iArrSrc = null;
int[] m_iArrOutput = null;
int m_iSrcWidth = 0, m_iSrcHeight = 0;
int m_iOutputWidth = 0, m_iOutputHeight = 0;
int scaleR = 16;
@Override
protected void onResume(){
super.onResume();
m_bTesting = false;
}
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
m_ImgViewSRC = (ImageView) findViewById(R.id.imageViewSRC);
m_ImgViewOutput = (ImageView) findViewById(R.id.imageViewOutput);
mTextView01 = (TextView) findViewById(R.id.textView);
m_TextViewJAVA = (TextView) findViewById(R.id.textViewJava);
m_TextViewNDK = (TextView) findViewById(R.id.textViewNDK);
m_bTesting = false;
mTestBitmap = BitmapFactory.decodeResource(getResources(), R.drawable.test);
m_iSrcWidth = mTestBitmap.getWidth();
m_iSrcHeight = mTestBitmap.getHeight();
m_iOutputWidth = m_iSrcWidth*scaleR;
m_iOutputHeight = m_iSrcHeight*scaleR;
m_iArrSrc = new int[m_iSrcWidth * m_iSrcHeight];
m_iArrOutput = new int[m_iOutputWidth * m_iOutputHeight];
mTestBitmap.getPixels(m_iArrSrc,0,m_iSrcWidth,0,0,m_iSrcWidth,m_iSrcHeight);
//기본이미지 보여주기
m_ImgViewSRC.setImageBitmap( mTestBitmap);
//기본이미지 해상도 표시
mTextView01.setText(String.format("width=%d height=%d", mTestBitmap.getWidth(), mTestBitmap.getHeight()));
m_ButtonJAVA = (Button) findViewById(R.id.buttonJAVA);
m_ButtonJAVA.setOnClickListener(new View.OnClickListener(){
@Override
public void onClick(View view) {
StartJAVATest();
}
});
m_ButtonNDK = (Button) findViewById(R.id.buttonNDK);
m_ButtonNDK.setOnClickListener(new View.OnClickListener(){
@Override
public void onClick(View view) {
StartNDKTest();
}
});
m_ButtonClear = (Button) findViewById(R.id.buttonClear);
m_ButtonClear.setOnClickListener(new View.OnClickListener(){
@Override
public void onClick(View view) {
clearImage();
}
});
}
public void clearImage(){
Arrays.fill(m_iArrOutput,0);
m_ImgViewOutput.setImageBitmap(Bitmap.createBitmap(m_iArrOutput,m_iOutputWidth,m_iOutputHeight, Bitmap.Config.ARGB_8888));
}
public void StartNDKTest(){
if( m_bTesting == true)
return;
m_bTesting = true;
clearImage();
long stNDK = System.currentTimeMillis();
for(int i=0; i<100; i++) {
resizeBilinearNDK(m_iArrSrc, m_iArrOutput, m_iSrcWidth, m_iSrcHeight, m_iSrcWidth * scaleR, m_iSrcHeight * scaleR);
}
long etNDK = System.currentTimeMillis();
long totalTime = etNDK-stNDK;
m_ImgViewOutput.setImageBitmap(Bitmap.createBitmap(m_iArrOutput,m_iOutputWidth,m_iOutputHeight, Bitmap.Config.ARGB_8888));
m_TextViewNDK.setText(String.format("[NDK] fps = %d, Avr = %d ms", 1000/(totalTime/100), (totalTime/100) ));
m_bTesting = false;
}
public void StartJAVATest(){
if( m_bTesting == true)
return;
m_bTesting = true;
clearImage();
long stJava = System.currentTimeMillis();
for(int i=0; i<100; i++){
resizeBilinear(m_iArrSrc, m_iArrOutput, m_iSrcWidth, m_iSrcHeight, m_iSrcWidth *scaleR, m_iSrcHeight *scaleR);
}
long etJava = System.currentTimeMillis();
long totalTime = etJava-stJava;
m_ImgViewOutput.setImageBitmap(Bitmap.createBitmap(m_iArrOutput,m_iOutputWidth,m_iOutputHeight, Bitmap.Config.ARGB_8888));
m_TextViewJAVA.setText(String.format("[Java] fps = %d, Avr = %d ms", 1000/(totalTime/100), (totalTime/100) ));
m_bTesting = false;
}
public void resizeBilinear(int[] inPixels, int[] outPixels, int w, int h, int w2, int h2) {
int a, b, c, d, x, y, index ;
float x_ratio = ((float)(w-1))/w2 ;
float y_ratio = ((float)(h-1))/h2 ;
float x_diff, y_diff, blue, red, green ;
int offset = 0 ;
for (int i=0;i<h2;i++) {
for (int j=0;j<w2;j++) {
x = (int)(x_ratio * j) ;
y = (int)(y_ratio * i) ;
x_diff = (x_ratio * j) - x ;
y_diff = (y_ratio * i) - y ;
index = (y*w+x) ;
a = inPixels[index] ;
b = inPixels[index+1] ;
c = inPixels[index+w] ;
d = inPixels[index+w+1] ;
// blue element
// Yb = Ab(1-w)(1-h) + Bb(w)(1-h) + Cb(h)(1-w) + Db(wh)
blue = (a&0xff)*(1-x_diff)*(1-y_diff) + (b&0xff)*(x_diff)*(1-y_diff) +
(c&0xff)*(y_diff)*(1-x_diff) + (d&0xff)*(x_diff*y_diff);
// green element
// Yg = Ag(1-w)(1-h) + Bg(w)(1-h) + Cg(h)(1-w) + Dg(wh)
green = ((a>>8)&0xff)*(1-x_diff)*(1-y_diff) + ((b>>8)&0xff)*(x_diff)*(1-y_diff) +
((c>>8)&0xff)*(y_diff)*(1-x_diff) + ((d>>8)&0xff)*(x_diff*y_diff);
// red element
// Yr = Ar(1-w)(1-h) + Br(w)(1-h) + Cr(h)(1-w) + Dr(wh)
red = ((a>>16)&0xff)*(1-x_diff)*(1-y_diff) + ((b>>16)&0xff)*(x_diff)*(1-y_diff) +
((c>>16)&0xff)*(y_diff)*(1-x_diff) + ((d>>16)&0xff)*(x_diff*y_diff);
outPixels[offset++] =
0xff000000 | // hardcode alpha
((((int)red)<<16)&0xff0000) |
((((int)green)<<8)&0xff00) |
((int)blue) ;
}
}
}
}
MainActivity.java
//
// Created by linsoo on 2016. 8. 30..
//
#include "kr_co_linsoo_ndktest1_MainActivity.h"
JNIEXPORT void JNICALL Java_kr_co_linsoo_ndktest1_MainActivity_resizeBilinearNDK
(JNIEnv *env, jobject obj, jintArray inPixels, jintArray outPixels, jint w, jint h, jint w2, jint h2) {
jint *pInPixels, *pOutPixels;
int size;
jboolean isCopy = false;
size = env->GetArrayLength(inPixels);
pInPixels = env->GetIntArrayElements(inPixels, &isCopy);
size = env->GetArrayLength(outPixels);
pOutPixels = env->GetIntArrayElements(outPixels, &isCopy);
int a, b, c, d, x, y, index;
float x_ratio = ((float) (w - 1)) / w2;
float y_ratio = ((float) (h - 1)) / h2;
float x_diff, y_diff, blue, red, green;
int offset = 0;
for (int i = 0; i < h2; i++) {
for (int j = 0; j < w2; j++) {
x = (int) (x_ratio * j);
y = (int) (y_ratio * i);
x_diff = (x_ratio * j) - x;
y_diff = (y_ratio * i) - y;
index = (y * w + x);
a = pInPixels[index];
b = pInPixels[index+1] ;
c = pInPixels[index+w] ;
d = pInPixels[index+w+1] ;
// blue element
// Yb = Ab(1-w)(1-h) + Bb(w)(1-h) + Cb(h)(1-w) + Db(wh)
blue = (a&0xff)*(1-x_diff)*(1-y_diff) + (b&0xff)*(x_diff)*(1-y_diff) +
(c&0xff)*(y_diff)*(1-x_diff) + (d&0xff)*(x_diff*y_diff);
// green element
// Yg = Ag(1-w)(1-h) + Bg(w)(1-h) + Cg(h)(1-w) + Dg(wh)
green = ((a>>8)&0xff)*(1-x_diff)*(1-y_diff) + ((b>>8)&0xff)*(x_diff)*(1-y_diff) +
((c>>8)&0xff)*(y_diff)*(1-x_diff) + ((d>>8)&0xff)*(x_diff*y_diff);
// red element
// Yr = Ar(1-w)(1-h) + Br(w)(1-h) + Cr(h)(1-w) + Dr(wh)
red = ((a>>16)&0xff)*(1-x_diff)*(1-y_diff) + ((b>>16)&0xff)*(x_diff)*(1-y_diff) +
((c>>16)&0xff)*(y_diff)*(1-x_diff) + ((d>>16)&0xff)*(x_diff*y_diff);
pOutPixels[offset++] =
0xff000000 | // hardcode alpha
((((int)red)<<16)&0xff0000) |
((((int)green)<<8)&0xff00) |
((int)blue) ;
}
}
env->ReleaseIntArrayElements(inPixels, pInPixels, 0);
env->ReleaseIntArrayElements(outPixels, pOutPixels, 0);
}
kr_co_linsoo_ndktest1_MainActivity.cpp
/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class kr_co_linsoo_ndktest1_MainActivity */
#ifndef _Included_kr_co_linsoo_ndktest1_MainActivity
#define _Included_kr_co_linsoo_ndktest1_MainActivity
#ifdef __cplusplus
extern "C" {
#endif
/*
* Class: kr_co_linsoo_ndktest1_MainActivity
* Method: resizeBilinearNDK
* Signature: ([I[IIIII)V
*/
JNIEXPORT void JNICALL Java_kr_co_linsoo_ndktest1_MainActivity_resizeBilinearNDK
(JNIEnv *, jobject, jintArray, jintArray, jint, jint, jint, jint);
#ifdef __cplusplus
}
#endif
#endif
kr_co_linsoo_ndktest1_MainActivity.h
Source download : NDKTest1
ps. 이미지 워터마크 박을때 고양시에서 배포하는 고양체 썼는데 일부 기호는 고양이가 나오네 ㅋㅋㅋ
답글 남기기