首页 诗词 字典 板报 句子 名言 友答 励志 学校 网站地图
当前位置: 首页 > 教程频道 > 软件管理 > 软件架构设计 >

预览图生成谋略

2012-11-09 
预览图生成策略有一段时间内存老涨,跟踪到图片上传可能有问题测了一下:LoadRunner场景:每10秒加16个VUserV

预览图生成策略
有一段时间内存老涨,跟踪到图片上传可能有问题
测了一下:

LoadRunner场景:
每10秒加16个VUser
VUser总量800
分4台PC代理执行
全部robot相册表单上传图片一个功能点

分步压力测试:
1.使用现有程序
20多分钟后大概上传了5000多张1024*768的桌面图片后,
jprofiler显示内存堆占据极大,主要是byte类占据
top一下,发现java进程占了1.9G内存,而且还在涨。
负载也很高:load average: 24.18, 0.15, 0.09

2.将上传图片功能完全注释掉,只剩下空壳
半个小时后,内存保持在600M-800M
负载正常:load average: 0.07, 0.04, 0.15
正常
如此看来上传图片功能是有问题。

3.将上传图片功能留下,把其后的生成预览图注释掉。
半个小时后,内存保持在800M-900M
正常
如此看来commons-upload没问题(否则就要试一下改用cos等上传组件),
可能是生成预览图有问题

检查生成预览图程序,是直接用J2D的Graphic2D将图画出来,用iio保存的

优化:
将预览图程序重构为策略模式。
将所有可能方案作为策略实现,一个个试。
通过查资料,
找到:awt,j2d,jai,iio,jmagick,imagej,imagemanip等方案。

package com.sanook.hompy.util.image.preview;import java.awt.Dimension;import java.io.File;import java.io.IOException;import org.apache.commons.io.FileUtils;import org.apache.commons.logging.Log;import org.apache.commons.logging.LogFactory;/** *  * 此类负责与调用者打交道 *  * @author 梁飞 liangfei0201@163.com * */public class PreivewManager {protected static final Log log = LogFactory.getLog(PreivewManager.class);private PreviewGenerator previewGenerator;public PreivewManager(PreviewGenerator previewGenerator) {this.previewGenerator = previewGenerator;}/** * 一些通用的处理就在这先做了 *  * @param source * @param target * @param width * @param height * @throws IOException */public void generatePreview(String source, String target, int width, int height) throws IOException {if (source == null || ! new File(source).exists()) {return;}checkDir(target);Dimension d = previewGenerator.getSize(source);int w = (int)d.getWidth();int h = (int)d.getHeight();if (w <= width && h <= height) {copy(source, target); //如果图片较小,就直接copy过去} else {//同比缩放if (w > width || h > height) {if(w * height > h * width){height = h * width / w;}else{width = w * height / h;}} else {width = w;height = h;}previewGenerator.generate(source, target, width, height);}return ;}private void checkDir(String target) {File dir = new File(target).getParentFile();if (! dir.exists()) {dir.mkdirs();log.warn(dir.getAbsolutePath() + " not exists! already auto made!");}}private void copy(String source, String target) throws IOException {FileUtils.copyFile(new File(source), new File(target));}}


package com.sanook.hompy.util.image.preview;import java.awt.Dimension;import java.io.IOException;/** *  * 预览图生成的策略接口 *  * @author 梁飞 liangfei0201@163.com * */public interface PreviewGenerator {/** * 主要用于判断是否为大图,小图就copy * @param source 源图片位置 * @return Dimension 图片的大小 * @throws IOException */public Dimension getSize(String source) throws IOException;/** * 处理生成预览图的策略方法 * 大小判定,比例调整已在PreivewManager,请直接使用width,height * @param source 源图片位置 * @param target 保存预览图位置 * @param width 预览图宽度 * @param height 预览图高度 * @throws IOException */public void generate(String source, String target, int width, int height) throws IOException;}


package com.sanook.hompy.util.image.preview;import java.awt.image.BufferedImage;import java.io.IOException;/** *  * BufferedImage 的读写处理策略接口 *  * @author 梁飞 liangfei0201@163.com * */public interface ImageProvider {public BufferedImage readImage(String source) throws IOException;public void saveImage(BufferedImage image, String target) throws IOException;}


package com.sanook.hompy.util.image.preview;import java.awt.image.BufferedImage;import java.io.BufferedInputStream;import java.io.BufferedOutputStream;import java.io.FileInputStream;import java.io.FileOutputStream;import java.io.IOException;import com.sun.image.codec.jpeg.JPEGCodec;import com.sun.image.codec.jpeg.JPEGImageDecoder;import com.sun.image.codec.jpeg.JPEGImageEncoder;/** *  * Sun的非标准JPEG处理类,没ImageIO之前,全靠它撑着 *  * @author 梁飞 liangfei0201@163.com * */public class CodecImageProvider implements ImageProvider {public BufferedImage readImage(String source) throws IOException {BufferedInputStream bis = null;try {bis = new BufferedInputStream(new FileInputStream(source));JPEGImageDecoder decoder = JPEGCodec.createJPEGDecoder(bis);return decoder.decodeAsBufferedImage();} finally {if (bis != null) {bis.close();}}}public void saveImage(BufferedImage image, String target) throws IOException {BufferedOutputStream bos = null;try {bos = new BufferedOutputStream(new FileOutputStream(target));JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(bos);encoder.encode(image);} finally {if (bos != null) {bos.close();}}}}


package com.sanook.hompy.util.image.preview;import java.awt.image.BufferedImage;import java.io.File;import java.io.IOException;import java.util.Iterator;import javax.imageio.IIOImage;import javax.imageio.ImageIO;import javax.imageio.ImageTypeSpecifier;import javax.imageio.ImageWriteParam;import javax.imageio.ImageWriter;import javax.imageio.stream.ImageOutputStream;/** *  * 标准的Java实现图形处理包,处理性能待考量 *  * @author 梁飞 liangfei0201@163.com * */public class IioImageProvider implements ImageProvider {public BufferedImage readImage(String source) throws IOException {return ImageIO.read(new File(source));}public void saveImage(BufferedImage image, String target) throws IOException {File targetFile = new File(target);ImageWriter writer = null;ImageOutputStream outputStream = null;try {ImageTypeSpecifier type = ImageTypeSpecifier.createFromRenderedImage(image);Iterator iterator = ImageIO.getImageWriters(type, "JPEG");if (!iterator.hasNext()) {return;}writer = (ImageWriter) iterator.next();IIOImage iioImage = new IIOImage(image, null, null);ImageWriteParam param = writer.getDefaultWriteParam();param.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);param.setCompressionQuality(1.0f);outputStream = ImageIO.createImageOutputStream(targetFile);writer.setOutput(outputStream);writer.write(null, iioImage, param);} finally {if (outputStream != null) {outputStream.close();}if (writer != null) {writer.abort();}}}}


package com.sanook.hompy.util.image.preview;import java.awt.Dimension;import java.awt.Image;import java.awt.Toolkit;import java.io.IOException;/** *  * 最原始的方法,Toolkit一般用于桌面加载图片,调用的地方到处是ImageObserver *  * @author 梁飞 liangfei0201@163.com * */public class AwtPreviewGenerator implements PreviewGenerator {public Dimension getSize(String source) throws IOException {Image image = Toolkit.getDefaultToolkit().getImage(source);return new Dimension(image.getWidth(null), image.getHeight(null));}public void generate(String source, String target, int width, int height)throws IOException {Image image = Toolkit.getDefaultToolkit().getImage(source);image = image.getScaledInstance(width, height, Image.SCALE_FAST);// TODO Image保存方式待定// imageProvider.saveImage((BufferedImage)image, target);}}


package com.sanook.hompy.util.image.preview;import java.awt.Color;import java.awt.Dimension;import java.awt.Graphics2D;import java.awt.RenderingHints;import java.awt.image.BufferedImage;import java.io.IOException;/** *  * 这个是自己操大刀,用Graphics直接画出来,再保存 *  * 注:重构前,就此类+IioImageProvider的实现方式 *  * @author 梁飞 liangfei0201@163.com * */public class J2dPreviewGenerator implements PreviewGenerator {private ImageProvider imageProvider;public J2dPreviewGenerator(ImageProvider imageProvider) {this.imageProvider = imageProvider;}public Dimension getSize(String source) throws IOException {BufferedImage image = imageProvider.readImage(source);return new Dimension(image.getWidth(), image.getHeight());}public void generate(String source, String target, int width, int height) throws IOException {BufferedImage sourceImage = imageProvider.readImage(source);BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);Graphics2D g = null;try {g = (Graphics2D) image.createGraphics();g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);g.setRenderingHint(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_QUALITY);// TODO hints待调整 ...g.setColor(Color.white);g.fillRect(0, 0, width, height);g.drawImage(sourceImage, 0, 0, width, height, null);} finally {if (g != null) {g.dispose();}}imageProvider.saveImage(image, target);}}


package com.sanook.hompy.util.image.preview;import java.awt.Dimension;import java.awt.RenderingHints;import java.awt.geom.AffineTransform;import java.awt.image.AffineTransformOp;import java.awt.image.BufferedImage;import java.io.IOException;/** *  * 标准的Image处理的装饰器模式实现,感觉比直接操Graphic好些 *  * @author 梁飞 liangfei0201@163.com * */public class OpPreviewGenerator implements PreviewGenerator {private ImageProvider imageProvider;public OpPreviewGenerator(ImageProvider imageProvider) {this.imageProvider = imageProvider;}public Dimension getSize(String source) throws IOException {BufferedImage image = imageProvider.readImage(source);return new Dimension(image.getWidth(), image.getHeight());}public void generate(String source, String target, int width, int height) throws IOException {BufferedImage sourceImage = imageProvider.readImage(source);AffineTransform affineTransform = new AffineTransform();affineTransform.scale(width, height);RenderingHints hints = new RenderingHints(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_DEFAULT);// TODO hints待调整 ...AffineTransformOp affineTransformOp = new AffineTransformOp(affineTransform, hints);BufferedImage image = new BufferedImage(width, height, sourceImage.getType());image = affineTransformOp.filter(sourceImage, image);imageProvider.saveImage(image, target);}}


package com.sanook.hompy.util.image.preview;import java.awt.Dimension;import java.awt.image.renderable.ParameterBlock;import java.awt.image.renderable.RenderableImage;import java.io.File;import java.io.FileOutputStream;import java.io.IOException;import javax.media.jai.JAI;import javax.media.jai.ParameterBlockJAI;import javax.media.jai.PlanarImage;import com.sun.media.jai.codec.ImageCodec;import com.sun.media.jai.codec.ImageEncoder;import com.sun.media.jai.codec.JPEGEncodeParam;/** *  * ImageIO的前身,但保持独立发展,比ImageIO绑定JDK,更新速度快多了 *  * @author 梁飞 liangfei0201@163.com * */public class JaiPreviewGenerator implements PreviewGenerator {public Dimension getSize(String source) throws IOException {ParameterBlockJAI loadPB = new ParameterBlockJAI("fileload");loadPB.setParameter("filename", source);PlanarImage image = JAI.create("fileload", loadPB);return new Dimension(image.getWidth(), image.getHeight());}public void generate(String source, String target, int width, int height) throws IOException {ParameterBlock pb = new ParameterBlock();pb.addSource(source);RenderableImage render = JAI.createRenderable("renderable", pb);PlanarImage image = (PlanarImage) render.createScaledRendering(width, height, null);save(image, target);}private void save(PlanarImage image, String target) throws IOException {File targetFile = new File(target);FileOutputStream out = null;try {JPEGEncodeParam param = new JPEGEncodeParam();param.setQuality(1.00f);out = new FileOutputStream(targetFile);ImageEncoder encoder = ImageCodec.createImageEncoder("JPEG", out, param);encoder.encode(image);} finally {if (out != null) {out.close();}}}}


package com.sanook.hompy.util.image.preview;import java.awt.Dimension;import java.io.IOException;import magick.CompressionType;import magick.ImageInfo;import magick.MagickApiException;import magick.MagickException;import magick.MagickImage;/** *  * Unix下最著名的ImageMagicK库的Java调用 * JMagicK并不处理什么,只是ImageMagicK到Java的一个jni接口,其方法大部分是native的 *  * 注:已放在正式环境下run了 *  * @author 梁飞 liangfei0201@163.com * */public class JmagickPreviewGenerator implements PreviewGenerator {public JmagickPreviewGenerator() {System.setProperty("jmagick.systemclassloader", "no");}public Dimension getSize(String source) throws IOException {try {ImageInfo info = new ImageInfo(source);MagickImage image = new MagickImage(info);return image.getDimension();} catch (MagickApiException ex) {throw new IOException(ex.getMessage());} catch (MagickException ex) {throw new IOException(ex.getMessage());}}public void generate(String source, String target, int width, int height) throws IOException {try {ImageInfo info = new ImageInfo(source);MagickImage image = new MagickImage(info);MagickImage canvasImage = image.scaleImage(width, height);save(canvasImage, target);} catch (MagickApiException ex) {ex.printStackTrace();throw new IOException(ex.getMessage());} catch (MagickException ex) {ex.printStackTrace();throw new IOException(ex.getMessage());}}private void save(MagickImage image, String target) throws IOException {try {image.setCompression(CompressionType.JPEGCompression);image.setFileName(target);image.writeImage(new ImageInfo());} catch (MagickApiException ex) {ex.printStackTrace();throw new IOException(ex.getMessage());} catch (MagickException ex) {ex.printStackTrace();throw new IOException(ex.getMessage());}}}


package com.sanook.hompy.util.image.preview;import java.awt.Dimension;import java.io.IOException;/** * 在生物方面用的较多,如细胞透视图等的处理 *  * @author 梁飞 liangfei0201@163.com * */public class ImagejPreviewGenerator implements PreviewGenerator {public Dimension getSize(String source) throws IOException {// TODO 没写完return null;}public void generate(String source, String target, int width, int height) throws IOException {// TODO 没写完}}


结果:
用Op比直接用Graphic要好些,
JMagicK的内存占有会低很多,
速度也比iio快3到5倍。
但负载还是偏高。

资源:
ImageMagicK: http://www.imagemagick.org/ (提供C/C++原生接口)
JMagicK: http://www.yeo.id.au/jmagick/  (Java接口)
RMagicK: http://rmagick.rubyforge.org/ (Ruby接口)
也支持其它语言:
LISP: http://common-lisp.net/project/cl-magick/
PASCAL: http://wiki.lazarus.freepascal.org/index.html/PascalMagick
PHP: http://www.magickwand.org/download/php/
PYTHON: http://www.imagemagick.org/download/python/
TCL: http://tclmagick.sourceforge.net/

BTW:
ImageMagicK和JMagicK安装很麻烦哦,最好都用源代码包编译安装,rpm包安装常出错。
package com.sanook.hompy.util.image.preview;import java.awt.Color;import java.awt.Dimension;import java.awt.Graphics2D;import java.awt.RenderingHints;import java.awt.image.BufferedImage;import java.io.IOException;import com.fmsware.gif.AnimatedGifEncoder;import com.fmsware.gif.GifDecoder;/** * * GIF动画的缩略实现策略 * * @see PreviewGenerator * * @author 梁飞 liangfei0201@163.com * */public class GifPreviewGenerator implements PreviewGenerator {private PreviewGenerator nonGifPreviewGenerator;public GifPreviewGenerator(PreviewGenerator nonGifPreviewGenerator) {this.nonGifPreviewGenerator = nonGifPreviewGenerator;}public Dimension getSize(String source) throws IOException {if (! isGif(source)) {return nonGifPreviewGenerator.getSize(source);}GifDecoder decoder = new GifDecoder();int status = decoder.read(source);if (status != GifDecoder.STATUS_OK) {throw new IOException("read image " + source + " error!");}return decoder.getFrameSize();}public void generate(String source, String target, int width, int height) throws IOException {if (! isGif(source)) {nonGifPreviewGenerator.generate(source, target, width, height);return ;}GifDecoder decoder = new GifDecoder();int status = decoder.read(source);if (status != GifDecoder.STATUS_OK) {throw new IOException("read image " + source + " error!");}AnimatedGifEncoder encoder = new AnimatedGifEncoder();encoder.start(target);encoder.setRepeat(decoder.getLoopCount());for (int i = 0; i < decoder.getFrameCount(); i ++) {encoder.setDelay(decoder.getDelay(i));encoder.addFrame(scale(decoder.getFrame(i), width, height));}encoder.finish();}// TODO GIF的格式待改为ContentType元数据判定private boolean isGif(String source) {if (source == null || source.length() < 4) {return false;}return ".gif".equalsIgnoreCase(source.substring(source.length() - 4));}// TODO BufferedImage的缩略,多个PreviewGenerator重复,待重构private BufferedImage scale(BufferedImage sourceImage, int width, int height) {BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);Graphics2D g = null;try {g = image.createGraphics();g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);g.setRenderingHint(RenderingHints.KEY_RENDERING, RenderingHints.VALUE_RENDER_QUALITY);// TODO Hint待调整g.setColor(Color.white);g.fillRect(0, 0, width, height);g.drawImage(sourceImage, 0, 0, width, height, null);} finally {if (g != null) {g.dispose();}}return image;}}
4 楼 galaxystar 2007-01-09   jmagic不错! 5 楼 yangzhihuan 2007-04-05   我在遇到了使用ImageIO.read(InputSteram in)方法时,JVM狂吃内存的情况,使用JProdfile看的时候是byte[]狂占内存,而且还是用完了之后不释放,请问题楼主这个问题能否解决?到底是不是ImageIO本身的问题呢?

热点排行