Upload a File Through an Http Form, via Multipartentitybuilder, with a Progress Bar

Upload a file through an HTTP form, via MultipartEntityBuilder, with a progress bar

The winning code (in spectacular Java-Heresy(tm) style) is:

public static String postFile(String fileName, String userName, String password, String macAddress) throws Exception {

    HttpClient client = new DefaultHttpClient();
    HttpPost post = new HttpPost(SERVER + "uploadFile");
    MultipartEntityBuilder builder = MultipartEntityBuilder.create();        
    builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);

    final File file = new File(fileName);
    FileBody fb = new FileBody(file);

    builder.addPart("file", fb);  
    builder.addTextBody("userName", userName);
    builder.addTextBody("password", password);
    builder.addTextBody("macAddress",  macAddress);
    final HttpEntity yourEntity = builder.build();

    class ProgressiveEntity implements HttpEntity {
        @Override
        public void consumeContent() throws IOException {
            yourEntity.consumeContent();                
        }
        @Override
        public InputStream getContent() throws IOException,
                IllegalStateException {
            return yourEntity.getContent();
        }
        @Override
        public Header getContentEncoding() {             
            return yourEntity.getContentEncoding();
        }
        @Override
        public long getContentLength() {
            return yourEntity.getContentLength();
        }
        @Override
        public Header getContentType() {
            return yourEntity.getContentType();
        }
        @Override
        public boolean isChunked() {             
            return yourEntity.isChunked();
        }
        @Override
        public boolean isRepeatable() {
            return yourEntity.isRepeatable();
        }
        @Override
        public boolean isStreaming() {             
            return yourEntity.isStreaming();
        } // CONSIDER put a _real_ delegator into here!

        @Override
        public void writeTo(OutputStream outstream) throws IOException {

            class ProxyOutputStream extends FilterOutputStream {
                /**
                 * @author Stephen Colebourne
                 */

                public ProxyOutputStream(OutputStream proxy) {
                    super(proxy);    
                }
                public void write(int idx) throws IOException {
                    out.write(idx);
                }
                public void write(byte[] bts) throws IOException {
                    out.write(bts);
                }
                public void write(byte[] bts, int st, int end) throws IOException {
                    out.write(bts, st, end);
                }
                public void flush() throws IOException {
                    out.flush();
                }
                public void close() throws IOException {
                    out.close();
                }
            } // CONSIDER import this class (and risk more Jar File Hell)

            class ProgressiveOutputStream extends ProxyOutputStream {
                public ProgressiveOutputStream(OutputStream proxy) {
                    super(proxy);
                }
                public void write(byte[] bts, int st, int end) throws IOException {

                    // FIXME  Put your progress bar stuff here!

                    out.write(bts, st, end);
                }
            }

            yourEntity.writeTo(new ProgressiveOutputStream(outstream));
        }

    };
    ProgressiveEntity myEntity = new ProgressiveEntity();

    post.setEntity(myEntity);
    HttpResponse response = client.execute(post);        

    return getContent(response);

} 

public static String getContent(HttpResponse response) throws IOException {
    BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
    String body = "";
    String content = "";

    while ((body = rd.readLine()) != null) 
    {
        content += body + "\n";
    }
    return content.trim();
}

#  NOTE ADDED LATER: as this blasterpiece gets copied into various code lineages, 
#  The management reminds the peanut gallery that "Java-Heresy" crack was there
#  for a reason, and (as commented) most of that stuff can be farmed out to off-
#  the-shelf jar files and what-not. That's for the java lifers to tool up. This
#  pristine hack shall remain obviousized for education, and for use in a pinch.

#  What are the odds??

HttpUrlConnection multipart file upload with progressBar

Because you have to deal with upload, I'd suppose most time is taken when doing entity.writeTo(os);. Maybe the first contact to the server takes some time as well (DNS resolution, SSL-handshake, ...). The markers you set for "the real upload" are not correct IMO.

Now it depends on your Multipart-library, whether you can intercept writeTo. If it is clever and resource-efficient, it's iterating over the parts and streams the content one-by-one to the output stream. If not, and the .build() operation is creating a big fat byte[], then you could take this array, stream it in chunks to the server and tell your user how many percent of the upload is already done.

From a resource perspective I'd prefer not really knowing what happens. But if feedback is that important and if the movies are only a few megabytes in size, you could stream the Multipart-Entity first to a ByteArrayOutputStream and then write little chunks of the created byte-array to the server while notifying your user about progress. The following code is not validated or tested (you can see it as pseudo-code):

ByteArrayOutputStream baos = new ByteArrayOutputStream();
entity.writeTo(baos);
baos.close();
byte[] payload = baos.toByteArray();
baos = null;

OutputStream os = conn.getOutputStream();

int totalSize = payload.length;
int bytesTransferred = 0;
int chunkSize = 2000;

while (bytesTransferred < totalSize) {
    int nextChunkSize = totalSize - bytesTransferred;
    if (nextChunkSize > chunkSize) {
        nextChunkSize = chunkSize;
    }
    os.write(payload, bytesTransferred, nextChunkSize); // TODO check outcome!
    bytesTransferred += nextChunkSize;

    // Here you can call the method which updates progress
    // be sure to wrap it so UI-updates are done on the main thread!
    updateProgressInfo(100 * bytesTransferred / totalSize);
}
os.close();

A more elegant way would be to write an intercepting OutputStream which registers progress and delegates the real write-operations to the underlaying "real" OutputStream.

Edit

@whizzzkey wrote:

I've re-checked it many times - entity.writeTo(os) DOESN'T do a real upload, it does conn.getResponseCode() or conn.getInputStream()

Now it's clear. HttpURLConnection is buffering your upload data, because it doesn't know the content-length. You've set the header 'Content-length', but oviously this is ignored by HUC. You have to call

conn.setFixedLengthStreamingMode(entity.getContentLength());

Then you should better remove the call to conn.setRequestProperty("Content-length", entity.getContentLength() + "");

In this case, HUC can write the headers and entity.writeTo(os) can really stream the data to the server. Otherwise the buffered data is sent when HUC knows how many bytes will be transferred. So in fact, getInputStream() tells HUC that you're finished, but before really reading the response, all the collected data has to be sent to the server.

I wouldn't recommend changing your code, but for those of you who don't know the exact size of the transferred data (in bytes, not characters!!), you can tell HUC that it should transfer the data in chunks without setting the exact content-length:

conn.setChunkedStreamingMode(-1); // use default chunk size

File Upload with Java (with progress bar)

I ended up stumbling across an open source Java uploader applet and found everything I needed to know within its code. Here are links to a blog post describing it as well as the source:

Article

Source Code

Get Progress of File Upload using HttpPost in Android

Since I don't see any solution for this, I suppose the answer is to use a spinning animation without a progress percent. Since nothing can be done until the transmission is complete anyway. Oh well... atleast it solved my problem.

How to get a progress bar for a file upload with Apache HttpClient 4?

Hello guys!

I solved the problem myself and made ​​a simple example to it.

If there are any questions, feel free to ask.

Here we go!

ApplicationView.java

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.io.File;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JProgressBar;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.HttpVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPut;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.HttpParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.http.util.EntityUtils;

public class ApplicationView implements ActionListener
{

  File file = new File("C:/Temp/my-upload.avi");
  JProgressBar progressBar = null;

  public ApplicationView()
  {
    super();
  }

  public void createView()
  {
    JFrame frame = new JFrame("File Upload with progress bar - Example");
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    frame.setBounds(0, 0, 300, 200);
    frame.setVisible(true);

    progressBar = new JProgressBar(0, 100);
    progressBar.setBounds(20, 20, 200, 30);
    progressBar.setStringPainted(true);
    progressBar.setVisible(true);

    JButton button = new JButton("upload");
    button.setBounds(progressBar.getX(),
            progressBar.getY() + progressBar.getHeight() + 20,
            100,
            40);
    button.addActionListener(this);

    JPanel panel = (JPanel) frame.getContentPane();
    panel.setLayout(null);
    panel.add(progressBar);
    panel.add(button);
    panel.setVisible(true);
  }

  public void actionPerformed(ActionEvent e)
  {
    try
    {
      sendFile(this.file, this.progressBar);
    }
    catch (Exception ex)
    {
      System.out.println(ex.getLocalizedMessage());
    }
  }

  private void sendFile(File file, JProgressBar progressBar) throws Exception
  {
    String serverResponse = null;
    HttpParams params = new BasicHttpParams();
    params.setParameter(HttpProtocolParams.USE_EXPECT_CONTINUE, true);
    HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
    HttpClient client = new DefaultHttpClient(params);
    HttpPut put = new HttpPut("http://localhost:8080/" + file.getName());

    ProgressBarListener listener = new ProgressBarListener(progressBar);
    FileEntityWithProgressBar fileEntity = new FileEntityWithProgressBar(file, "binary/octet-stream", listener);
    put.setEntity(fileEntity);

    HttpResponse response = client.execute(put);
    HttpEntity entity = response.getEntity();
    if (entity != null)
    {
      serverResponse = EntityUtils.toString(entity);
      System.out.println(serverResponse);
    }
  }
}

FileEntityWithProgressBar.java

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.http.entity.AbstractHttpEntity;

/**
 * File entity which supports a progress bar.<br/>
 * Based on "org.apache.http.entity.FileEntity".
 * @author Benny Neugebauer (www.bennyn.de)
 */
public class FileEntityWithProgressBar extends AbstractHttpEntity implements Cloneable
{

  protected final File file;
  private final ProgressBarListener listener;
  private long transferredBytes;

  public FileEntityWithProgressBar(final File file, final String contentType, ProgressBarListener listener)
  {
    super();
    if (file == null)
    {
      throw new IllegalArgumentException("File may not be null");
    }
    this.file = file;
    this.listener = listener;
    this.transferredBytes = 0;
    setContentType(contentType);
  }

  public boolean isRepeatable()
  {
    return true;
  }

  public long getContentLength()
  {
    return this.file.length();
  }

  public InputStream getContent() throws IOException
  {
    return new FileInputStream(this.file);
  }

  public void writeTo(final OutputStream outstream) throws IOException
  {
    if (outstream == null)
    {
      throw new IllegalArgumentException("Output stream may not be null");
    }
    InputStream instream = new FileInputStream(this.file);
    try
    {
      byte[] tmp = new byte[4096];
      int l;
      while ((l = instream.read(tmp)) != -1)
      {
        outstream.write(tmp, 0, l);
        this.transferredBytes += l;
        this.listener.updateTransferred(this.transferredBytes);
      }
      outstream.flush();
    }
    finally
    {
      instream.close();
    }
  }

  public boolean isStreaming()
  {
    return false;
  }

  @Override
  public Object clone() throws CloneNotSupportedException
  {
    return super.clone();
  }
}

ProgressBarListener.java

import javax.swing.JProgressBar;

public class ProgressBarListener
{

  private int transferedMegaBytes = 0;
  private JProgressBar progressBar = null;

  public ProgressBarListener()
  {
    super();
  }

  public ProgressBarListener(JProgressBar progressBar)
  {
    this();
    this.progressBar = progressBar;
  }

  public void updateTransferred(long transferedBytes)
  {
    transferedMegaBytes = (int) (transferedBytes / 1048576);
    this.progressBar.setValue(transferedMegaBytes);
    this.progressBar.paint(progressBar.getGraphics());
    System.out.println("Transferred: " + transferedMegaBytes + " Megabytes.");
  }
}

Happy Coding!

---

Related Topics