Upload Large File in Android Without Outofmemory Error

Upload large file in Android without outofmemory error

Did you try using

con.setChunkedStreamingMode(1024);

This will help you to chunk your data into specific size, so that you need not keep your entire file in the memory.

UPDATE:

Try using the below method. I use this method to upload a 80 mb file without Exception.

public String sendFileToServer(String filename, String targetUrl) {
    String response = "error";
    Log.e("Image filename", filename);
    Log.e("url", targetUrl);
    HttpURLConnection connection = null;
    DataOutputStream outputStream = null;
    // DataInputStream inputStream = null;

    String pathToOurFile = filename;
    String urlServer = targetUrl;
    String lineEnd = "\r\n";
    String twoHyphens = "--";
    String boundary = "*****";
    DateFormat df = new SimpleDateFormat("yyyy_MM_dd_HH:mm:ss");

    int bytesRead, bytesAvailable, bufferSize;
    byte[] buffer;
    int maxBufferSize = 1 * 1024;
    try {
        FileInputStream fileInputStream = new FileInputStream(new File(
                pathToOurFile));

        URL url = new URL(urlServer);
        connection = (HttpURLConnection) url.openConnection();

        // Allow Inputs & Outputs
        connection.setDoInput(true);
        connection.setDoOutput(true);
        connection.setUseCaches(false);
        connection.setChunkedStreamingMode(1024);
        // Enable POST method
        connection.setRequestMethod("POST");

        connection.setRequestProperty("Connection", "Keep-Alive");
        connection.setRequestProperty("Content-Type",
                "multipart/form-data;boundary=" + boundary);

        outputStream = new DataOutputStream(connection.getOutputStream());
        outputStream.writeBytes(twoHyphens + boundary + lineEnd);

        String connstr = null;
        connstr = "Content-Disposition: form-data; name=\"uploadedfile\";filename=\""
                + pathToOurFile + "\"" + lineEnd;
        Log.i("Connstr", connstr);

        outputStream.writeBytes(connstr);
        outputStream.writeBytes(lineEnd);

        bytesAvailable = fileInputStream.available();
        bufferSize = Math.min(bytesAvailable, maxBufferSize);
        buffer = new byte[bufferSize];

        // Read file
        bytesRead = fileInputStream.read(buffer, 0, bufferSize);
        Log.e("Image length", bytesAvailable + "");
        try {
            while (bytesRead > 0) {
                try {
                    outputStream.write(buffer, 0, bufferSize);
                } catch (OutOfMemoryError e) {
                    e.printStackTrace();
                    response = "outofmemoryerror";
                    return response;
                }
                bytesAvailable = fileInputStream.available();
                bufferSize = Math.min(bytesAvailable, maxBufferSize);
                bytesRead = fileInputStream.read(buffer, 0, bufferSize);
            }
        } catch (Exception e) {
            e.printStackTrace();
            response = "error";
            return response;
        }
        outputStream.writeBytes(lineEnd);
        outputStream.writeBytes(twoHyphens + boundary + twoHyphens
                + lineEnd);

        // Responses from the server (code and message)
        int serverResponseCode = connection.getResponseCode();
        String serverResponseMessage = connection.getResponseMessage();
        Log.i("Server Response Code ", "" + serverResponseCode);
        Log.i("Server Response Message", serverResponseMessage);

        if (serverResponseCode == 200) {
            response = "true";
        }

        String CDate = null;
        Date serverTime = new Date(connection.getDate());
        try {
            CDate = df.format(serverTime);
        } catch (Exception e) {
            e.printStackTrace();
            Log.e("Date Exception", e.getMessage() + " Parse Exception");
        }
        Log.i("Server Response Time", CDate + "");

        filename = CDate
                + filename.substring(filename.lastIndexOf("."),
                        filename.length());
        Log.i("File Name in Server : ", filename);

        fileInputStream.close();
        outputStream.flush();
        outputStream.close();
        outputStream = null;
    } catch (Exception ex) {
        // Exception handling
        response = "error";
        Log.e("Send file Exception", ex.getMessage() + "");
        ex.printStackTrace();
    }
    return response;
}

Upload large file with progress bar and without OutOfMemory Error in Android

Finally I got solution of my question I want to share it ...

1.First solution Link1 This solution is ok with small files like image upload upto 15MB .But I could not get rid from OutOfMemory error if file is very large.

2.Second solution(Link2) is really a good solution and for showing progress bar I used a custom MultipartEntity class. Code is here:

    import java.io.FilterOutputStream;
    import java.io.IOException;
    import java.io.OutputStream;
    import java.nio.charset.Charset;

    import org.apache.http.entity.mime.HttpMultipartMode;
    import org.apache.http.entity.mime.MultipartEntity;

    public class CustomMultiPartEntity extends MultipartEntity

{

    private final ProgressListener listener;

    public CustomMultiPartEntity(final ProgressListener listener)
    {
        super();
        this.listener = listener;
    }

    public CustomMultiPartEntity(final HttpMultipartMode mode, final ProgressListener listener)
    {
        super(mode);
        this.listener = listener;
    }

    public CustomMultiPartEntity(HttpMultipartMode mode, final String boundary, final Charset charset, final ProgressListener listener)
    {
        super(mode, boundary, charset);
        this.listener = listener;
    }

    @Override
    public void writeTo(final OutputStream outstream) throws IOException
    {
        super.writeTo(new CountingOutputStream(outstream, this.listener));
    }

    public static interface ProgressListener
    {
        void transferred(long num);
    }

    public static class CountingOutputStream extends FilterOutputStream
    {

        private final ProgressListener listener;
        private long transferred;

        public CountingOutputStream(final OutputStream out, final ProgressListener listener)
        {
            super(out);
            this.listener = listener;
            this.transferred = 0;
        }

        public void write(byte[] b, int off, int len) throws IOException
        {
            out.write(b, off, len);
            this.transferred += len;
            this.listener.transferred(this.transferred);
        }

        public void write(int b) throws IOException
        {
            out.write(b);
            this.transferred++;
            this.listener.transferred(this.transferred);
        }
    }
}

And my activity code is

import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntity;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.entity.mime.content.StringBody;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;

import com.example.fileupload.CustomMultiPartEntity.ProgressListener;

import android.app.Activity;
import android.os.AsyncTask;
import android.os.Bundle;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.ProgressBar;
import android.widget.TextView;

public class MainActivity extends Activity implements OnClickListener {
    private ProgressBar pb;
    private final String filename = "/mnt/sdcard/vid.mp4";
    // private final String filename = "/mnt/sdcard/a.3gp";
    private String urlString = "http://10.0.2.2:8080/FileUploadServlet1/UploadServlet";
    private TextView tv;
    long totalSize = 0;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        tv = (TextView) findViewById(R.id.textView1);
        findViewById(R.id.button1).setOnClickListener(this);
        tv.setText("init");
        pb = (ProgressBar) findViewById(R.id.progressBar1);

    }

    private class Uploadtask extends AsyncTask<Void, Integer, String> {
        @Override
        protected void onPreExecute() {
            pb.setProgress(0);
            tv.setText("shuru");
            super.onPreExecute();
        }

        @Override
        protected void onProgressUpdate(Integer... progress) {
            pb.setProgress(progress[0]);
        }

        @Override
        protected String doInBackground(Void... params) {
            return upload();
        }

        private String upload() {
            String responseString = "no";

            File sourceFile = new File(filename);
            if (!sourceFile.isFile()) {
                return "not a file";
            }
            HttpClient httpclient = new DefaultHttpClient();
            HttpPost httppost = new HttpPost(urlString);


            try {
                CustomMultiPartEntity entity=new CustomMultiPartEntity(new ProgressListener() {

                    @Override
                    public void transferred(long num) {
                        publishProgress((int) ((num / (float) totalSize) * 100));                       
                    }
                });

                entity.addPart("type", new StringBody("video"));
                entity.addPart("uploadedfile", new FileBody(sourceFile));
                totalSize = entity.getContentLength();
                httppost.setEntity(entity);
                HttpResponse response = httpclient.execute(httppost);
                HttpEntity r_entity = response.getEntity();
                responseString = EntityUtils.toString(r_entity);

            } catch (ClientProtocolException e) {
                responseString = e.toString();
            } catch (IOException e) {
                responseString = e.toString();
            }

            return responseString;

        }


        @Override
        protected void onPostExecute(String result) {
            tv.setText(result);
            super.onPostExecute(result);
        }

    }

    @Override
    public void onClick(View v) {
        // TODO Auto-generated method stub
        new Uploadtask().execute();
    }

}

Uploading large file from Android app gives out of memory error despite buffering

The problem is most probably caused because the system is buffering the request before sending it to the server. To prevent that (the request form being buffered), you can use:

conn.setChunkedStreamingMode(....);

Where conn is an instance of HttpURLConnection.

More details

Upload large video file use Retrofit(Android) have OutOfMemoryError

I have faced same error while uploading large video using retrofit,but
Removing

loggingInterceptor.setLevel(HttpLoggingInterceptor.Level.BODY)

or set is as NONE
worked for me.

Getting Out Of Memory Error when saving big files

Not tested but you probably want a append the bytes to the FileOutputStream and write the buffer in the loop :

public static File saveUri(Uri uri, File file, ContentResolver resolver) {
    try (final OutputStream os = new FileOutputStream(file, true)) {
        final InputStream inputStream = resolver.openInputStream(uri);
        if (inputStream != null) {
            byte[] buffer = new byte[1024];
            int bytesRead;
            while ((bytesRead = inputStream.read(buffer)) != -1) {
                os.write(buffer, 0, bytesRead);
            }
            inputStream.close();
        }
    } catch (IOException e) {
        Log.e(TAG, "Error writing file");
    }
    return file;
}

There are many ways to do this...

Upload file in Android with outofmemory error

You should confirm at what point your error occurs. I suspect that it's during reading the response. In this case, it seems that server may be responding with a lot of data that you place in the StringBuffer. Do you actually need to consume the entire response and keep it in memory? If it's a file, save it rather than keeping in memory.

---

I did some more research and here is one other possibility. Android JVM by default has 16mb max heap. See this for some details.

On the other hand, if your server does not actually consume the data, most of it will reside on the client. So if you have more than max heap of data the client will fail.

So I suspect that your server just does not read the data from the stream.

The following class (which is a snippet of relevant parts of your code) illustrates the problem. Run it on any JVM like following:

java -Xmx16m -cp . Test

and it will produce OOM very quickly. In fact, much earlier than expected.

import java.io.*;
import java.net.*;

public class Test {
  public static void main(String argv[]) throws Exception {
    new Thread() {
      public void run() {
        try {
           new ServerSocket(12000).accept();
        } catch (Throwable t) {
          t.printStackTrace();
        } 
      }
    }.start();

    URL url = new URL("http://localhost:12000/");
    HttpURLConnection con = (HttpURLConnection) url.openConnection();
    con.setDoInput(true);
    con.setDoOutput(true);
    con.setUseCaches(false);
    DataOutputStream ds = new DataOutputStream(con.getOutputStream());
    int bufferSize = 1024;
    byte[] buffer = new byte[bufferSize];
    for (int i=0;i<100000;i++) {
      ds.write(buffer, 0, 1024);
      System.out.println("Written chunk " + i);
    }       
    ds.flush();
  }
}

---

Related Topics