问题描述
大家好，我的一个List<String>中有10W条记录，要把这个List的的内容拼接起来。我启动5个线程同时处理，每个线程处理2w条记录。然后把5个线程处理的内容拼接起来。 请大家给一个简单的demo。谢谢
解决方案
不用这么复杂,这样就好了..import java.util.ArrayList;import java.util.List;import java.util.concurrent.Callable;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.Future;public class Test {public static void main(String[] args) {try {List<String> list = new ArrayList<>();for (int i = 0; i < 100; i++) {list.add(i + ",");}System.out.println(new Test().list2Str(list, 5));} catch (Exception e) {e.printStackTrace();}}public String list2Str(List<String> list, final int nThreads) throws Exception {if (list == null || list.isEmpty()) {return null;}StringBuffer ret = new StringBuffer();int size = list.size();ExecutorService executorService = Executors.newFixedThreadPool(nThreads);List<Future<String>> futures = new ArrayList<Future<String>>(nThreads);for (int i = 0; i < nThreads; i++) {final List<String> subList = list.subList(size / nThreads * i, size / nThreads * (i + 1));Callable<String> task = new Callable<String>() {@Overridepublic String call() throws Exception {StringBuffer sb = new StringBuffer();for (String str : subList) {sb.append(str);}return sb.toString();}};futures.add(executorService.submit(task));}for (Future<String> future : futures) {ret.append(future.get());}executorService.shutdown();return ret.toString();}}
解决方案二：
有个问题，如果5个线程有一个失败，其他线程就会阻塞哦
解决方案三：
解决方案四：
针对 @blackstreet 提出的内存溢出的问题,代码做出如下修改.1,多加一次list的遍历来预设StringBuffer的长度,而多的这次循环本身几乎没有性能开销(13000000次循环仅70ms左右,当然环境不同实际时间会有偏差)2,对参数list不进行subList操作,因为我们可以发现list在后续操作中是只读,不涉及并发带来的线程安全问题,所以循环时直接通过index来做取值区间判断.(奇怪的是,去除subList的操作似乎并没有对性能产生什么影响)3,关于StringBuilder和StringBuffer,13000000数据测试了几次,居然也没有什么差别.不解,可能是jvm默认调了优,也可能我测试数据不够..总之测试比较少,没有权威性,一切以实际为准.各位就姑且一看吧.public String list2Str(final List<String> list, final int nThreads) throws Exception {if (list == null || list.isEmpty()) {return null;}int len = 0;for (String str : list) {len += str.length();}StringBuffer ret = new StringBuffer(len);final int size = list.size();ExecutorService executorService = Executors.newFixedThreadPool(nThreads);List<Future<String>> futures = new ArrayList<Future<String>>(nThreads);try {for (int i = 0; i < nThreads; i++) {final int j = i;Callable<String> task = new Callable<String>() {@Overridepublic String call() throws Exception {int len = 0;for (int n = size / nThreads * j; n < size / nThreads * (j + 1); n++) {len += list.get(n).length();}StringBuffer sb = new StringBuffer(len);for (int n = size / nThreads * j; n < size / nThreads * (j + 1); n++) {sb.append(list.get(n));}return sb.toString();}};futures.add(executorService.submit(task));}for (Future<String> future : futures) {ret.append(future.get());}} finally {executorService.shutdown();}return ret.toString();}
解决方案五：
刚试了一下list 集合在13000000个for (int i = 0; i < 13000000; i++) {list.add("a" + i);｝ 直接用StringBuilder foreach append：每次稳定6.1秒多线程在11.7秒时内存jvm outof环境是在：win7 8个cpu核上运行意外地的我在运行单线程StringBuilder foreach append这种时，cpu的每个核都是在工作达到100%.谁解释下
解决方案六：
fork/join框架解决
解决方案七：
import java.util.ArrayList;import java.util.List;public class Test7 {public static void main(String[] args){List<String> list = new ArrayList<String>();for(int i=0;i<10;i++){list.add(i+"");}List<StringBuffer> sbs = new ArrayList<StringBuffer>();for(int i=0;i<5;i++){sbs.add(new StringBuffer());}Thread sumThread = new Thread(new Sum(sbs));for(int i=0;i<5;i++){ Thread thread = new Thread(new MyHander(list,i*2,i*2+2,sbs.get(i))); thread.start();}sumThread.start();}} class Sum implements Runnable{ private List<StringBuffer> sbs; public Sum(List<StringBuffer> sbs){ this.sbs =sbs; }public void run() {synchronized (this) { while(true){ if(MyHander.getCounter()<5){try {wait(100);} catch (InterruptedException e) {// TODO Auto-generated catch blocke.printStackTrace();} }else{ break; } } StringBuffer value = new StringBuffer();for(int i=0;i<5;i++){value.append(sbs.get(i).toString());}System.out.println(value.toString());}} } class MyHander implements Runnable { private List<String> list; private int start; private int end; private StringBuffer sb; private static Integer counter=0; public MyHander(List<String> list,int start,int end,StringBuffer sb){ this.list =list; this.start = start; this.end = end; this.sb = sb; } public static Integer getCounter(){ return counter; } @Overridepublic void run() {for(int i=start; i<end;i++){ sb.append(list.get(i));}synchronized(counter){counter++; }} }
解决方案八：
你用这个类啊ArrayDeque，然后启动5个线程 每个线程内部循环2万次 去调用ArrayDeque.poll()方法就行了