Click here to Skip to main content
15,888,461 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:

i try to execute and compile this code java mapreduce on my eclipse in local, but this probleme is showed up please help where is the issue?


and this is the error showed up:
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
    at LogFile.TraitServeur.main(TraitServeur.java:63)"



the line error 63 is about the output format:
FileOutputFormat.setOutputPath(conf, new Path(args[1]))


What I have tried:

this is my code source

Java
import java.io.IOException ; 
import java.util.* ;
import org.apache.hadoop.fs.Path ;
//import org.apache.hadoop.conf.* ;/*Package de apache hadoop utilisé dans le développement*/
import org.apache.hadoop.io.* ;
import org.apache.hadoop.mapred.* ;
//import org.apache.hadoop.util.* ;
public class TraitServeur {
    //phase Map
public static class TokenizerMapper extends MapReduceBase implements Mapper<LongWritable,
Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1) ;
private Text map = new Text();
public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { 
String line = value.toString() ;
String[] rows = line.split("\\s+") ; 
StringTokenizer tokenizer = new StringTokenizer(rows[3]);
tokenizer = new StringTokenizer(rows[3]) ;
int count = 0 ;
String date = rows[0] ;
String day = rows[1] ; 
String gravite = rows[3] ;
while (tokenizer.hasMoreTokens()) {
map.set(tokenizer.nextToken()) ;
map.set(date + day + "\t" + gravite) ;
output.collect(map, one) ;
count= count+1;
}
}
}
//phase reduce
public static class Reduce extends MapReduceBase implements Reducer<Text,
IntWritable, Text, IntWritable> { 
public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text,
IntWritable> output, Reporter reporter) throws IOException {
int sum = 0 ;
while (values.hasNext()) {
sum += values.next().get() ;  }
output.collect(key, new IntWritable(sum)) ;
}
}
}
public static void main(String[] args) throws Exception {
JobConf conf = new JobConf(TraitServeur.class) ; 
conf.setJobName("dpgs") ; 
conf.setOutputKeyClass(Text.class) ; 
conf.setOutputValueClass(IntWritable.class) ;
conf.setMapperClass(TokenizerMapper.class) ; 
conf.setCombinerClass(Reduce.class) ;
conf.setReducerClass(Reduce.class) ;
conf.setInputFormat(TextInputFormat.class) ; 
conf.setOutputFormat(TextOutputFormat.class) ;
FileInputFormat.setInputPaths(conf, new Path(args[0])) ; 
FileOutputFormat.setOutputPath(conf, new Path(args[1])) ;
JobClient.runJob(conf) ;
}
Posted
Updated 28-May-18 5:30am
v2
Comments
Jochen Arndt 28-May-18 7:17am    
Read the error message. It contains a file name and a line number (63). Locate that line in your code. There is an index used in that line which is outside the allowed range.

Quote:
the line error 63 is about the output format:
Java
FileOutputFormat.setOutputPath(conf, new Path(args[1]));
and the error message is
java.lang.ArrayIndexOutOfBoundsException
So there is no second command line argument present when executing the application.


You have to execute the application like
NameOfApp InputPath OutputPath
or better add code to check if all required command line parameters are present:
Java
public static void main(String[] args) throws Exception {
    if (args.length < 2) {
        System.err.println("Must pass InputPath and OutputPath.");
        System.exit(1);
    }
    // ...
}
 
Share this answer
 
yeah thank you realy it was the probleme that i didn't made a outputPath but it showed me another error alawys about native librery haddoop and another one:

<pre>2018-05-28 16:27:24,687 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-05-28 16:27:29,189 INFO  [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1274)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2018-05-28 16:27:29,193 INFO  [main] jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2018-05-28 16:27:29,251 INFO  [main] jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/jackson/map/JsonMappingException
	at org.apache.hadoop.mapreduce.Job.getJobSubmitter(Job.java:1291)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1302)
	at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:578)
	at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:573)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:573)
	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:564)
	at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:873)
	at LogFile.TraitServeur.main(TraitServeur.java:70)
Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.map.JsonMappingException
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 11 more




and the line 70 of my code is :

Java
JobClient.runJob(conf) ;


where is the issue now
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900