J.C.
BOSE UNIVERSITY OF SCIENCE & TECHNOLOGY,
YMCA, FARIDABAD
Big Data Lab
Degree of Bachelor of Technology in CE,
Specialization in Data Science
Submitted To
Dr. Dimple Chahal
Submitted By
Aditya Pandey
22001016005
INDEX
S.No. Title Date Signature
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
Program 01: Write a program in Java for various arithmetic operations.
Solution 01:
public class ArithmeticOperations {
public static void main(String[] args) {
int num1 = 20;
int num2 = 10;
int sum = num1 + num2;
int difference = num1 - num2;
int product = num1 * num2;
int quotient = num1 / num2;
int remainder = num1 % num2;
System.out.println("Arithmetic Operations:");
System.out.println("Numbers: " + num1 + " and " + num2);
System.out.println("Addition: " + sum);
System.out.println("Subtraction: " + difference);
System.out.println("Multiplication: " + product);
System.out.println("Division: " + quotient);
System.out.println("Modulus: " + remainder);
}
}
OUTPUT:
Arithmetic Operations:
Numbers: 20 and 10
Addition: 30
Subtraction: 10
Multiplication: 200
Division: 2
Modulus: 0
Program 02: Write a program in Java for factorial of a number
Solution 02:
import java.util.Scanner;
public class Factorial {
public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
System.out.print("Enter a number: ");
int num = sc.nextInt();
long factorial = 1;
for(int i = 1; i <= num; i++) {
factorial *= i;
}
System.out.println("Factorial of " + num + " is: " + factorial);
sc.close();
}
}
OUTPUT:
Enter a number: 5
Factorial of 5 is: 120
Program 03: Write a program in Java for fibonacci series.
Solution 03:
import java.util.Scanner;
public class FibonacciSeries {
public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
System.out.print("Enter the number of terms for Fibonacci series: ");
int n = sc.nextInt();
int first = 0, second = 1;
System.out.println("Fibonacci Series up to " + n + " terms:");
for (int i = 1; i <= n; i++) {
System.out.print(first + " ");
int next = first + second;
first = second;
second = next;
}
sc.close();
}
}
OUTPUT:
Enter the number of terms for Fibonacci series: 10
Fibonacci Series up to 10 terms:
0 1 1 2 3 5 8 13 21 34
Program 04: Write a program in Java for linear search.
Solution 04:
import java.util.Scanner;
public class LinearSearch {
public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
System.out.print("Enter the size of the array: ");
int size = sc.nextInt();
int[] arr = new int[size];
System.out.println("Enter " + size + " elements:");
for (int i = 0; i < size; i++) {
arr[i] = sc.nextInt();
}
System.out.print("Enter the element to search: ");
int key = sc.nextInt();
int index = -1;
for (int i = 0; i < size; i++) {
if (arr[i] == key) {
index = i;
break;
}
}
if (index == -1) {
System.out.println("Element not found.");
} else {
System.out.println("Element found at index: " + index);
}
sc.close();
}
}
OUTPUT:
Enter the size of the array: 4
Enter 4 elements:
10 20 30 40
Enter the element to search: 25
Element not found.
Program 05: Write a program in Java for Bubble sort.
Solution 05:
public class BubbleSort {
public static void bubbleSort(int[] arr) {
int n = arr.length;
boolean swapped;
for (int i = 0; i < n - 1; i++) {
swapped = false;
for (int j = 0; j < n - i - 1; j++) {
if (arr[j] > arr[j + 1]) {
int temp = arr[j];
arr[j] = arr[j + 1];
arr[j + 1] = temp;
swapped = true;
}
}
if (!swapped) break;
}
}
public static void printArray(int[] arr) {
for (int num : arr) {
System.out.print(num + " ");
}
System.out.println();
}
public static void main(String[] args) {
int[] arr = {64, 34, 25, 12, 22, 11, 90};
System.out.println("Original array:");
printArray(arr);
bubbleSort(arr);
System.out.println("Sorted array:");
printArray(arr);
}
}
OUTPUT:
Original array:
64 34 25 12 22 11 90
Sorted array:
11 12 22 25 34 64 90
Program 06: Write a program to implement word count in java (Eclipse IDE)
Solution 06:
import java.util.Scanner;
public class WordCount {
public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
System.out.println("Enter a sentence:");
String input = sc.nextLine();
String[] words = input.toLowerCase().split("\\s+");
String[] uniqueWords = new String[words.length];
int[] counts = new int[words.length];
int uniqueCount = 0;
for (String word : words) {
int index = findIndex(uniqueWords, word, uniqueCount);
if (index == -1) {
uniqueWords[uniqueCount] = word;
counts[uniqueCount] = 1;
uniqueCount++;
} else {
counts[index]++;
}
}
System.out.println("\nWord frequencies:");
for (int i = 0; i < uniqueCount; i++) {
System.out.println(uniqueWords[i] + ": " + counts[i]);
}
sc.close();
}
public static int findIndex(String[] words, String word, int size) {
for (int i = 0; i < size; i++) {
if (words[i].equals(word)) {
return i;
}
}
return -1;
}
}
OUTPUT:
Enter a sentence:
Java is fun and Java is powerful
Word frequencies:
java: 2
is: 2
fun: 1
and: 1
powerful: 1
Program 07: Interact with HDFS and learn various commands
Solution 07:
(i) Open the browser.
(ii) Download a document Shakespeare.txt (link to the file:
http://ocw.mit.edu/ans7870/6/6.006/s08/lecturenotes/files/t8.shakespeare.txt
)
(iii) Open terminal shell and run
cd Downloads : change to downloads directory.
ls : to see that the text file was saved.
(iv) Copy file to HDFS
Hadoop fs -copyFromLocal AdityaPandey22001016005.txt
(v) Verify the file was copied to HDFS using hdfs fs -ls
(vi) copy the file within HDFS itself
hadoop fs -cp AdityaPandey22001016005.txt words2.txt
then run Hadoop fs -ls to see the file is copied or not.
(vii) Copy the file from HDFS.
hadoop fs -copyToLocal words2.txt : copies a file from HDFS to local system.
Then run ls to check if the file is copied from HDFS.
(viii) Deleting the file in HDFS.
hadoop fs -rm words2.txt
Program 08: Run a Word Count program in Eclipse in HDFS.
Solution 08:
(i) Open eclipse present on cloudera desktop.
(ii) Create a new project. File > New > Project > Java Project > Next.
Use “WordCount” as project name and click “Finish.”
(iii) Add the Hadoop libraries to the project. Right click on WordCount and
select “Properties”. Click on “Java Build Path”.
Click on Add External Jars,
then File System > usr > lib > Hadoop
Select all jars, and click OK.
We need more external libs. Click on “Add External JARs…” again, then select all
jar files in “client” folder and click “OK” button.
(iv) Create a Java Mapper, reducer program. Right click on “src” folder of the
project wordcount.
New>> Class >> In Name textbox give it as “WordCount” and click Finish.
(v) Write down the code in WordCount.java file.
import java.io.IOException;
import java.util.StringTokenizer;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
public class WordCount {
public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()) {
word.set(tokenizer.nextToken());
context.write(word, one); // Emit <word, 1>
public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> {
public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException,
InterruptedException {
int sum = 0;
for (IntWritable val : values) {
sum += val.get(); // Sum up occurrences of each word
context.write(key, new IntWritable(sum)); // Emit <word, total_count>
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "wordcount");
job.setJarByClass(WordCount.class);
job.setMapperClass(Map.class);
job.setReducerClass(Reduce.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
FileInputFormat.addInputPath(job, new Path(args[0])); // Input path from HDFS
FileOutputFormat.setOutputPath(job, new Path(args[1])); // Output path in HDFS
System.exit(job.waitForCompletion(true) ? 0 : 1);
(vi) Export the project as JAR. (Right click on WordCount and select
“Export”>Java>Jar File>Next
In the JAR file textbox given as /home/cloudera/WordCount.jar Click Finish> Ok
(vii) Open terminal and type
cd /home/cloudera
ls
(viii) Create the input file for MapReduce Program.
vi /home/cloudera/inputfile.txt
This will open inputfile.txt in an editor. Now press ‘i’ on keyboard to write on
file.
To save it press “esc+:” then type wq.
(ix) Move the file to Hadoop file system for processing the program.
To look at hadoop file system type on the terminal hdfs dfs -ls / Here / indicates
the root directory of the hadoop files system(hdfs).
(x) Create a input directory in hdfs using : hdfs dfs -mkdir /input
(xi) Move the inputfile.txt to Hadoop file system using :
hdfs dfs -put /home/cloudera/inputfile.txt /input/
(xii) Use ‘hdfs dfs -cat /input/inputfile.txt’ to view the file.
(xiii) Run the MapReduce program using :
hadoop jar /home/cloudera/WordCount.jar WordCount /input/inputfile.txt
/output_1
(xiv) Use ‘hdfs dfs -ls /output_1’ to view the output directory.
(xv) View the output file using : hdfs dfs -cat /output_1/part-r-00000
Program 09: Run a max temperature for each city.
Solution 09:
(i) Make a new Java Project by the name of MaxTempMapReduce in eclipse.
(ii) Import the external jars for the project.
(iii) Create a new Class by the same name.
(iv) Write the following code.
(v)Click on Files > Export > Java >Jar File.
(vi) Open terminal.
(vii) run the command ‘vi /home/cloudera/input_final.txt’ and write the
following as input for the input file.
(viii) Create an input directory using
‘hdfs dfs -mkdir /input_final
(viii) Move the input file into input directory using ‘hdfs dfs -put
/home/cloudera/inputfile_final.txt /input_final/’
(ix) Run the command
‘hadoop jar /home/cloudera/MaxTempMapreduce.jar MaxTempMapReduce
/input_final/input_final.txt /output_final’.
(x) Run hdfs dfs -cat /output_final/part-r-00000, to view the result.
Program 10: Run a Word Count program in HDFS terminal.
Solution 10:
(i) Open the browser and search for ‘hadoop examples.jar’, and download it.
(ii) Cut and paste the file into ‘/home/cloudera’.
(iii) Run ‘hadoop jar /home/cloudera/hadoop-mapreduce-examples-3.3.6.jar’.
This way our Jar file gets successfully uploaded and excuted, but it requires a
specific examole program as an argument.
(iv) To run WordCount using a text file run the following command
‘hadoop jar /jar/cloudera/hadoop-mapreduce-examples-3.3.6.jar wordcount
AdityaPandey22001016005.txt’
The status of the file is being displayed.
(v)Check for WordCount output directory using ‘hadoop fs -ls’.
(vi) Run ‘hadoop fs -copyToLocal out/part-r-00000 locat.txt’ to copy part-r-
00000 to local file.
(vii) Run ‘more locat.txt’ to view the results.
