wish helps you You need to make sure /app/hadoop is created in hdfs, not the local fs. In your check for the directory you use ls -l which checks on the local filesystem which is separate from the hdfs namespace. Try hadoop fs -ls /app/hadoop. If it's not there then create it with hadoop fs -mkdir. – snkherv
hadoop hdfs java - what is the best way to copy a list of files from hdfs to hdfs
By : Reborn_Huan
Date : November 21 2020, 12:01 PM
I hope this helps you . The list is probably 100-200 lines long each file is a few megabytes.
cat pairs-file | while read pair; do hdfs dfs -cp $pair; done
Copy hdfs file to another hdfs location using java
will help you DistCp (distributed copy) is a tool used for copying data between clusters. It uses MapReduce to effect its distribution, error handling and recovery, and reporting. It expands a list of files and directories into input to map tasks, each of which will copy a partition of the files specified in the source list. Usage: $ hadoop distcp