logo
down
shadow

Bash script to split file into two files while changing output file during processing


Bash script to split file into two files while changing output file during processing

By : user2953926
Date : November 21 2020, 07:31 AM
seems to work fine I am trying to split a text file into two different files, file looks something like this: , This can be done with awk:
code :
awk '/burrito/ {f="burrito"} /taco/ {f="taco"} {print > f}' file
awk 'BEGIN {f="another_file"} /burrito/ {f="burrito"} /taco/ {f="taco"} {print > f}' file


Share : facebook icon twitter icon
Processing output of python file called inside a bash script

Processing output of python file called inside a bash script


By : jianglei
Date : March 29 2020, 07:55 AM
I wish this helpful for you this is just the beginning of the fun that I'm going to have with bash scripting , If you mean the output of python, you should test it with $() instead
code :
#!/bin/bash

if test "$(python /var/lib/scripts/Hudson.py result)" = "Success"
then
     Run next command
else
     Exit the script
fi  
#!/bin/bash

if [[ "$(python /var/lib/scripts/Hudson.py result)" == "Success" ]]
then
     Run next command
else
     Exit the script
fi  
#!/bin/bash

if python /var/lib/scripts/Hudson.py result
then
     Run next command
else
     Exit the script
fi  
Issue with using While loop in bash script (to split a file into multiple files)

Issue with using While loop in bash script (to split a file into multiple files)


By : Viet Hoang Duc
Date : March 29 2020, 07:55 AM
fixed the issue. Will look into that further Here's a rough attempt -- it's not as fast as a pure-awk solution would be, but it's much, much faster than what you already had:
code :
#!/bin/bash

# two external parameters: input file name, and max size in bytes (default to 3MB)
InputFile=$1
MaxFileSize=${2:-$(( 3 * 1024 * 1024 ))}

BaseName=${InputFile%.*} # strip extension
Ext=${InputFile##*.}     # store extension
FileSeq=0                # start output file at sequence 0

# redirect stdin from the input file, stdout to the first output file
exec <"$InputFile" || exit
exec >"${BaseName}.${FileSeq}.${Ext}" || exit

# read the header; copy it to the first output file, and initialize CurFileSize
IFS= read -r Header || exit
printf '%s\n' "$Header" || exit
CurFileSize=$(( ${#Header} + 1 ))

# ...then loop over our inputs, and copy appropriately
while IFS= read -r line; do
  if [[ $line = DWH,* ]] && (( CurFileSize > MaxFileSize )); then
    (( FileSeq++ ))
    exec >"${BaseName}.${FileSeq}.${Ext}" || exit
    printf '%s\n' "$Header" || exit
    CurFileSize=$(( ${#Header} + 1 ))
  fi
  printf '%s\n' "$line" || exit
  (( CurFileSize += ${#line} + 1 ))
done
$ cat >input.csv <<'EOF'
Src,AgentNum,PhoneNum
DWH,Agent_1234,phone1
NULL,NULL,phone2
NULL,NULL,phone3
DWH,Agent_5678,phone1
NULL,NULL,phone2
NULL,NULL,phone3
DWH,Agent_9999,phone1
NULL,NULL,phone2
NULL,NULL,phone3
EOF

$ ./splitCSV input.csv 100  ## split at first boundary after 100 bytes

$ cat input.0.csv
Src,AgentNum,PhoneNum
DWH,Agent_1234,phone1
NULL,NULL,phone2
NULL,NULL,phone3
DWH,Agent_5678,phone1
NULL,NULL,phone2
NULL,NULL,phone3

$ cat input.1.csv
Src,AgentNum,PhoneNum
DWH,Agent_9999,phone1
NULL,NULL,phone2
NULL,NULL,phone3
BASH SHELL SCRIPT to split a big xml file into multiple small files

BASH SHELL SCRIPT to split a big xml file into multiple small files


By : fuzza
Date : March 29 2020, 07:55 AM
help you fix your problem I have a an XML file in below format , Not pure answer but you can tune this yourself:
Bash Script changing the output file for every file

Bash Script changing the output file for every file


By : Henric Galaxhjälten
Date : March 29 2020, 07:55 AM
it fixes the issue It's bad practice to write shell loops just to process text (see https://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice). This is all you need:
Bash script to Split a file into n files with each file containing x number of records

Bash script to Split a file into n files with each file containing x number of records


By : LarsErikO
Date : March 29 2020, 07:55 AM
I think the issue was by ths following , you either need to give -n parameter or -l parameter. not both of them together.
code :
split -l1000 --numeric-suffixes yourFile.txt
Related Posts Related Posts :
  • Security of su root in bash script
  • how to replace a string at a specific position in a csv file using bash
  • how to delete duplicate lines in a text file in unix bash?
  • Compare a variables in bash
  • How do I use sed on a string?
  • Using bash variables for range in sed
  • convert a DURATION HH:MM:SS to seconds in bash
  • Shell make system paths click-able when printed out by my shell script
  • I want to use parallel-ssh to run a bash script on multiple servers, but it simple prints the echo statements
  • AWK between 2 patterns - first occurence
  • Nginx on docker - daemonize and attach to bash
  • Delete or Replace nth line in a file after a pattern match using shell scripts
  • Actual return code for SCP
  • How to exit a bash function that expects a return
  • Pass command line arguments via sbatch
  • variable substitution (what do you call/name this method)?
  • Call function from another shell script
  • How to Grep and Replace With Exclusions
  • shell command to remove characters after a special character in bash/shell
  • How do I run a script for 1st working day of every month in cron?
  • Bash Unexpected End of File Error After Installing RVM
  • Nested Loop with increment on inner loop?
  • video orientation detection in bash
  • Interrupt sleep in bash with a signal trap
  • Want file rename script to add number at end with fixed length
  • What's the DRYest way to replace a file with a symlink in bash
  • How to run Bitcoin's autogen.sh file?
  • Renaming files in folder and subfolders using a list- Bash Script/Unix
  • Bash command to compare more than two files
  • If It is a directory append text
  • What's wrong with my WHILE-DO loop, or why my script can't find the file I'm giving it on a plate?
  • shadow
    Privacy Policy - Terms - Contact Us © ourworld-yourmove.org