logo
down
shadow

using wget to mirror a website with path and subfolder that have the same name


using wget to mirror a website with path and subfolder that have the same name

By : user2955058
Date : November 22 2020, 10:33 AM
around this issue I figured it out!
The problem was my assumption that /news/index.html was the URL that I needed. After closely reading the man page, I found that -E (--adjust-extension) solved my problem. This flag forces wget to apply the .html extension onto all of the HTML files that it downloads.
code :
http://example.com/news           -->  /news.html
http://example.com/news/article1  -->  /news/article1.html
wget -p -r -l4 -E -k -nH http://example.com


Share : facebook icon twitter icon
Wget Mirror HTML only

Wget Mirror HTML only


By : Yash J. Doshi
Date : March 29 2020, 07:55 AM
I wish did fix the issue. The -R and -A options are used to reject or accept specific file types.
Also consider the bandwidth used to download a whole website. You may want to add the --random-wait option as well.
code :
wget --mirror --random-wait -R gif,jpg,pdf <url>
Https website physical path points to subfolder

Https website physical path points to subfolder


By : Rondel Temple
Date : March 29 2020, 07:55 AM
With these it helps If your 1st site in IIS has a path set to "c:\web\site1" and is bound to port 80, then...
The url http://mydomain will map to "c:\web\site1" http://mydomain/site1 will map to "c:\web\site1\site1"
wget --no-check-certificate mirror

wget --no-check-certificate mirror


By : user3512926
Date : March 29 2020, 07:55 AM
I wish did fix the issue. This doesn't work because there is no option called --mirror-no-check-certificate.
You may have intended to use the two separate options --mirror and --no-check-certificate:
code :
wget --mirror --no-check-certificate [your site goes here]
Subfolder redirect issue with static website hosting using S3, CloudFront and Origin Path

Subfolder redirect issue with static website hosting using S3, CloudFront and Origin Path


By : johan Vikström
Date : March 29 2020, 07:55 AM
I hope this helps . I ended up solving it by using routing rules for the S3 bucket
https://docs.aws.amazon.com/AmazonS3/latest/dev/HowDoIWebsiteConfiguration.html#configure-bucket-as-website-routing-rule-syntax
code :
<RoutingRules>
    <RoutingRule>
        <Condition>
            <KeyPrefixEquals>websites/website1/websites/website1/</KeyPrefixEquals>
        </Condition>
        <Redirect>
            <HostName>example.cloudfront.net</HostName>
            <ReplaceKeyPrefixWith></ReplaceKeyPrefixWith>
        </Redirect>
    </RoutingRule>
</RoutingRules>
Setting the proper root / redirect path for nginx website in subfolder

Setting the proper root / redirect path for nginx website in subfolder


By : Chungus
Date : March 29 2020, 07:55 AM
I think the issue was by ths following , I have 2 websites that can be both accessed at the same URL, only one is accessible from a subfolder like this: , It works with the current configuration:
code :
if (!-e $request_filename) {
    set $test P;
}
if ($uri !~ ^/(plesk-stat|webstat|webstat-ssl|ftpstat|anon_ftpstat|awstats-icon|internal-nginx-static-location|blog)) {
    set $test "${test}C";
}
if ($uri ~ ^/(blog)) {
    set $test "${test}B";
}
if ($test = PC ) {
    rewrite ^/(.*)$ /index.php?$1;
}
if ($test = PB ) {
    rewrite ^/(.*)$ /blog/index.php?$1;
}
Related Posts Related Posts :
  • Executable Deleting Itself on linux
  • How to change the apache httpd port for serving files
  • Installing Patches in linux
  • Comparative analysis between libkvm on linux and NetBSD
  • How to join multiple txt files into based on column?
  • setting LINK variable in Makefile generated by qmake (5.0)
  • Automake - difficulty with installing version 'automake-1.14.1'
  • How to use Linux Network Namespaces for per processes routing?
  • How can I write a Gruntfile with different shell command behaviour for OS X vs. Linux?
  • Using at command in shell script for scheduling a command
  • Is there a method to figure out the audio channel layout in Linux?
  • Setting environment varilable for daemon / root process
  • string replacement in shell script
  • Add specific column to the end of a CSV file
  • Are there programmable automount/autofs hooks in linux/systemd?
  • Renaming files like 20141207_190822.jpg to "2014-12-07 19.08.22.jpg" in linux or MacOS X
  • undocumented kernel boot parameter?
  • Using awk to find data matching date range in text file
  • Installing xorg-x11-devel package in SLES 11 SP2
  • Setting result of find to variable, which has variables within it
  • Maximum/Minimum physical address - Linux kernel
  • Freepbx custom context outbound route
  • memcpy implementation in linux kernel
  • How to reverse a list of words in a shell string?
  • Concatenate and pass as parameter in bash
  • Add user to Docker container
  • How to add a content in a file in linux without opening the file using vi or cat command
  • Which SVN subversion can I use on linux, Mac, Windows?
  • Work with ZIP archive at Bash
  • bash script not running as expected from cron vs. shell.
  • Compare one field, Remove duplicate if value of another field is greater
  • sudoers NOPASSWD: sudo: no tty present and no askpass program specified
  • How do I use pipes and variable substitution in a shell script (for toggling haproxy active server)?
  • Linux/Unix Find Command
  • Not able to copy data in hdfs with hdfs dfs commands
  • Linux : Remove/Delete .txt files ending with exactly five digits
  • ARM assembly "retne" instruction
  • Migrate from LXC to LXD
  • sbt on linux not accepting -jvm-debug 5005
  • Supervisord events only sent to one eventlistener in pool
  • Finding all .c files in linux using grep
  • Extract part of a file name in bash
  • Bash Script to Comment/uncomment lines in file
  • Linux audio control web alsamixer
  • Can't remove first two encode characters using text editors in linux
  • shadow
    Privacy Policy - Terms - Contact Us © ourworld-yourmove.org