Help! Transfer data between EC2 and S3

This forum is for issues related to parallel processing
and OpenSees using the new interpreters OpenSeesSP and OpenSeesMP

Help! Transfer data between EC2 and S3

Postby blzopensees » Thu Dec 22, 2016 12:11 am

I am trying to transfer earthquake records from S3 to EC2. I tryed to use Starcluster to start the instances with IAM role with reference to the following page:
http://star.mit.edu/cluster/mlarchives/2314.html
However, when I tried to copy the file 'main.tcl' from my bucket 'wodecunchutong' to directory 'root' on the instances, an error occurred as follows:

C:\Users\asus>starcluster put mycluster s3://wodecunchutong/main.tcl /root
StarCluster - (http://star.mit.edu/cluster) (v. 0.95.6)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu

!!! ERROR - Local file or directory does not exist:
!!! ERROR - s3://wodecunchutong/main.tcl

I also tried to use AWS CLI with reference to the following page:
http://stackoverflow.com/questions/1591 ... nd-aws-ec2
, but still failed:

C:\Users\asus>aws s3 cp s3://wodecunchutong/main.tcl /root --recursive
C:\Users\asus>starcluster sshmaster mycluster
StarCluster - (http://star.mit.edu/cluster) (v. 0.95.6)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu

>>> Starting Pure-Python SSH shell...
Line-buffered terminal emulation. Press F6 or ^Z to send EOF.

eval $(resize)
root@master:~# eval $(resize)
7
resize: Time out occurred
root@master:~# ls /root
ls /root

That is to say, nothing was copied to the /root directory on the EC2 instance.

Can anyone tell me how to copy files from EC2 to S3 bucket? I am new to Amazon EC2 and S3.Please give me some advice.
Thanks!
blzopensees
 
Posts: 2
Joined: Tue Dec 06, 2016 6:00 am

Re: Help! Transfer data between EC2 and S3

Postby fmk » Mon Jan 30, 2017 9:41 pm

it obviosly does not like the directory: s3://wodecunchutong/main.tcl

you could try just cd'ing into the directory and then issing the command:
starcluster put mycluster main.tcl /root
fmk
Site Admin
 
Posts: 5860
Joined: Fri Jun 11, 2004 2:33 pm
Location: UC Berkeley


Return to Parallel Processing

Who is online

Users browsing this forum: No registered users and 1 guest