EC2 AMI version

This forum is for issues related to parallel processing
and OpenSees using the new interpreters OpenSeesSP and OpenSeesMP
kengawk
Posts: 104
Joined: Wed Oct 31, 2007 11:55 pm
Location: Beijing JIaotong University
Contact:

Re: EC2 AMI version

Post by kengawk » Tue Aug 23, 2016 7:12 am

Thanks for your help!
Try it as your suggestion goes.

And now I think I found the problem is that the setuptool can not be installed successfullly.
The following is error when I am installing setuptool.
value error comes up.It warns that Entry points must be listed in groups.

https://drive.google.com/file/d/0B2M0Ty ... sp=sharing
Extracting in c:\users\admini~1.use\appdata\local\temp\tmpteovsq
Now working in c:\users\admini~1.use\appdata\local\temp\tmpteovsq\setuptools-26.
0.0
Installing Setuptools
Traceback (most recent call last):
File "setup.py", line 194, in <module>
dist = setuptools.setup(**setup_params)
File "C:\Python27\lib\distutils\core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "c:\users\admini~1.use\appdata\local\temp\tmpteovsq\setuptools-26.0.0\set
uptools\dist.py", line 349, in __init__
for ep in pkg_resources.iter_entry_points('distutils.setup_keywords'):
File "c:\users\admini~1.use\appdata\local\temp\tmpteovsq\setuptools-26.0.0\pkg
_resources\__init__.py", line 730, in iter_entry_points
entries = dist.get_entry_map(group)
File "c:\users\admini~1.use\appdata\local\temp\tmpteovsq\setuptools-26.0.0\pkg
_resources\__init__.py", line 2629, in get_entry_map
self._get_metadata('entry_points.txt'), self
File "c:\users\admini~1.use\appdata\local\temp\tmpteovsq\setuptools-26.0.0\pkg
_resources\__init__.py", line 2363, in parse_map
raise ValueError("Entry points must be listed in groups")
ValueError: Entry points must be listed in groups
Something went wrong during the installation.
See the error message above.


And I installed VS2012.

Thanks again! I really appreciate it.

cgtydmrc wrote:
> You shoud open command prompt in this directory
> "C:\Python27\Scripts" by shift button and right click. Then you
> should type "pip install StarCluster".
>
>
> ***You may also need to install Visual Studio if you confront any problems
> when installing StarCluster.*** This is very important.
>
> Hope this helps.
Kai Zhang
------------------------------
PH.D. candidate
School of Civil Engineering&Architecture
Beijing Jiaotong University

kengawk
Posts: 104
Joined: Wed Oct 31, 2007 11:55 pm
Location: Beijing JIaotong University
Contact:

Re: EC2 AMI version

Post by kengawk » Wed Aug 24, 2016 6:16 pm

Dear Cagatay,

Now I found that it's proper for me to make 32bit python installation so that I could install starcluster successfully

according to the instruction that you shared with me.

What is the most important info in the config file? I really feel a little bit confused.
Now I changed the segments in config file as below:
1AWS_ACCESS_KEY_ID = #your_aws_access_key_id
2AWS_SECRET_ACCESS_KEY = #your_secret_access_key
3AWS_USER_ID= #your userid
4[key mykey]
5KEY_LOCATION=~/.ssh/mykey.rsa
6KEYNAME = mykey

How can I revise the keypair? I creat key according to the contents from 'starcluster createkey -help'

G:\peer\AmazonEC2>starcluster createkey -i pk-APKAICAVVT6INVOFHD4Q.pem -o myfirs
t.rsa myfirst
StarCluster - (http://star.mit.edu/cluster) (v. 0.95.6)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu

>>> Successfully created keypair: myfirst
>>> fingerprint: 98:3a:ea:f0:60:65:80:be:aa:1b:61:32:52:1f:a0:85


and it seems to create a rsa file from keypair file with a keyname,but where's the rsa file? I can't find it in my computer.
How can I revise the config file and create key file correctly?

G:\peer\AmazonEC2>starcluster start smallcluster
StarCluster - (http://star.mit.edu/cluster) (v. 0.95.6)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu

>>> Using default cluster template: smallcluster
>>> Validating cluster template settings...
!!! ERROR - Cluster settings are not valid:
!!! ERROR - key_location
!!! ERROR - 'C:\Users\Administrator.USER-20160705YO/.ssh/myfirst.rsa'
!!! ERROR - does not exist

Cheers,

Kai

@cgtydmrc
cgtydmrc wrote:
> You shoud open command prompt in this directory
> "C:\Python27\Scripts" by shift button and right click. Then you
> should type "pip install StarCluster".
>
>
> ***You may also need to install Visual Studio if you confront any problems
> when installing StarCluster.*** This is very important.
>
> Hope this helps.
Kai Zhang
------------------------------
PH.D. candidate
School of Civil Engineering&Architecture
Beijing Jiaotong University

cgtydmrc
Posts: 61
Joined: Mon Dec 01, 2014 7:04 am
Location: United Kingdom

Re: EC2 AMI version

Post by cgtydmrc » Thu Aug 25, 2016 8:35 am

Hi Kai,

1AWS_ACCESS_KEY_ID = #your_aws_access_key_id
2AWS_SECRET_ACCESS_KEY = #your_secret_access_key
3AWS_USER_ID= #your userid
4[key mykey]
5KEY_LOCATION=~/.ssh/mykey.rsa
6KEYNAME = mykey
7Cluster size

Those information should be enough. Also cluster size you need to check.

cgtydmrc
Posts: 61
Joined: Mon Dec 01, 2014 7:04 am
Location: United Kingdom

Re: EC2 AMI version

Post by cgtydmrc » Thu Aug 25, 2016 8:46 am

You can also watch the video in the following link which helps you how to modify your own config file. http://star.mit.edu/cluster/index.html

kengawk
Posts: 104
Joined: Wed Oct 31, 2007 11:55 pm
Location: Beijing JIaotong University
Contact:

Re: EC2 AMI version

Post by kengawk » Fri Aug 26, 2016 7:27 am

Thank you,Cagatay!!
It is really a great help for me!

Now I met a problem once more according to that demo video, could you give me some tips?

I cannot start the cluster,the error messages is as the following.
C:\Users\Administrator.USER-20160705YO>starcluster start myfirstcluster
StarCluster - (http://star.mit.edu/cluster) (v. 0.95.6)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu

Thanks again.
I solved that problem.
1 starcluster createkey mykey -o ~/.ssh/mykey.rsa
2Revised the keypair name mykey to agree with the name from starcluster!
[key mykey]
KEY_LOCATION=C:/Users/Administrator.USER-20160705YO/.ssh/mykey.rsa
[cluster smallcluster]
# change this to the name of one of the keypair sections defined above
KEYNAME = mykey



C:\Users\Administrator.USER-20160705YO>starcluster start myfirstcluster
StarCluster - (http://star.mit.edu/cluster) (v. 0.95.6)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster@mit.edu

>>> Using default cluster template: smallcluster
>>> Validating cluster template settings...
>>> Cluster template settings are valid
>>> Starting cluster...
>>> Launching a 2-node cluster...
>>> Creating security group @sc-myfirstcluster...
Reservation:r-0b21043af727c806b
>>> Waiting for instances to propagate...
2/2 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Waiting for cluster to come up... (updating every 30s)
>>> Waiting for all nodes to be in a 'running' state...
2/2 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Waiting for SSH to come up on all nodes...
2/2 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Waiting for cluster to come up took 2.542 mins
>>> The master node is ec2-54-163-55-141.compute-1.amazonaws.com
>>> Configuring cluster...
>>> Running plugin starcluster.clustersetup.DefaultClusterSetup
>>> Configuring hostnames...
2/2 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Creating cluster user: sgeadmin (uid: 1001, gid: 1001)
2/2 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Configuring scratch space for user(s): sgeadmin
2/2 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Configuring /etc/hosts on each node
2/2 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Starting NFS server on master
>>> Configuring NFS exports path(s):
/home
>>> Mounting all NFS export path(s) on 1 worker node(s)
1/1 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Setting up NFS took 0.439 mins
>>> Configuring passwordless ssh for root
>>> Configuring passwordless ssh for sgeadmin
>>> Running plugin starcluster.plugins.sge.SGEPlugin
>>> Configuring SGE...
>>> Configuring NFS exports path(s):
/opt/sge6
>>> Mounting all NFS export path(s) on 1 worker node(s)
1/1 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Setting up NFS took 0.271 mins
>>> Installing Sun Grid Engine...
1/1 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Creating SGE parallel environment 'orte'
2/2 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 100%
>>> Adding parallel environment 'orte' to queue 'all.q'
>>> Configuring cluster took 3.287 mins
>>> Starting cluster took 5.919 mins

The cluster is now ready to use. To login to the master node
as root, run:

$ starcluster sshmaster myfirstcluster

If you're having issues with the cluster you can reboot the
instances and completely reconfigure the cluster from
scratch using:

$ starcluster restart myfirstcluster

When you're finished using the cluster and wish to terminate
it and stop paying for service:

$ starcluster terminate myfirstcluster

Alternatively, if the cluster uses EBS instances, you can
use the 'stop' command to shutdown all nodes and put them
into a 'stopped' state preserving the EBS volumes backing
the nodes:

$ starcluster stop myfirstcluster

WARNING: Any data stored in ephemeral storage (usually /mnt)
will be lost!

You can activate a 'stopped' cluster by passing the -x
option to the 'start' command:

$ starcluster start -x myfirstcluster

This will start all 'stopped' nodes and reconfigure the
cluster.







>>> Using default cluster template: smallcluster
>>> Validating cluster template settings...
!!! ERROR - Cluster settings are not valid:
!!! ERROR - Keypair 'mykeypairname' doe-------------s not exist in region 'us-east-1'

@cgtydmrc wrote:
> You can also watch the video in the following link which helps you how to
> modify your own config file. http://star.mit.edu/cluster/index.html
Kai Zhang
------------------------------
PH.D. candidate
School of Civil Engineering&Architecture
Beijing Jiaotong University

cgtydmrc
Posts: 61
Joined: Mon Dec 01, 2014 7:04 am
Location: United Kingdom

Re: EC2 AMI version

Post by cgtydmrc » Wed Sep 07, 2016 5:52 am

Do you still have any problem?

If you are developing large models in OpenSees, I would say, using starcluster is not an efficient way to perform your analysis. If you just have frame models, than you can use starcluster to run your analyses quicker.

Cheers,

egzonk
Posts: 1
Joined: Fri Oct 21, 2016 3:05 pm
Contact:

Re: EC2 AMI version

Post by egzonk » Fri Oct 21, 2016 3:09 pm

cgtydmrc wrote:
> You can also watch the video in the following link which helps you how to
> modify your own config file. http://star.mit.edu/cluster/index.html

Thank you very mutch sir , it rreally helped me. 8)
Free VPS 1
----------------]
Free VPS 2

Mick
Posts: 2
Joined: Fri Nov 30, 2018 4:32 am

Re: Hosting provider

Post by Mick » Sun Dec 02, 2018 11:45 pm

Nice to see your article. It is too much interesting.
http://www.onedollarwebhostings.com/

vickytaneja
Posts: 1
Joined: Fri Mar 22, 2019 5:22 am

Re: EC2 AMI version

Post by vickytaneja » Fri Mar 22, 2019 5:26 am

Your Content is as decent as you seem to be.

A debt of gratitude is in order for sharing such profitable data.

Kindly see this - https://malwarebrains.blogspot.com/2019 ... ivers.html

jk7070
Posts: 2
Joined: Wed May 11, 2011 11:55 am
Location: Netherlands

Re: EC2 AMI version

Post by jk7070 » Wed Jun 05, 2019 5:20 am

Hi,

Can the AmazonEC2 cluster be used with opensees files written in python (.py) rather the .tcl? I have models built using opensees.py and it would be nice if we could run these without having to rewrite in .tcl.

If it is currently not supported, is it likely it will be supported in the future and if so how far away is this likely to be?

Regards

Post Reply