How to create a project on My DataLake Services
This article walks you through the step-by-step process of creating a project on My DataLake Services — the gateway to accessing DestinE’s advanced big data analytics and edge computing capabilities. A project on My DataLake Services is an entity grouping users and roles. You’ll learn how to submit your project request, choose appropriate services, and configure access for your team.
What we are going to cover
Prerequisites
No. 1 Registered account
You need to have an account registered on My DataLake Services: How to create a profile on My DataLake Services
No. 2 How to install services
While creating My DataLake Services project (as described further on in this article), you will be offered to install one or more of the four available Edge services. Study these specialized articles before committing yourself:
How to submit request for hook service
How to submit request for stack-jupyter service
How to submit request for dask service
How to submit request for islet service
No. 3 Project types in DestinE
Article How to create a project on My DataLake Services explains in depth six different types of DestinE projects so that you can prepare all entries in advance.
Navigate to project creation form
Sign in to My DataLake Services if you haven’t already: https://application.data.destination-earth.eu
Use the panel on the left side of the website to navigate to page called Project:
If you can’t see this panel, you can display it by clicking the “hamburger button” in the top left corner of the website:
Create your own project or become a member of another project
Click on option Project and get two options, one is to create a project of your own, the other is to become a member of someone else’s project.
- Add project
Create a project of your own; cannot accept invitations but can send them.
- Use invitation code
Accept the invitation that someone else with a My DataLake Services account has sent you. Cannot create a project of your own (within one and the same account, that is).
In this article, we create a project only. For invitations, see
How to invite a user to a project on My DataLake Services
How to manage users within a project on My DataLake Services
Step 1: Decide to create a project
Click Add project:
You should now get to the following form:
Click on Next to come to Step 2.
Step 2: Fill in the project creation form
Start filling in the form.
Drop-down menu Project type allows you to choose the type of project you want to build:
Here is a summary of the six available project types you can choose from in the application form:
Project Type |
Description |
|---|---|
Public authority |
For EU government institutions and regulatory bodies |
Academia & research |
Universities and research institutions contributing to DestinE |
SME’s & start-ups |
Innovative businesses and technology developers |
3rd country public authority |
Government agencies outside the EU |
DestinE 3Es Staff |
Employees of ESA, ECMWF, or EUMETSAT |
DestinE 3Es Contractors |
Contractors working on behalf of the above agencies |
Refer to Prerequisite No. 3 for a full explanation of the six project types listed above. Here, we will just list the options for those six project types.
Project type: Public authority
If you choose this option, you will be prompted for EU legal entity:
Project type: Academia & research
If you choose this option, you will also be prompted for Academia description and the specific way you will be contributing to the development of DestinE project:
Project type: SME’s & start-ups
SME stands for Small or Medium-sized Enterprise (SME) while start-ups usually refer to newly formed businesses with high growth potential.
Project type: 3rd country public authority
If you choose this option, you will also be prompted for the type of your public authority:
Project type: DestinE 3Es ESA/ECMWF/EUMETSAT Staff
Are you currently employed by one of the core organizations behind the DestinE initiative (ESA, ECMWF, or EUMETSAT)?
Project type: DestinE 3Es ESA/ECMWF/EUMETSAT Contractors
Are you already working as one of the contractors for the base DestinE organizations?
Click on Next.
Step 3: Edge services
Select and use one or more of the four services offered in the project. The table below summarizes the four Edge services available when configuring your project:
Service |
Description |
|---|---|
hook |
A serverless data processing environment designed for triggering workflows on-demand or on a schedule. |
stack-jupyter |
Interactive JupyterLab environment for running Python code, exploring EO data, and building notebooks. |
stack-dask |
Scalable distributed computing service using Dask for parallel processing of large Earth Observation datasets. |
islet |
Provisioning of virtual machines and Kubernetes clusters through OpenStack for advanced data analysis tasks. |
You do not have to select any service now, but you can also choose one or more, up to all four.
See Prerequisite No. 2 for separate articles on autonomous installing of these four services.
Selecting a service also reserves associated resources for your project. They should align with the data you entered in Step 2. For instance, if you are a research team from academia, with the entire personnel of 12, it stands to reason to ask for islet-high quota role, which is meant to be used by up to 12 people. On the contrary, for a personnel of 3, the islet-low role quota should suffice.
Warning
The entire amount of resources you claim here will be taken into account when the Governance Board starts deciding whether to approve of your project or not.
When you decide, click on Next.
Step 4: Set the user limit
User limit determines how many individuals can join and contribute to your project. You, as the project admin, are always counted among them.
After choosing the services, you get to decide the number of users your project can have.
The minimum number is three and you will be counted as one of those three, being a project administrator.
Each user will claim their own resources which the Governance Board will also take into account. A conservative strategy for beginning is to start small on quota roles and expand further only when there is a real need.
Step 5: Finalize and submit the project request
In this last step of defining a project in My DataLake Services, you must explain why you want it and what you intend to do with it.
You must provide the following information:
thematic,
context,
objective,
intended usage of the requested EDGE services,
outcome and contribution to DestinE.
Mention any involvement you have in DestinE initiative or EU Horizon projects. The description of planned activities will be considered to support the decision for approving or rejecting the request (500 characters minimum).
Checklist before submitting
I selected the correct project type.
I justified the project with thematic, objective, and usage details.
I chose appropriate EDGE services and user limits.
I verified all entries and saved the request.
Finally, click on the Submit button. You should get the following confirmation:
Step 6: Adjust project request if necessary
Wait until your request receives approval from the Acceptance Board.
Click on Project requests to see the request in the list of requests. At first, it will be in the state of Pending. Click on Details to see the current state of data entry:
If you want to edit the entries, you can do that right away. When you are done editing, click on Edit request to fix the new values.
How to delete the request
The Delete request button will delete the project request before the operator has had a chance to approve it (or not).
Click on Delete request and confirm you want the current project request to be deleted:
Confirming it will actually delete the request:
and you will be back to Add project / Use invitation code options when clicking on Project.
If you add another project, the list of Project requests will now show two entries:
Availability of role requests
Even if your project is not yet officially approved, a new option under Access will appear: Role requests. Clicking on it will provide you the view of all roles that you can acquire in My DataLake Services:
As the default role, you have hda role activated, with the following parameters:
Depending on the services activated, the left side menu will change and provide further options for requests.
The project has been rejected
It may well happen that you requested a project but that the operator has not approved it. You may have even tried several times without success – a typical situation would be like this, after clicking on Project requests:
When there is no project available (none was requested or all of the existing projects have been rejected) the option Role requests will immediately cease to be available.
The project has been approved
When the project has been approved, it will be available in Project requests:
Click on Details to see the data entries for the project.
The main menu will have new options:
Click on Project and see the current state of project:
This is a typical situation right after the project has been created: you are the only user at the moment and the system automatically promotes you to admin role; and since you are the only admin, you are forbidden to leave the project – you may, however, invite other users to your project and assign them the admin role. Once they are admins, you will be able to leave the project.
Warning
You cannot create a project, have it accepted and then delete it, say, in order to start anew or explore new ideas and so on.
To send invitations to other users of your project, see
How to invite a user to a project on My DataLake Services
How to manage users within a project on My DataLake Services
Request access and quota roles for available services
Once you get a project in your My DataLake Services account, you have access to option Role requests, from which you will be able to request access and quota roles for all available services:
- eodata
Grants access to Earth Observation (EO) data repositories.
See How to obtain eodata S3 keys through My DataLake Services
- hda
Harmonised Data Access to existing data collections
See How to submit request for hda roles on My DataLake Services
- hook
Processing
See How to request roles for hook service on My DataLake Services
- islet
Access to OpenStack, virtual machines and Kubernetes clusters
See How to request roles for Islet service on My DataLake Services
- s3-object-storage
Access to data containers (object storage)
See How to obtain object storage S3 keys through My DataLake Services
- stack-dask
Dask environment
See How to request roles for Stack Dask on My DataLake Services
- stack-jupyter
Interactive programming in Python
See How to request roles for JupyterHub on My DataLake Services
What To Do Next
Once your project is approved, you can start working immediately or invite collaborators to join your team and request access roles.
If there is a problem with resources, experiment with quota roles. If still not sufficient to resolve the problems you are having, the links for support are in FAQs.