Date   
Goodbye

Xing Yang
 

Hi everyone,


I want to let you know that I won’t be involved in OpenSDS any more due to a career change.  OpenSDS community has grown a lot since I joined at the end of 2017.  We had 4 releases (Zealand, Aruba, Bali, and Capri) and built lots of new features.   We have an amazing developer community and a great end user community.  I am so proud of the achievement we’ve made so far.


Although I won’t be working on OpenSDS any more, I will still be in the open source community.   I’m sure our paths will cross in the near future.  I think OpenSDS is at an exciting time as it is transitioning into SODA.  I look forward to hearing good news from SODA!  Thank you all for the great experience! 


Best regards,

Xing

OpenSDS Technical Meeting Notice - September 6, 2019, 1 AM UTC [Zoom ID : 229-373-941]

Sanil Kumar
 

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: September 6, 2019, 1 AM UTC
Join Zoom Meeting
https://zoom.us/j/229373941  

Meeting agenda is here:

Please note that I have added one section in the agenda sheet for future topics of interest. Please feel free to add or inform me. Thanks,

Additional Meeting Information:
OpenSDS Project is inviting you to a scheduled Zoom meeting.

Topic: [OpenSDS Technical Meeting Invitation] Friday Biweekly 1am to 2am UTC
Time: Sep 6, 2019 01:00 AM Universal Time UTC
        Every 2 weeks on Fri, until Dec 27, 2019, 9 occurrence(s)
        Sep 6, 2019 01:00 AM
        Sep 20, 2019 01:00 AM
        Oct 4, 2019 01:00 AM
        Oct 18, 2019 01:00 AM
        Nov 1, 2019 01:00 AM
        Nov 15, 2019 01:00 AM
        Nov 29, 2019 01:00 AM
        Dec 13, 2019 01:00 AM
        Dec 27, 2019 01:00 AM
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/v5YkdO6rpz8re4QNiemevZQdNHp8ymdSfw/ics?icsToken=7c907caf5fa3d1b447a58f41d1274a91af82d89500405f878a5ca2a5c9e42231

Join Zoom Meeting
https://zoom.us/j/229373941

One tap mobile
+19294362866,,229373941# US (New York)
+16699006833,,229373941# US (San Jose)

Dial by your location
        +1 929 436 2866 US (New York)
        +1 669 900 6833 US (San Jose)
Meeting ID: 229 373 941
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Mon, Aug 26, 2019 at 11:57 AM Sanil Kumar <skdsanil@...> wrote:

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: Aug 27, 2019 01:00 PM Universal Time UTC
Join Zoom Meeting
https://zoom.us/j/777978108

Meeting agenda is here:
[Please free to suggest topics]


Additional info:
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/upMqfu6grzsit2MOj5WStzsoRXLYq_gSgQ/ics?icsToken=292ae42cfcf6f4b639bb21c89e15e2a736567d31908a994a8e2988f6819247c0

One tap mobile
+16699006833,,777978108# US (San Jose)
+19294362866,,777978108# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 929 436 2866 US (New York)
Meeting ID: 777 978 108
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Fri, Aug 23, 2019 at 8:01 AM Xing Yang <xingyang105@...> wrote:

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

Re: [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019

Xing Yang
 

Just to clarify.  User has two choices.  1. Use YIG.  In this case, Ceph radosgw is enhanced by YIG so it is no longer the same.  2. Use Ceph object store as the way it is.  Ceph radosgw remains the same.


On Wed, Aug 28, 2019 at 4:53 PM Xing Yang <xingyang105@...> wrote:
Ceph radosgw is enhanced by YIG so it is no longer the same.

On Wed, Aug 28, 2019 at 4:40 PM Steven Tan <steven.tan@...> wrote:
Are we making any changes to the Ceph radosgw, or using as is?

Steven Tan

VP & CTO Cloud Solution, Storage

Linux Foundation OpenSDS TSC Chair 

email: steven.tan@...

twitter: stevenphtan


This e-mail and any attachments may contain confidential information from Futurewei/Huawei, which are intended only for the person or entity whose email address appears above. Any use of the information attached or contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or any unapproved dissemination) by persons other than the intended recipient(s) is prohibited. If you have received this e-mail in error, please notify the sender by phone or response email immediately and delete this original message.



From: Sanil Kumar <skdsanil@...>
Sent: Wednesday, August 28, 2019 8:01 AM
To: Xing Yang <xing.yang@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>; Steven Tan <steven.tan@...>
Subject: Re: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
+some points to Xing. 
(Thanks Ashit)

Ceph (radosgw)

YIG (Yet another Index Gateway)

S3 and Swift compatible Object Storage Gateway

S3 compatible Object Storage Gateway

Performance impact while storing small files

Uses KV store to overcome small file storage issue

Performance impact for read and write of large files

Overcome read write issue of large files using optimized libradosstripper API

Not so scalable Ceph OSD nodes

Using write consistency and dynamic buf it enables scaling of ceph OSD nodes


Avoids data movement and IO drop due o scale out clusters


Allows more than one ceph cluster to work together and extend the storage pool capacity

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Tue, Aug 27, 2019 at 10:58 PM Xing Yang <xing.yang@...> wrote:
"Ceph" uses Ceph's original S3 gateway.  "YIG S3" uses enhanced Ceph S3 gateway from YIG.  The plan is to keep both so user have two choices.

From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Steven Tan via Lists.Opensds.Io <steven.tan=futurewei.com@...>
Sent: Tuesday, August 27, 2019 12:00 PM
To: skdsanil@... <skdsanil@...>; Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: Re: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
What are the differences between Ceph and YIG S3?

Steven Tan
VP & CTO Cloud Solution, Storage
OpenSDS TSC Chairman
————————————————————————
This e-mail and any attachments may contain confidential information from Futurewei/Huawei, which are intended only for the person or entity whose email address appears above. Any use of the information attached or contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or any unapproved dissemination) by persons other than the intended recipient(s) is prohibited. If you have received this e-mail in error, please notify the sender by phone or response email immediately and delete this original message.

From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Sanil Kumar via Lists.Opensds.Io <skdsanil=gmail.com@...>
Sent: Tuesday, August 27, 2019 7:25:14 AM
To: opensds-tech-discuss@... <opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
Dear All,
Please find the the key updates of today's technical meeting.
Topic Discussed:
  • [MultiCloud : ChinaUnicom(yu zou)/Fanny/Ashit ] Multicloud Design Refactoring - Key updates

    • What is new, architecture and how we plan to go about

    • Integration of YIG (YIG Ceph S3 added! And other features from YIG to multicloud)

    • We keep the current ceph s3 along with new yig ceph s3. User has the option to choose.

    • YIG team will merge the updated code to Gelato/Multicloud

    • Action : YIG features and new features can be segregated; table for support from different clouds and backend can be prepared.

    • Currently support TiDB first. 

    • [HelpNeeded] for MongoDB

    • [HelpNeeded]GCP Adapter can be refactored

    • IBM Cloud support - Need to check. Check with Rakesh.

    • DB Adapter separation should consider the microservice architecture. The communication between microservice is grpc

    • Action : Architecture diagram update and add to the existing design doc

  • The slides presented today is available at
  • https://drive.google.com/file/d/1JsulzsDrmwgAtme8MIO2LIoAIuAbsbMO/view?usp=sharing 
  • The MoM of today's meeting and agenda for the next meeting available at https://docs.google.com/document/d/1JlxAAOtvZvvf_KhVr8XQa6mUD7lkHOXlxuGruTKEukE/edit# (You can suggest meeting topics for next meetings there)
- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Mon, Aug 26, 2019 at 11:57 AM Sanil Kumar <skdsanil@...> wrote:

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: Aug 27, 2019 01:00 PM Universal Time UTC
Join Zoom Meeting
https://zoom.us/j/777978108

Meeting agenda is here:
[Please free to suggest topics]


Additional info:
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/upMqfu6grzsit2MOj5WStzsoRXLYq_gSgQ/ics?icsToken=292ae42cfcf6f4b639bb21c89e15e2a736567d31908a994a8e2988f6819247c0

One tap mobile
+16699006833,,777978108# US (San Jose)
+19294362866,,777978108# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 929 436 2866 US (New York)
Meeting ID: 777 978 108
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Fri, Aug 23, 2019 at 8:01 AM Xing Yang <xingyang105@...> wrote:

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

Re: [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019

Xing Yang
 

Ceph radosgw is enhanced by YIG so it is no longer the same.

On Wed, Aug 28, 2019 at 4:40 PM Steven Tan <steven.tan@...> wrote:
Are we making any changes to the Ceph radosgw, or using as is?

Steven Tan

VP & CTO Cloud Solution, Storage

Linux Foundation OpenSDS TSC Chair 

email: steven.tan@...

twitter: stevenphtan


This e-mail and any attachments may contain confidential information from Futurewei/Huawei, which are intended only for the person or entity whose email address appears above. Any use of the information attached or contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or any unapproved dissemination) by persons other than the intended recipient(s) is prohibited. If you have received this e-mail in error, please notify the sender by phone or response email immediately and delete this original message.



From: Sanil Kumar <skdsanil@...>
Sent: Wednesday, August 28, 2019 8:01 AM
To: Xing Yang <xing.yang@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>; Steven Tan <steven.tan@...>
Subject: Re: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
+some points to Xing. 
(Thanks Ashit)

Ceph (radosgw)

YIG (Yet another Index Gateway)

S3 and Swift compatible Object Storage Gateway

S3 compatible Object Storage Gateway

Performance impact while storing small files

Uses KV store to overcome small file storage issue

Performance impact for read and write of large files

Overcome read write issue of large files using optimized libradosstripper API

Not so scalable Ceph OSD nodes

Using write consistency and dynamic buf it enables scaling of ceph OSD nodes


Avoids data movement and IO drop due o scale out clusters


Allows more than one ceph cluster to work together and extend the storage pool capacity

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Tue, Aug 27, 2019 at 10:58 PM Xing Yang <xing.yang@...> wrote:
"Ceph" uses Ceph's original S3 gateway.  "YIG S3" uses enhanced Ceph S3 gateway from YIG.  The plan is to keep both so user have two choices.

From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Steven Tan via Lists.Opensds.Io <steven.tan=futurewei.com@...>
Sent: Tuesday, August 27, 2019 12:00 PM
To: skdsanil@... <skdsanil@...>; Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: Re: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
What are the differences between Ceph and YIG S3?

Steven Tan
VP & CTO Cloud Solution, Storage
OpenSDS TSC Chairman
————————————————————————
This e-mail and any attachments may contain confidential information from Futurewei/Huawei, which are intended only for the person or entity whose email address appears above. Any use of the information attached or contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or any unapproved dissemination) by persons other than the intended recipient(s) is prohibited. If you have received this e-mail in error, please notify the sender by phone or response email immediately and delete this original message.

From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Sanil Kumar via Lists.Opensds.Io <skdsanil=gmail.com@...>
Sent: Tuesday, August 27, 2019 7:25:14 AM
To: opensds-tech-discuss@... <opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
Dear All,
Please find the the key updates of today's technical meeting.
Topic Discussed:
  • [MultiCloud : ChinaUnicom(yu zou)/Fanny/Ashit ] Multicloud Design Refactoring - Key updates

    • What is new, architecture and how we plan to go about

    • Integration of YIG (YIG Ceph S3 added! And other features from YIG to multicloud)

    • We keep the current ceph s3 along with new yig ceph s3. User has the option to choose.

    • YIG team will merge the updated code to Gelato/Multicloud

    • Action : YIG features and new features can be segregated; table for support from different clouds and backend can be prepared.

    • Currently support TiDB first. 

    • [HelpNeeded] for MongoDB

    • [HelpNeeded]GCP Adapter can be refactored

    • IBM Cloud support - Need to check. Check with Rakesh.

    • DB Adapter separation should consider the microservice architecture. The communication between microservice is grpc

    • Action : Architecture diagram update and add to the existing design doc

  • The slides presented today is available at
  • https://drive.google.com/file/d/1JsulzsDrmwgAtme8MIO2LIoAIuAbsbMO/view?usp=sharing 
  • The MoM of today's meeting and agenda for the next meeting available at https://docs.google.com/document/d/1JlxAAOtvZvvf_KhVr8XQa6mUD7lkHOXlxuGruTKEukE/edit# (You can suggest meeting topics for next meetings there)
- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Mon, Aug 26, 2019 at 11:57 AM Sanil Kumar <skdsanil@...> wrote:

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: Aug 27, 2019 01:00 PM Universal Time UTC
Join Zoom Meeting
https://zoom.us/j/777978108

Meeting agenda is here:
[Please free to suggest topics]


Additional info:
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/upMqfu6grzsit2MOj5WStzsoRXLYq_gSgQ/ics?icsToken=292ae42cfcf6f4b639bb21c89e15e2a736567d31908a994a8e2988f6819247c0

One tap mobile
+16699006833,,777978108# US (San Jose)
+19294362866,,777978108# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 929 436 2866 US (New York)
Meeting ID: 777 978 108
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Fri, Aug 23, 2019 at 8:01 AM Xing Yang <xingyang105@...> wrote:

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

Re: [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019

Steven Tan
 

Are we making any changes to the Ceph radosgw, or using as is?

Steven Tan

VP & CTO Cloud Solution, Storage

Linux Foundation OpenSDS TSC Chair 

email: steven.tan@...

twitter: stevenphtan


This e-mail and any attachments may contain confidential information from Futurewei/Huawei, which are intended only for the person or entity whose email address appears above. Any use of the information attached or contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or any unapproved dissemination) by persons other than the intended recipient(s) is prohibited. If you have received this e-mail in error, please notify the sender by phone or response email immediately and delete this original message.



From: Sanil Kumar <skdsanil@...>
Sent: Wednesday, August 28, 2019 8:01 AM
To: Xing Yang <xing.yang@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>; Steven Tan <steven.tan@...>
Subject: Re: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
+some points to Xing. 
(Thanks Ashit)

Ceph (radosgw)

YIG (Yet another Index Gateway)

S3 and Swift compatible Object Storage Gateway

S3 compatible Object Storage Gateway

Performance impact while storing small files

Uses KV store to overcome small file storage issue

Performance impact for read and write of large files

Overcome read write issue of large files using optimized libradosstripper API

Not so scalable Ceph OSD nodes

Using write consistency and dynamic buf it enables scaling of ceph OSD nodes


Avoids data movement and IO drop due o scale out clusters


Allows more than one ceph cluster to work together and extend the storage pool capacity

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Tue, Aug 27, 2019 at 10:58 PM Xing Yang <xing.yang@...> wrote:
"Ceph" uses Ceph's original S3 gateway.  "YIG S3" uses enhanced Ceph S3 gateway from YIG.  The plan is to keep both so user have two choices.

From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Steven Tan via Lists.Opensds.Io <steven.tan=futurewei.com@...>
Sent: Tuesday, August 27, 2019 12:00 PM
To: skdsanil@... <skdsanil@...>; Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: Re: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
What are the differences between Ceph and YIG S3?

Steven Tan
VP & CTO Cloud Solution, Storage
OpenSDS TSC Chairman
————————————————————————
This e-mail and any attachments may contain confidential information from Futurewei/Huawei, which are intended only for the person or entity whose email address appears above. Any use of the information attached or contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or any unapproved dissemination) by persons other than the intended recipient(s) is prohibited. If you have received this e-mail in error, please notify the sender by phone or response email immediately and delete this original message.

From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Sanil Kumar via Lists.Opensds.Io <skdsanil=gmail.com@...>
Sent: Tuesday, August 27, 2019 7:25:14 AM
To: opensds-tech-discuss@... <opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
Dear All,
Please find the the key updates of today's technical meeting.
Topic Discussed:
  • [MultiCloud : ChinaUnicom(yu zou)/Fanny/Ashit ] Multicloud Design Refactoring - Key updates

    • What is new, architecture and how we plan to go about

    • Integration of YIG (YIG Ceph S3 added! And other features from YIG to multicloud)

    • We keep the current ceph s3 along with new yig ceph s3. User has the option to choose.

    • YIG team will merge the updated code to Gelato/Multicloud

    • Action : YIG features and new features can be segregated; table for support from different clouds and backend can be prepared.

    • Currently support TiDB first. 

    • [HelpNeeded] for MongoDB

    • [HelpNeeded]GCP Adapter can be refactored

    • IBM Cloud support - Need to check. Check with Rakesh.

    • DB Adapter separation should consider the microservice architecture. The communication between microservice is grpc

    • Action : Architecture diagram update and add to the existing design doc

  • The slides presented today is available at
  • https://drive.google.com/file/d/1JsulzsDrmwgAtme8MIO2LIoAIuAbsbMO/view?usp=sharing 
  • The MoM of today's meeting and agenda for the next meeting available at https://docs.google.com/document/d/1JlxAAOtvZvvf_KhVr8XQa6mUD7lkHOXlxuGruTKEukE/edit# (You can suggest meeting topics for next meetings there)
- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Mon, Aug 26, 2019 at 11:57 AM Sanil Kumar <skdsanil@...> wrote:

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: Aug 27, 2019 01:00 PM Universal Time UTC
Join Zoom Meeting
https://zoom.us/j/777978108

Meeting agenda is here:
[Please free to suggest topics]


Additional info:
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/upMqfu6grzsit2MOj5WStzsoRXLYq_gSgQ/ics?icsToken=292ae42cfcf6f4b639bb21c89e15e2a736567d31908a994a8e2988f6819247c0

One tap mobile
+16699006833,,777978108# US (San Jose)
+19294362866,,777978108# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 929 436 2866 US (New York)
Meeting ID: 777 978 108
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Fri, Aug 23, 2019 at 8:01 AM Xing Yang <xingyang105@...> wrote:

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

Re: [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019

Sanil Kumar
 

+some points to Xing. 
(Thanks Ashit)

Ceph (radosgw)

YIG (Yet another Index Gateway)

S3 and Swift compatible Object Storage Gateway

S3 compatible Object Storage Gateway

Performance impact while storing small files

Uses KV store to overcome small file storage issue

Performance impact for read and write of large files

Overcome read write issue of large files using optimized libradosstripper API

Not so scalable Ceph OSD nodes

Using write consistency and dynamic buf it enables scaling of ceph OSD nodes


Avoids data movement and IO drop due o scale out clusters


Allows more than one ceph cluster to work together and extend the storage pool capacity

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Tue, Aug 27, 2019 at 10:58 PM Xing Yang <xing.yang@...> wrote:
"Ceph" uses Ceph's original S3 gateway.  "YIG S3" uses enhanced Ceph S3 gateway from YIG.  The plan is to keep both so user have two choices.

From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Steven Tan via Lists.Opensds.Io <steven.tan=futurewei.com@...>
Sent: Tuesday, August 27, 2019 12:00 PM
To: skdsanil@... <skdsanil@...>; Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: Re: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
What are the differences between Ceph and YIG S3?

Steven Tan
VP & CTO Cloud Solution, Storage
OpenSDS TSC Chairman
————————————————————————
This e-mail and any attachments may contain confidential information from Futurewei/Huawei, which are intended only for the person or entity whose email address appears above. Any use of the information attached or contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or any unapproved dissemination) by persons other than the intended recipient(s) is prohibited. If you have received this e-mail in error, please notify the sender by phone or response email immediately and delete this original message.

From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Sanil Kumar via Lists.Opensds.Io <skdsanil=gmail.com@...>
Sent: Tuesday, August 27, 2019 7:25:14 AM
To: opensds-tech-discuss@... <opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
Dear All,
Please find the the key updates of today's technical meeting.
Topic Discussed:
  • [MultiCloud : ChinaUnicom(yu zou)/Fanny/Ashit ] Multicloud Design Refactoring - Key updates

    • What is new, architecture and how we plan to go about

    • Integration of YIG (YIG Ceph S3 added! And other features from YIG to multicloud)

    • We keep the current ceph s3 along with new yig ceph s3. User has the option to choose.

    • YIG team will merge the updated code to Gelato/Multicloud

    • Action : YIG features and new features can be segregated; table for support from different clouds and backend can be prepared.

    • Currently support TiDB first. 

    • [HelpNeeded] for MongoDB

    • [HelpNeeded]GCP Adapter can be refactored

    • IBM Cloud support - Need to check. Check with Rakesh.

    • DB Adapter separation should consider the microservice architecture. The communication between microservice is grpc

    • Action : Architecture diagram update and add to the existing design doc

  • The slides presented today is available at
  • https://drive.google.com/file/d/1JsulzsDrmwgAtme8MIO2LIoAIuAbsbMO/view?usp=sharing 
  • The MoM of today's meeting and agenda for the next meeting available at https://docs.google.com/document/d/1JlxAAOtvZvvf_KhVr8XQa6mUD7lkHOXlxuGruTKEukE/edit# (You can suggest meeting topics for next meetings there)
- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Mon, Aug 26, 2019 at 11:57 AM Sanil Kumar <skdsanil@...> wrote:

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: Aug 27, 2019 01:00 PM Universal Time UTC
Join Zoom Meeting
https://zoom.us/j/777978108

Meeting agenda is here:
[Please free to suggest topics]


Additional info:
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/upMqfu6grzsit2MOj5WStzsoRXLYq_gSgQ/ics?icsToken=292ae42cfcf6f4b639bb21c89e15e2a736567d31908a994a8e2988f6819247c0

One tap mobile
+16699006833,,777978108# US (San Jose)
+19294362866,,777978108# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 929 436 2866 US (New York)
Meeting ID: 777 978 108
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Fri, Aug 23, 2019 at 8:01 AM Xing Yang <xingyang105@...> wrote:

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

Re: [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019

Xing Yang
 

"Ceph" uses Ceph's original S3 gateway.  "YIG S3" uses enhanced Ceph S3 gateway from YIG.  The plan is to keep both so user have two choices.


From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Steven Tan via Lists.Opensds.Io <steven.tan=futurewei.com@...>
Sent: Tuesday, August 27, 2019 12:00 PM
To: skdsanil@... <skdsanil@...>; Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: Re: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
What are the differences between Ceph and YIG S3?

Steven Tan
VP & CTO Cloud Solution, Storage
OpenSDS TSC Chairman
Email: steven.tan@...
————————————————————————
This e-mail and any attachments may contain confidential information from Futurewei/Huawei, which are intended only for the person or entity whose email address appears above. Any use of the information attached or contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or any unapproved dissemination) by persons other than the intended recipient(s) is prohibited. If you have received this e-mail in error, please notify the sender by phone or response email immediately and delete this original message.

From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Sanil Kumar via Lists.Opensds.Io <skdsanil=gmail.com@...>
Sent: Tuesday, August 27, 2019 7:25:14 AM
To: opensds-tech-discuss@... <opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
Dear All,
Please find the the key updates of today's technical meeting.
Topic Discussed:
  • [MultiCloud : ChinaUnicom(yu zou)/Fanny/Ashit ] Multicloud Design Refactoring - Key updates

    • What is new, architecture and how we plan to go about

    • Integration of YIG (YIG Ceph S3 added! And other features from YIG to multicloud)

    • We keep the current ceph s3 along with new yig ceph s3. User has the option to choose.

    • YIG team will merge the updated code to Gelato/Multicloud

    • Action : YIG features and new features can be segregated; table for support from different clouds and backend can be prepared.

    • Currently support TiDB first. 

    • [HelpNeeded] for MongoDB

    • [HelpNeeded]GCP Adapter can be refactored

    • IBM Cloud support - Need to check. Check with Rakesh.

    • DB Adapter separation should consider the microservice architecture. The communication between microservice is grpc

    • Action : Architecture diagram update and add to the existing design doc

  • The slides presented today is available at
  • https://drive.google.com/file/d/1JsulzsDrmwgAtme8MIO2LIoAIuAbsbMO/view?usp=sharing 
  • The MoM of today's meeting and agenda for the next meeting available at https://docs.google.com/document/d/1JlxAAOtvZvvf_KhVr8XQa6mUD7lkHOXlxuGruTKEukE/edit# (You can suggest meeting topics for next meetings there)
- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Mon, Aug 26, 2019 at 11:57 AM Sanil Kumar <skdsanil@...> wrote:

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: Aug 27, 2019 01:00 PM Universal Time UTC
Join Zoom Meeting
https://zoom.us/j/777978108

Meeting agenda is here:
[Please free to suggest topics]


Additional info:
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/upMqfu6grzsit2MOj5WStzsoRXLYq_gSgQ/ics?icsToken=292ae42cfcf6f4b639bb21c89e15e2a736567d31908a994a8e2988f6819247c0

One tap mobile
+16699006833,,777978108# US (San Jose)
+19294362866,,777978108# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 929 436 2866 US (New York)
Meeting ID: 777 978 108
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Fri, Aug 23, 2019 at 8:01 AM Xing Yang <xingyang105@...> wrote:

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

Re: [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019

Steven Tan
 

What are the differences between Ceph and YIG S3?

Steven Tan
VP & CTO Cloud Solution, Storage
OpenSDS TSC Chairman
Email: steven.tan@...
————————————————————————
This e-mail and any attachments may contain confidential information from Futurewei/Huawei, which are intended only for the person or entity whose email address appears above. Any use of the information attached or contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or any unapproved dissemination) by persons other than the intended recipient(s) is prohibited. If you have received this e-mail in error, please notify the sender by phone or response email immediately and delete this original message.


From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Sanil Kumar via Lists.Opensds.Io <skdsanil=gmail.com@...>
Sent: Tuesday, August 27, 2019 7:25:14 AM
To: opensds-tech-discuss@... <opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: [Opensds-tech-discuss] [Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019
 
Dear All,
Please find the the key updates of today's technical meeting.
Topic Discussed:
  • [MultiCloud : ChinaUnicom(yu zou)/Fanny/Ashit ] Multicloud Design Refactoring - Key updates

    • What is new, architecture and how we plan to go about

    • Integration of YIG (YIG Ceph S3 added! And other features from YIG to multicloud)

    • We keep the current ceph s3 along with new yig ceph s3. User has the option to choose.

    • YIG team will merge the updated code to Gelato/Multicloud

    • Action : YIG features and new features can be segregated; table for support from different clouds and backend can be prepared.

    • Currently support TiDB first. 

    • [HelpNeeded] for MongoDB

    • [HelpNeeded]GCP Adapter can be refactored

    • IBM Cloud support - Need to check. Check with Rakesh.

    • DB Adapter separation should consider the microservice architecture. The communication between microservice is grpc

    • Action : Architecture diagram update and add to the existing design doc

  • The slides presented today is available at
  • https://drive.google.com/file/d/1JsulzsDrmwgAtme8MIO2LIoAIuAbsbMO/view?usp=sharing 
  • The MoM of today's meeting and agenda for the next meeting available at https://docs.google.com/document/d/1JlxAAOtvZvvf_KhVr8XQa6mUD7lkHOXlxuGruTKEukE/edit# (You can suggest meeting topics for next meetings there)
- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Mon, Aug 26, 2019 at 11:57 AM Sanil Kumar <skdsanil@...> wrote:

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: Aug 27, 2019 01:00 PM Universal Time UTC
Join Zoom Meeting
https://zoom.us/j/777978108

Meeting agenda is here:
[Please free to suggest topics]


Additional info:
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/upMqfu6grzsit2MOj5WStzsoRXLYq_gSgQ/ics?icsToken=292ae42cfcf6f4b639bb21c89e15e2a736567d31908a994a8e2988f6819247c0

One tap mobile
+16699006833,,777978108# US (San Jose)
+19294362866,,777978108# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 929 436 2866 US (New York)
Meeting ID: 777 978 108
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Fri, Aug 23, 2019 at 8:01 AM Xing Yang <xingyang105@...> wrote:

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

[Minutes of Meeting] OpenSDS Technical Meeting - August 27, 2019

Sanil Kumar
 

Dear All,
Please find the the key updates of today's technical meeting.
Topic Discussed:
  • [MultiCloud : ChinaUnicom(yu zou)/Fanny/Ashit ] Multicloud Design Refactoring - Key updates

    • What is new, architecture and how we plan to go about

    • Integration of YIG (YIG Ceph S3 added! And other features from YIG to multicloud)

    • We keep the current ceph s3 along with new yig ceph s3. User has the option to choose.

    • YIG team will merge the updated code to Gelato/Multicloud

    • Action : YIG features and new features can be segregated; table for support from different clouds and backend can be prepared.

    • Currently support TiDB first. 

    • [HelpNeeded] for MongoDB

    • [HelpNeeded]GCP Adapter can be refactored

    • IBM Cloud support - Need to check. Check with Rakesh.

    • DB Adapter separation should consider the microservice architecture. The communication between microservice is grpc

    • Action : Architecture diagram update and add to the existing design doc

  • The slides presented today is available at
  • https://drive.google.com/file/d/1JsulzsDrmwgAtme8MIO2LIoAIuAbsbMO/view?usp=sharing 
  • The MoM of today's meeting and agenda for the next meeting available at https://docs.google.com/document/d/1JlxAAOtvZvvf_KhVr8XQa6mUD7lkHOXlxuGruTKEukE/edit# (You can suggest meeting topics for next meetings there)
- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Mon, Aug 26, 2019 at 11:57 AM Sanil Kumar <skdsanil@...> wrote:

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: Aug 27, 2019 01:00 PM Universal Time UTC
Join Zoom Meeting
https://zoom.us/j/777978108

Meeting agenda is here:
[Please free to suggest topics]


Additional info:
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/upMqfu6grzsit2MOj5WStzsoRXLYq_gSgQ/ics?icsToken=292ae42cfcf6f4b639bb21c89e15e2a736567d31908a994a8e2988f6819247c0

One tap mobile
+16699006833,,777978108# US (San Jose)
+19294362866,,777978108# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 929 436 2866 US (New York)
Meeting ID: 777 978 108
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Fri, Aug 23, 2019 at 8:01 AM Xing Yang <xingyang105@...> wrote:

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

OpenSDS Technical Meeting Notice - August 27, 2019, 1PM UTC [Zoom ID : 777978108]

Sanil Kumar
 

Dear All,

Please find the meeting information for next OpenSDS technical meeting
Time: Aug 27, 2019 01:00 PM Universal Time UTC
Join Zoom Meeting
https://zoom.us/j/777978108

Meeting agenda is here:
[Please free to suggest topics]


Additional info:
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://zoom.us/meeting/upMqfu6grzsit2MOj5WStzsoRXLYq_gSgQ/ics?icsToken=292ae42cfcf6f4b639bb21c89e15e2a736567d31908a994a8e2988f6819247c0

One tap mobile
+16699006833,,777978108# US (San Jose)
+19294362866,,777978108# US (New York)

Dial by your location
        +1 669 900 6833 US (San Jose)
        +1 929 436 2866 US (New York)
Meeting ID: 777 978 108
Find your local number: https://zoom.us/u/acN4guPFkJ

- 
Thanks and Regards
Sanil.
"Lets make things simple and better, today, tomorrow and always!"


On Fri, Aug 23, 2019 at 8:01 AM Xing Yang <xingyang105@...> wrote:

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

OpenSDS Technical Meeting Minutes - August 22, 2019

Xing Yang
 

Thank you for attending today's technical meeting. Here are the meeting minutes.


Agenda/Notes:

  • Design reviews

    • [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

      • Click2Cloud discussed about Abort Migration feature.

      • Design is described here: https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

      • Approach 1 proposes to add an AbortMigration flag.

      • Status of each object should be tracked.  Roll back should happen if aborted.

      • Source object should not be deleted until all objects are migrated. This way it is easier to roll back.

      • Need a state machine to keep track of object status.

      • Aborted and Failed can share lots of common code path.

      • Consider using Redis for caching when it is added with YIG integration.

  • Development update

    • [Naju and Himanshu)] Naju gave an update on VMware vSphere Web Client integration.  Plugin will be named to “OpenSDS Storage Plugin”. Implemented for block storage create/attach operation.  Tested on LVM. Third party storage support can be added. Himanshu gave a demo to show how the plugin works.

    • Does this work with Hotpot multi-dock (one end-point)?  In theory it should work. Will need to verify. Also Hotpot multi-dock itself has bugs that need to be ironed out first.

    • Design spec will be updated.

  • Note: As mentioned in the previous meeting, Tuesday’s meetings will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).  Sanil will send out a new meeting invite.


Thanks,
Xing



On Wed, Aug 21, 2019 at 8:01 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

OpenSDS Technical Meeting Reminder

Xing Yang
 

Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 21, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 22, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

OpenSDS Technical Meeting Minutes - August 8, 2019

Xing Yang
 

Thank you for attending today's technical meeting.  Here are the meeting minutes.

Agenda/Notes:

  • Design reviews

    • [Naju and Himanshu)] Naju talked about VMware vSphere Web Client integration and did a live demo

      • We need to investigate more on whether this plugin can be easily extended to support other storage systems

      • Slides attached.

    • [Ashit] Ashit presented YIG integration design spec: https://github.com/opensds/design-specs/pull/45

      • Kei has concerns about integration with YIG

      • Ashit will update the design spec to clarify the following:

        • Ceph object store backend is not required after YIG integration (because some end users don’t want to use Ceph).

        • What benefits from YIG are for all object store backends including AWS S3, Azure Blob, and other non-Ceph local object store backends.

        • What benefits from YIG are only for Ceph.

        • List new features that will be brought by YIG integration and existing multi-cloud features separately.

      • We’ll need more discussions on this integration

    •  [Click2Cloud] Multi-Cloud Project enhancement (Abort Migration)

https://drive.google.com/file/d/1SoarTRLJTWuiiYoZ9IjfuE4TS2UA_G2T/view?usp=sharing

  • Running out of time. Will discuss in the next meeting.


  • Next week’s meeting on August 13th will be canceled.


Thanks,
Xing


On Wed, Aug 7, 2019 at 10:12 PM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 8, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 9, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

Re: [opensds-installer] feed backs from Capri installation

Xing Yang
 

Hi Itoh-san,

As discussed on slack, please open issues in the opensds-installer repo so that it is easier to track them.  We will address them.

Thank you so much for your feedback!

Xing


From: Opensds-tech-discuss@... <Opensds-tech-discuss@...> on behalf of Masanori Itoh via Lists.Opensds.Io <masanori.itoh=gmail.com@...>
Sent: Thursday, August 8, 2019 9:06 PM
To: opensds-tech-discuss@... <opensds-tech-discuss@...>
Cc: Opensds-tech-discuss@... <Opensds-tech-discuss@...>
Subject: [Opensds-tech-discuss] [opensds-installer] feed backs from Capri installation
 
Hello,
Recently I started working on OpenSDS Capri using opensds-installer,
and I got some suggestions/issue like below.

As this is a summary report, it's not my intention discussing details
here on this mail thread. I want to know if it's ok splitting them and
creating issues on the github opensds(-installer) project?

I have some workarounds/fixes on them and can work on them together.

Thanks in advance,
Masanori


1) [suggestion] opensds-installer
  - Switching to the current Ubuntu LTS (18.04(bionic)):

    Let's update base OS to Ubuntu 18.04(bionic) in the documentation below.
      https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopensds%2Fopensds%2Fwiki%2FOpenSDS-Cluster-Installation-through-Ansible&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126799474&amp;sdata=9VvoE9%2FNosq0bLoIE4732XiOG833014PdsmYh%2FNXe0c%3D&amp;reserved=0
    - This is Uubuntu 16.04 based and it's already out-of-date.

2) [suggestion] opensds-installer
  - Switching to the latest ansible release:

    Lets' use the latest stable ansible (not bionic included one) and write
    down installation procedure as described below.
    Now the latest stable is 2.8.3 (Ubuntu bionic bundles 2.5.1).
      https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.ansible.com%2Fansible%2Flatest%2Finstallation_guide%2Fintro_installation.html%23latest-releases-via-apt-ubuntu&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126799474&amp;sdata=8NsU%2B5r1VlXsGAAlvdUQTG6bRjLzTYlkxnRPX9EvkBI%3D&amp;reserved=0
    - In this sense, the cluster installation document in 1) refers
      'install_ansible.sh' but it does not exist any more, I think.
      I checked master and stable/capri branches.

3) [suggestion] opensds-installer
  - Switching to standard docker-ce:

    Let's use standard docker-ce through docker apt line because
    opensds must be docker version agnostic, and it's better to use stable
    and standard ones.
    - The below procedure gives us docker-ce 19.03.1 now.
       https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.docker.com%2Finstall%2Flinux%2Fdocker-ce%2Fubuntu%2F&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126799474&amp;sdata=MbXZpyVopUzKTBU8ObSbOm44GLLhfBpIaF2xNI5%2FF%2FY%3D&amp;reserved=0
    - Also, let's use the standard docker-compose in Ubuntu bionic repository.
      Now it gives us docker-compose 1.17.1, it's a bit older than the
      documentation above. But it looks working (at least for me).
    - If folks don't like this, the latest stable of docker-compose looks
      like 1.24.1 now, and we can use this.
        https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdocker%2Fcompose%2Freleases%2Flatest&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126799474&amp;sdata=2xiMQJRTg%2BYWmSTDeDugbx0kF7ihfaw%2B%2BZlycv9880Q%3D&amp;reserved=0

4) [suggestion] opensds-installer
  - Adding docker installation role:

    Let's add a docker installation role (including docker-compose).

5) [suggestion(issue?)] opensds-installer
  - Removing docker-py installation tasks:

    If opensds components themselves do not rely on docker-py directly
    (I believe not), let's rely on docker-ce / docker-compose installation
    dependencies.
    Now 'pip install docker-py' installs v1.10.6 and this breaks
    the above (at least Ubuntu bionic bundled version) docker-compose behavior.

6) [suggestion] opensds-installer
  - Using golang backport ppa:

    Let's use backport ppa for golang installation.
    https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgolang%2Fgo%2Fwiki%2FUbuntu&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126799474&amp;sdata=84BvQJKmF2DpSQDWZCSFhO%2Br9ZPCqw1w3vhwj0GnvNI%3D&amp;reserved=0
    The above ppa (golang-go) now gives golang 1.12.7 by default.
    Also, we can add golang installation task to the playbook.

7) [issue]
  - keystone container installation returns wrong endpoint addresses:

    In case of keystone container installation, I got '0.0.0.0' for
    all the endpoint addresses even if I set HOST_IP (not only 'host_ip'
    in 'opensds-installer/ansible/group_vars/common.yml') like below.
    This does not make sense, and I think ansible/scripts/keystone.sh needs
    additional endpoint configuration (at least).


```
[ubuntu@ubuntu213 ~(opensds)]$ set | grep OS_
OS_AUTH_URL=http://192.168.1.214/identity
OS_IDENTITY_API_VERSION=3
OS_PASSWORD=opensds@123
OS_PROJECT_DOMAIN_NAME=Default
OS_PROJECT_NAME=service
OS_USERNAME=opensds
OS_USER_DOMAIN_NAME=Default
[ubuntu@ubuntu213 ~(opensds)]$ openstack endpoint list  -c "Service
Type" -c "URL"
+---------------+-------------------------------------------+
| Service Type  | URL                                       |
+---------------+-------------------------------------------+
| identity      | https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2F0.0.0.0%2Fidentity&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126799474&amp;sdata=ZdpycAPWJmMC6mcnkfmqn3LmuPw5xgE4ThDJr450IZg%3D&amp;reserved=0                   |
| identity      | https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2F0.0.0.0%2Fidentity&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126799474&amp;sdata=ZdpycAPWJmMC6mcnkfmqn3LmuPw5xgE4ThDJr450IZg%3D&amp;reserved=0                   |
| opensdsv1beta | https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2F0.0.0.0%3A50040%2Fv1beta%2F%25(tenant_id)s&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126809468&amp;sdata=1uV5zubawYJOU7XKBTSg%2ByE%2B3lJEx5Wt4Y%2FXuRygDS8%3D&amp;reserved=0 |
| opensdsv1beta | https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2F0.0.0.0%3A50040%2Fv1beta%2F%25(tenant_id)s&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126809468&amp;sdata=1uV5zubawYJOU7XKBTSg%2ByE%2B3lJEx5Wt4Y%2FXuRygDS8%3D&amp;reserved=0 |
| opensdsv1beta | https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2F0.0.0.0%3A50040%2Fv1beta%2F%25(tenant_id)s&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126809468&amp;sdata=1uV5zubawYJOU7XKBTSg%2ByE%2B3lJEx5Wt4Y%2FXuRygDS8%3D&amp;reserved=0 |
| multicloudv1  | https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2F0.0.0.0%3A8089%2Fv1%2F%25(tenant_id)s&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126809468&amp;sdata=yRsEzyyirqKOppq1eW4sHQDqwXbFJJ8oRJ5vnHAyQus%3D&amp;reserved=0      |
| multicloudv1  | https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2F0.0.0.0%3A8089%2Fv1%2F%25(tenant_id)s&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126809468&amp;sdata=yRsEzyyirqKOppq1eW4sHQDqwXbFJJ8oRJ5vnHAyQus%3D&amp;reserved=0      |
| multicloudv1  | https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2F0.0.0.0%3A8089%2Fv1%2F%25(tenant_id)s&amp;data=02%7C01%7Cxing.yang%40futurewei.com%7C31ee7558cef14455f4af08d71c65dacf%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637009096126809468&amp;sdata=yRsEzyyirqKOppq1eW4sHQDqwXbFJJ8oRJ5vnHAyQus%3D&amp;reserved=0      |
+---------------+-------------------------------------------+
```

8) [suggestion] hotpot? / opensds-installer
  - Using opensds endpoint given by keystone:

    In case of keystone authentication, it's better relying only on
    keystone endpoint (OS_AUTH_URL) and getting OPENSDS_ENDPOINT from
    keystone responses. For example, I mean using URLs of 'opensdsv1beta'
    services in 8).

  - BTW, now opensds endpoint resolution is working even with the issue 7)
    because opensds components such as 'osdsctl' uses OPENSDS_ENDPOINT
    directly (ignoring endpoints give by keystone responses).

9) [suggestion] opensds-installer
  - Adding tags in site.yml:

    Let's make the top level site.yml more modular by adding 'tags' per tasks
    so that we can controll the installation tasks execution more fine grained
    way. (like telemetry_tag/orchestration_tag)


10) [suggestion/issue?]
  - update etcd or making etcd container installation as default

    At least in my ubuntu bionic installation, etcd native installation was
    unstable. Actually etcd repeatedly crashed, and I simply updated etcd
    to the latest (v3.3.13) from v3.2.25. Also, container installation was
    stable (because 'ansible/roles/osdsdb/scenarios/container.yml' pulls
    the latest (but version was v3.3.8)).

11) [suggestion] hotpot / opensds-installer
  - Improving keystone env variables handling:

    Now, osdsctl requires OS_TENANT_NAME and OS_USER_DOMAIN_ID
    in case of keystone authentication. But, nowadays is's better using
    OS_PROJECT_NAME instead of OS_TENANT_NAME. Also, OS_USER_DOMAIN_NAME
    can be also used instead of OS_USER_DOMAIN_ID.


12) [suggestion] opensds-installer
  - Making various components (including etcd, keystone related ones etc.)
    installable on seprated nodes.

etc.



[opensds-installer] feed backs from Capri installation

Masanori Itoh
 

Hello,
Recently I started working on OpenSDS Capri using opensds-installer,
and I got some suggestions/issue like below.

As this is a summary report, it's not my intention discussing details
here on this mail thread. I want to know if it's ok splitting them and
creating issues on the github opensds(-installer) project?

I have some workarounds/fixes on them and can work on them together.

Thanks in advance,
Masanori


1) [suggestion] opensds-installer
- Switching to the current Ubuntu LTS (18.04(bionic)):

Let's update base OS to Ubuntu 18.04(bionic) in the documentation below.
https://github.com/opensds/opensds/wiki/OpenSDS-Cluster-Installation-through-Ansible
- This is Uubuntu 16.04 based and it's already out-of-date.

2) [suggestion] opensds-installer
- Switching to the latest ansible release:

Lets' use the latest stable ansible (not bionic included one) and write
down installation procedure as described below.
Now the latest stable is 2.8.3 (Ubuntu bionic bundles 2.5.1).
https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#latest-releases-via-apt-ubuntu
- In this sense, the cluster installation document in 1) refers
'install_ansible.sh' but it does not exist any more, I think.
I checked master and stable/capri branches.

3) [suggestion] opensds-installer
- Switching to standard docker-ce:

Let's use standard docker-ce through docker apt line because
opensds must be docker version agnostic, and it's better to use stable
and standard ones.
- The below procedure gives us docker-ce 19.03.1 now.
https://docs.docker.com/install/linux/docker-ce/ubuntu/
- Also, let's use the standard docker-compose in Ubuntu bionic repository.
Now it gives us docker-compose 1.17.1, it's a bit older than the
documentation above. But it looks working (at least for me).
- If folks don't like this, the latest stable of docker-compose looks
like 1.24.1 now, and we can use this.
https://github.com/docker/compose/releases/latest

4) [suggestion] opensds-installer
- Adding docker installation role:

Let's add a docker installation role (including docker-compose).

5) [suggestion(issue?)] opensds-installer
- Removing docker-py installation tasks:

If opensds components themselves do not rely on docker-py directly
(I believe not), let's rely on docker-ce / docker-compose installation
dependencies.
Now 'pip install docker-py' installs v1.10.6 and this breaks
the above (at least Ubuntu bionic bundled version) docker-compose behavior.

6) [suggestion] opensds-installer
- Using golang backport ppa:

Let's use backport ppa for golang installation.
https://github.com/golang/go/wiki/Ubuntu
The above ppa (golang-go) now gives golang 1.12.7 by default.
Also, we can add golang installation task to the playbook.

7) [issue]
- keystone container installation returns wrong endpoint addresses:

In case of keystone container installation, I got '0.0.0.0' for
all the endpoint addresses even if I set HOST_IP (not only 'host_ip'
in 'opensds-installer/ansible/group_vars/common.yml') like below.
This does not make sense, and I think ansible/scripts/keystone.sh needs
additional endpoint configuration (at least).


```
[ubuntu@ubuntu213 ~(opensds)]$ set | grep OS_
OS_AUTH_URL=http://192.168.1.214/identity
OS_IDENTITY_API_VERSION=3
OS_PASSWORD=opensds@123
OS_PROJECT_DOMAIN_NAME=Default
OS_PROJECT_NAME=service
OS_USERNAME=opensds
OS_USER_DOMAIN_NAME=Default
[ubuntu@ubuntu213 ~(opensds)]$ openstack endpoint list -c "Service
Type" -c "URL"
+---------------+-------------------------------------------+
| Service Type | URL |
+---------------+-------------------------------------------+
| identity | http://0.0.0.0/identity |
| identity | http://0.0.0.0/identity |
| opensdsv1beta | http://0.0.0.0:50040/v1beta/%(tenant_id)s |
| opensdsv1beta | http://0.0.0.0:50040/v1beta/%(tenant_id)s |
| opensdsv1beta | http://0.0.0.0:50040/v1beta/%(tenant_id)s |
| multicloudv1 | http://0.0.0.0:8089/v1/%(tenant_id)s |
| multicloudv1 | http://0.0.0.0:8089/v1/%(tenant_id)s |
| multicloudv1 | http://0.0.0.0:8089/v1/%(tenant_id)s |
+---------------+-------------------------------------------+
```

8) [suggestion] hotpot? / opensds-installer
- Using opensds endpoint given by keystone:

In case of keystone authentication, it's better relying only on
keystone endpoint (OS_AUTH_URL) and getting OPENSDS_ENDPOINT from
keystone responses. For example, I mean using URLs of 'opensdsv1beta'
services in 8).

- BTW, now opensds endpoint resolution is working even with the issue 7)
because opensds components such as 'osdsctl' uses OPENSDS_ENDPOINT
directly (ignoring endpoints give by keystone responses).

9) [suggestion] opensds-installer
- Adding tags in site.yml:

Let's make the top level site.yml more modular by adding 'tags' per tasks
so that we can controll the installation tasks execution more fine grained
way. (like telemetry_tag/orchestration_tag)


10) [suggestion/issue?]
- update etcd or making etcd container installation as default

At least in my ubuntu bionic installation, etcd native installation was
unstable. Actually etcd repeatedly crashed, and I simply updated etcd
to the latest (v3.3.13) from v3.2.25. Also, container installation was
stable (because 'ansible/roles/osdsdb/scenarios/container.yml' pulls
the latest (but version was v3.3.8)).

11) [suggestion] hotpot / opensds-installer
- Improving keystone env variables handling:

Now, osdsctl requires OS_TENANT_NAME and OS_USER_DOMAIN_ID
in case of keystone authentication. But, nowadays is's better using
OS_PROJECT_NAME instead of OS_TENANT_NAME. Also, OS_USER_DOMAIN_NAME
can be also used instead of OS_USER_DOMAIN_ID.


12) [suggestion] opensds-installer
- Making various components (including etcd, keystone related ones etc.)
installable on seprated nodes.

etc.

OpenSDS Technical Meeting Reminder

Xing Yang
 

Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: August 8, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or August 9, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing

Re: OpenSDS Technical Meeting Minutes - July 30, 2019

Xing Yang
 

Attached is the slide deck on YIG and multi-cloud integration discussed in today's meeting.

Thanks,
Xing


On Tue, Jul 30, 2019 at 3:17 PM Xing Yang <xingyang105@...> wrote:
Thank you for attending today's technical meeting.  Here are the meeting minutes.

Agenda/Notes:

  • Design reviews

    • [Shufang/China Unicom] Shufang discussed about YIG (Scalable Object Store) integration with multi-cloud data control.

      • YIG provides a S3 gateway that is built to support Ceph clusters, but the approach can be adopted to support other S3 backends.  It supports additional S3 features such as versioning, server-side encryption, etc.

      • Multi-cloud S3 service will be replaced with YIG’s implementation. Other services including Backend Service, Dataflow Service, and Data Mover will remain the same.

      • Database in multi-cloud data control will be changed from MongoDB to TiDB, an open source NewSQL database that is MySQL compatible and features horizontal scalability, strong consistency, and high availability.

      • Redis will be used for caching metadata to improve performance.

    • [Click2Cloud] Click2Cloud talked about 3 enhancements they are proposing in multi-cloud data control. https://drive.google.com/file/d/1VIeLeh8BQ5ZejTHe_ZxnHttgdnKXOC0v/view?usp=sharing

      • Show Progress of object migration process - Proposal looks okay. Will work on it and submit a PR.  Need to make sure it works with the YIG integration changes.

      • Show estimated time to complete the migration - Proposal looks okay. Will work on it and submit a PR.  Need to make sure it works with the YIG integration changes.

      • Cancel or Abort Migration - We are running out of time. Need more time to discuss about this.  Will continue in the next meeting.

  • Proposing to re-schedule Tuesday’s technical meetings in the future to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).


Thanks,
Xing

OpenSDS Technical Meeting Minutes - July 30, 2019

Xing Yang
 

Thank you for attending today's technical meeting.  Here are the meeting minutes.

Agenda/Notes:

  • Design reviews

    • [Shufang/China Unicom] Shufang discussed about YIG (Scalable Object Store) integration with multi-cloud data control.

      • YIG provides a S3 gateway that is built to support Ceph clusters, but the approach can be adopted to support other S3 backends.  It supports additional S3 features such as versioning, server-side encryption, etc.

      • Multi-cloud S3 service will be replaced with YIG’s implementation. Other services including Backend Service, Dataflow Service, and Data Mover will remain the same.

      • Database in multi-cloud data control will be changed from MongoDB to TiDB, an open source NewSQL database that is MySQL compatible and features horizontal scalability, strong consistency, and high availability.

      • Redis will be used for caching metadata to improve performance.

    • [Click2Cloud] Click2Cloud talked about 3 enhancements they are proposing in multi-cloud data control. https://drive.google.com/file/d/1VIeLeh8BQ5ZejTHe_ZxnHttgdnKXOC0v/view?usp=sharing

      • Show Progress of object migration process - Proposal looks okay. Will work on it and submit a PR.  Need to make sure it works with the YIG integration changes.

      • Show estimated time to complete the migration - Proposal looks okay. Will work on it and submit a PR.  Need to make sure it works with the YIG integration changes.

      • Cancel or Abort Migration - We are running out of time. Need more time to discuss about this.  Will continue in the next meeting.

  • Proposing to re-schedule Tuesday’s technical meetings in the future to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).


Thanks,
Xing

Invitation: (No Subject) @ Tue Jul 30, 2019 9am - 10am (EDT) (opensds-tech-discuss@lists.opensds.io)

Xing Yang
 

You have been invited to the following event.

(No Subject)

When
Tue Jul 30, 2019 9am – 10am Eastern Time - New York
Where
https://zoom.us/j/777978108 (map)
Calendar
opensds-tech-discuss@...
Who
xingyang105@... - organizer
opensds-tech-discuss@...
Hi everyone,

This week's OpenSDS Technical Meeting is re-scheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET).

Zoom Link: https://zoom.us/j/777978108

Agenda doc is here:
https://docs.google.com/document/d/1JlxAAOtvZvvf_KhVr8XQa6mUD7lkHOXlxuGruTKEukE/edit#

Thanks,
Xing

Going (opensds-tech-discuss@...)?   Yes - Maybe - No    more options »

Invitation from Google Calendar

You are receiving this courtesy email at the account opensds-tech-discuss@... because you are an attendee of this event.

To stop receiving future updates for this event, decline this event. Alternatively you can sign up for a Google account at https://www.google.com/calendar/ and control your notification settings for your entire calendar.

Forwarding this invitation could allow any recipient to send a response to the organizer and be added to the guest list, or invite others regardless of their own invitation status, or to modify your RSVP. Learn More.

Re: OpenSDS Technical Meeting Reminder

Xing Yang
 

Thank you for attending today's meeting.  Here are the meeting notes.

Agenda/Notes:


Note: Next Tuesday’s meeting will be rescheduled to 1:00pm-2:00pm UTC (6:00am-7:00am PT or 9:00am-10:00am ET). We'll be discussing YIG (Scalable Object Store) integration with multi-cloud data control.

Thanks,
Xing


On Wed, Jul 24, 2019 at 9:45 AM Xing Yang <xingyang105@...> wrote:
Hi everyone,

Just a reminder that we will have the OpenSDS Technical meeting this week.

Time: July 25, 2019 Thursday 6:00pm-7:00pm PT or 9:00pm-10:00pm ET (or July 26, 2019 Friday 1:00am-2:00am UTC)


Zoom Link: https://zoom.us/j/229373941


Meeting agenda is here:

Please add any item you want to discuss in the agenda doc.

Thanks,
Xing