这是在Dashborad界面得到以上报错
排查半天没有任何结果,发现keystone还是报认证错误。controller其他日志正常。
通过shell手动创建实例
[root@controller ~]#openstack server create --flavor m1.tiny --image cirros --nic net-id=3e4bc802-7e7f-49be-a9ae-f6216013b2c6 --availability-zone nova:compute-3:compute-3 test-node3
发行又出现了 No valid host was found. There are not enough hosts available.错误。
[root@controller nova]# cat nova-conductor.log ##日志报错信息还是指向NoValidHost: No valid host was found. There are not enough hosts available.
[root@compute-3 nova]# cat nova-compute.log ##一直在循环报错 Error updating resources for node compute-3: ResourceProviderCreationFailed: Failed to create resource provider compute-3
[root@compute-3 nova]# cat nova-compute.log
2020-12-17 14:49:06.272 2529 ERROR nova.scheduler.client.report [req-98d36e66-25a2-40de-8e25-7d1f489b0588 - - - - -] [req-5353573f-da11-48ff-8472-1e9a471269cd] Failed to create resource provider record in placement API for UUID b1c484e8-58cb-41d5-84d3-1e1f4ffab341. Got 409: {"errors": [{"status": 409, "request_id": "req-5353573f-da11-48ff-8472-1e9a471269cd", "detail": "There was a conflict when trying to complete your request.\n\n Conflicting resource provider name: compute-3.rocky2.yfb0801800.islab.net.cn already exists. ", "title": "Conflict"}]}.
2020-12-17 14:49:06.272 2529 ERROR nova.compute.manager [req-98d36e66-25a2-40de-8e25-7d1f489b0588 - - - - -] Error updating resources for node compute-3: ResourceProviderCreationFailed: Failed to create resource provider compute-3
2020-12-17 14:49:06.272 2529 ERROR nova.compute.manager Traceback (most recent call last):
因为已经做过多次的su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova。判断数据库出问题的可能性不大。怀疑keystone的nova用户权限有问题
检查openstack状态和nova计算节点,表面上看状态一切正常
准备keystone中所有nova相关配置删除重来。再有问题就只能删除nova数据库了
先删除nova用户
删除nova相关服务,endpoint也已经被删除了
创建nova用户和Nova服务和endpoint
#openstack user create --domain default --password $NOVA_PASSWD nova
#openstack role add --project service --user nova admin
#openstack service create --name nova --description "OpenStack Compute" compute
#openstack endpoint create --region RegionOne compute public http://${controllerHost}:8774/v2.1
#openstack endpoint create --region RegionOne compute internal http://${controllerHost}:8774/v2.1
#openstack endpoint create --region RegionOne compute admin http://${controllerHost}:8774/v2.1
完成上述操作后问题依旧
无意间查看 nova-placement-api.log发现虽然状态是INFO但是HTTP的status的代码为409,对比以前的一套实验系统发现status 响应代码应该是200。第一反应想到的是00-nova-placement-api.conf的问题,可以查看后肉眼没发现问题,本着活马当死马医的太多,还是把实验环境的00-nova-placement-api.conf复制了过来。
[root@controller nova]# cd nova-placement-api.log
发现一个奇怪的问题只有202-205四台node节点会报status 409,而201节点不汇报409.
然后手动在201节点上创建实例,compute node报错信息如下。
[root@compute-1 nova]# cat nova-compute.log | grep ERROR
查看compute节点的neutron配置文件没有问题。准备重刷keystone neutron相关配置。
openstack user create --domain default --password $NEUTRON_DB_PASSWD neutron
openstack role add --project service --user neutron admin
openstack service create --name neutron --description "OpenStack Networking" network
openstack endpoint create --region RegionOne network public http://${controllerHost}:9696
openstack endpoint create --region RegionOne network admin http://${controllerHost}:9696
openstack endpoint create --region RegionOne network internal http://${controllerHost}:9696
删除数据库,从头再来。
[root@controller keystone]# su -s /bin/sh -c "keystone-manage db_sync" keystone[root@controller keystone]# keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone
[root@controller keystone]# keystone-manage credential_setup --keystone-user keystone --keystone-group keystone
[root@controller keystone]# keystone-manage bootstrap --bootstrap-password KEYSTONE_PASSWD --bootstrap-admin-url http://controller.rocky2.yfb0801800.islab.net.cn:5000/v3/ --bootstrap-internal-url http://controller.rocky2.yfb0801800.islab.net.cn:5000/v3/ --bootstrap-public-url http://controller.rocky2.yfb0801800.islab.net.cn:5000/v3/ --bootstrap-region-id RegionOne
[root@controller keystone]# source ~/admin-openrc
[root@controller keystone]# openstack domain create --description "An Example Domain" example
[root@controller keystone]# openstack project create --domain default --description "Service Project" service
[root@controller keystone]# openstack project create --domain default --description "Demo Project" demo
[root@controller keystone]# openstack user create --domain default --password $DEMO_PASSWD demo
[root@controller keystone]# openstack role create user
[root@controller keystone]# openstack role add --project demo --user demo user
创建GLANCE相关服务和用户
[root@controller keystone]# openstack user create --domain default --password $GLANCE_PASSWD glance
[root@controller keystone]# openstack role add --project service --user glance admin
[root@controller keystone]# openstack service create --name glance --description "OpenStack Image" image
[root@controller keystone]# openstack endpoint create --region RegionOne image public http://${controllerHost}:9292
[root@controller keystone]# openstack endpoint create --region RegionOne image internal http://${controllerHost}:9292
[root@controller keystone]# openstack endpoint create --region RegionOne image admin http://${controllerHost}:9292
[root@controller keystone]# su -s /bin/sh -c "glance-manage db_sync" glance
[root@controller keystone]# systemctl restart openstack-glance-api.service openstack-glance-registry.service
su -s /bin/sh -c "glance-manage db_sync" glance
创建NOVA相关服务和用户
[root@controller ~]# openstack user create --domain default --password NOVA_PASSWD nova
[root@controller ~]# openstack role add --project service --user nova admin
[root@controller ~]# openstack service create --name nova --description "OpenStack Compute" compute
[root@controller ~]# openstack endpoint create --region RegionOne compute public http://${controllerHost}:8774/v2.1
[root@controller ~]# openstack endpoint create --region RegionOne compute internal http://${controllerHost}:8774/v2.1
[root@controller ~]# openstack endpoint create --region RegionOne compute admin http://${controllerHost}:8774/v2.1
创建PLACEMENT相关服务和用户
[root@controller ~]# openstack user create --domain default --password $PLACEMENT_PASSWD placement
[root@controller ~]# openstack role add --project service --user placement admin
[root@controller ~]# openstack service create --name placement --description "Placement API" placement
[root@controller ~]# openstack endpoint create --region RegionOne placement public http://${controllerHost}:8778
[root@controller ~]# openstack endpoint create --region RegionOne placement internal http://${controllerHost}:8778
[root@controller ~]# openstack endpoint create --region RegionOne placement admin http://${controllerHost}:8778
su -s /bin/sh -c "nova-manage api_db sync" nova
su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova
su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova
su -s /bin/sh -c "nova-manage db sync" nova
su -s /bin/sh -c "nova-manage cell_v2 list_cells" nova
创建NEUTRON相关服务和用户
[root@controller ~]# openstack user create --domain default --password $NEUTRON_PASSWD neutron
[root@controller ~]# openstack role add --project service --user neutron admin
[root@controller ~]# openstack service create --name neutron --description "OpenStack Networking" network
[root@controller ~]# openstack endpoint create --region RegionOne network public http://${controllerHost}:9696
[root@controller ~]# openstack endpoint create --region RegionOne network admin http://${controllerHost}:9696
[root@controller ~]# openstack endpoint create --region RegionOne network internal http://${controllerHost}:9696
su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron
创建CINDER相关服务和用户
openstack user create --domain default --password ${CINDER_DB_PASSWD} cinder
openstack role add --project service --user cinder admin
openstack service create --name cinderv2 --description "OpenStack Block Storage" volumev2
openstack endpoint create --region RegionOne volumev2 public http://${controllerHost}:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne volumev2 internal http://${controllerHost}:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne volumev2 admin http://${controllerHost}:8776/v2/%\(project_id\)s
openstack service create --name cinderv3 --description "OpenStack Block Storage" volumev3
openstack endpoint create --region RegionOne volumev3 public http://${controllerHost}:8776/v3/%\(project_id\)s
openstack endpoint create --region RegionOne volumev3 internal http://${controllerHost}:8776/v3/%\(project_id\)s
openstack endpoint create --region RegionOne volumev3 admin http://${controllerHost}:8776/v3/%\(project_id\)s
su -s /bin/sh -c "cinder-manage db sync" cinder
systemctl restart openstack-cinder-api.service openstack-cinder-scheduler.service
systemctl status openstack-cinder-api.service openstack-cinder-scheduler.service
创建实例类型
source ~/admin-openrc
openstack flavor create --vcpus 1 --ram 512 --disk 1 m1.tiny
openstack flavor create --vcpus 2 --ram 4096 --disk 40 m1.medium
openstack flavor create --vcpus 4 --ram 8192 --disk 80 m1.large
openstack flavor create --vcpus 1 --ram 2048 --disk 20 m1.small
openstack flavor create --vcpus 8 --ram 16384 --disk 160 m1.xlarge
openstack flavor create --vcpus 8 --ram 8192 --disk 80 m2.large
openstack flavor create --vcpus 8 --ram 16384 --disk 80 m3.large
openstack flavor create --vcpus 16 --ram 16384 --disk 80 Xcm.large
重启相关服务后故障排除。
这个问题花了我将近一周的时间才得以解决,事后分析,原因可能出在Keystone的数据库里。虽然nova、neutron服务和检测一直正常,但能在keystone的鉴权过程中出现数据不同步导致的。
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)