欢迎您的访问
专注于Java技术系列文章的Java技术分享网站

Sharding-JDBC(三)3.1.0版本实践

目录

  一、Sharding-JDBC依赖

  二、代码实践

  三、源码分析

在上一篇博文中,介绍了Sharding-JDBC的分片策略、分片键和分片算法的基本概念,以及2.0.3版本可以支持和无法支持的使用场景。

可以支持的场景:支持对SQL语句中的=、IN和BETWEEN AND的分片操作,但前提是分片键必须存在于SQL和数据表结构中。

无法支持的场景:分片键不存在于SQL和数据表结构中,即基于暗示(Hint)的数据分片操作([2.0.3版本的问题][2.0.3])。

无可厚非,缺少了Hint分片策略的支持,Sharding-JDBC 2.0.3版本的使用场景就非常受限了,但值得庆幸的是,此问题在3.x版本进行了修复(这里可以有掌声!),接下来的代码皆基于3.1.0版本。

一、Sharding-JDBC依赖

<!-- sharding-jdbc-core -->
<dependency>
    <groupId>io.shardingsphere</groupId>
    <artifactId>sharding-jdbc-core</artifactId>
    <version>3.1.0</version>
</dependency>
<!-- sharding-jdbc-spring-namespace -->
<dependency>
    <groupId>io.shardingsphere</groupId>
    <artifactId>sharding-jdbc-spring-namespace</artifactId>
    <version>3.1.0</version>
</dependency>

和2、0.3版本相比,依赖的名称有所改变,不要搞错了哦。

二、代码实践

业务背景就不再介绍了,不了解可移步至Sharding-JDBC(二)2.0.3版本实践

如下代码配置了标准分片策略中的精确分片算法PreciseShardingAlgorithm和Hint分片算法HintShardingAlgorithm。

XML配置:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:sharding="http://shardingsphere.io/schema/shardingsphere/sharding"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/tx
                           http://www.springframework.org/schema/tx/spring-tx.xsd
                           http://shardingsphere.io/schema/shardingsphere/sharding
                           http://shardingsphere.io/schema/shardingsphere/sharding/sharding.xsd">

    <!-- 标准分片策略 -->
    <sharding:standard-strategy id="settlementTableShardingStandardStrategy" sharding-column="pay_serial_number"
                                precise-algorithm-ref="preciseTableShardingAlgorithm"/>

    <!-- 基于暗示(Hint)的分片策略 -->
    <sharding:hint-strategy id="settlementHintTableShardingStrategy" algorithm-ref="hintTableShardingAlgorithm"/>
    <sharding:hint-strategy id="settlementHintDatabaseShardingStrategy" algorithm-ref="hintDatabaseShardingAlgorithm"/>

    <sharding:data-source id="shardingDataSource">
        <sharding:sharding-rule data-source-names="dataSource">
            <sharding:table-rules>
                <sharding:table-rule logic-table="settlement"
                                     table-strategy-ref="settlementTableShardingStandardStrategy"/>
                <!-- logic-table参数的大小写必须和SettlementMapper.xml中selectByExample方法的表名大小一致!!! -->
                <!-- logic-table必须和org.cellphone.finance.repo.SettlementRepository.querySettlements中的logicTable及SQL中的表名一致,否则无法找到分片策略 -->
                <!-- 逻辑表名,不需要和真实表名一致 -->
                <sharding:table-rule logic-table="settlement_hint"
                                     database-strategy-ref="settlementHintDatabaseShardingStrategy"
                                     table-strategy-ref="settlementHintTableShardingStrategy"/>
            </sharding:table-rules>
        </sharding:sharding-rule>
        <sharding:props>
            <prop key="sql.show">true</prop>
        </sharding:props>
    </sharding:data-source>

    <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
        <property name="dataSource" ref="shardingDataSource"/>
    </bean>
    <tx:annotation-driven/>

</beans>

精确分片算法:

package org.cellphone.finance.repo.sharding;

import com.google.common.collect.Lists;
import io.shardingsphere.api.algorithm.sharding.PreciseShardingValue;
import io.shardingsphere.api.algorithm.sharding.standard.PreciseShardingAlgorithm;
import org.apache.commons.collections.CollectionUtils;
import org.cellphone.common.constant.CommonConst;
import org.springframework.stereotype.Component;

import java.util.Collection;

/**
 * 精确分片算法,属于标准分片算法,用于处理=和IN的分片
 * <p>
 * 使用精确分片算法的前提:分片字段必须存在与SQL中、数据库表结构中,否则无法使用精确分片算法
 * <p>
 * 此分片算法应用于SETTLEMENT数据表,这里是按天分表
 * <p>
 * 特别说明:Sharding Jdbc版本:3.1.0
 * <p>
 * Created by on 2018/4/9.
 */
@Component("preciseTableShardingAlgorithm")
public class PreciseTableShardingAlgorithm implements PreciseShardingAlgorithm<String> {

    /**
     * 精确分片算法
     *
     * @param availableTargetNames 目标数据源名称或数据表名称,注意:是逻辑数据源名或逻辑数据表名,来自SQL
     * @param shardingValue        分片值,来自SQL中分片字段对应的值
     * @return 真实数据源名称或数据表名称
     */
    @Override
    public String doSharding(final Collection<String> availableTargetNames, final PreciseShardingValue<String> shardingValue) {
        // 默认数据表名称,有可能数据库中不存在这张表
        String tableName = "settlement";

        // 逻辑表名为空,返回默认表名
        if (CollectionUtils.isEmpty(availableTargetNames))
            return tableName;

        // availableTargetNames来自SQL,只有一个元素
        tableName = Lists.newArrayList(availableTargetNames).get(0);

        String paySerialNumber = shardingValue.getValue();
        String suffix = paySerialNumber.substring(5, 13);
        return tableName + CommonConst.UNDERLINE + suffix;
    }
}

Hint数据源分片算法:

package org.cellphone.finance.repo.sharding;

import io.shardingsphere.api.algorithm.sharding.ShardingValue;
import io.shardingsphere.api.algorithm.sharding.hint.HintShardingAlgorithm;
import org.springframework.stereotype.Component;

import java.util.Collection;

/**
 * Sharding Jdbc基于暗示(Hint)的数据分片算法
 *
 * 使用Sharding Jdbc 3.x版本时,此数据源分片算法这个一定要有!!!
 * 否则无法正常使用org.cellphone.finance.repo.sharding.HintTableShardingAlgorithm算法
 * <p>
 * Created by on 2019/4/25.
 */
@Component("hintDatabaseShardingAlgorithm")
public class HintDatabaseShardingAlgorithm implements HintShardingAlgorithm {

    @Override
    public Collection<String> doSharding(Collection<String> availableTargetNames, ShardingValue shardingValue) {
        return availableTargetNames;
    }
}

Hint数据表分片算法:

package org.cellphone.finance.repo.sharding;

import com.google.common.collect.Lists;
import io.shardingsphere.api.algorithm.sharding.ListShardingValue;
import io.shardingsphere.api.algorithm.sharding.ShardingValue;
import io.shardingsphere.api.algorithm.sharding.hint.HintShardingAlgorithm;
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.time.DateFormatUtils;
import org.apache.commons.lang3.time.DateUtils;
import org.cellphone.common.constant.CommonConst;
import org.cellphone.common.constant.DateConst;
import org.springframework.stereotype.Component;

import java.text.ParseException;
import java.util.*;

/**
 * Sharding Jdbc基于暗示(Hint)的数据分片算法
 * 版本:Sharding Jdbc 3.1.0
 *
 * 官方介绍(2.x版本):http://shardingsphere.apache.org/document/legacy/2.x/cn/02-guide/hint-sharding-value/
 * 官方介绍(4.x版本):https://shardingsphere.apache.org/document/current/cn/manual/sharding-jdbc/usage/hint/
 * <p>
 * <p>
 * <p>
 * <p>
 * 使用此算法的背景如下:
 * 1. SETTLEMENT(支付表)是按时间维度进行分表,该时间取自PAY_SERIAL_NUMBER中的时间数据,非表中的时间字段;
 * <p>
 * 2. 使用用户手机号此类条件查询数据时,请求参数除传入业务参数外,
 *      还需传入时间段(即分片字段,例:startTime和endTime)以确定分表范围;
 *      但是!!!startTime和endTime(即分片字段)不存在SQL中、数据库表结构中,而存在于外部业务逻辑
 * <p>
 * <p>
 * <p>
 * 因此,第2点导致无法直接使用Sharding Jdbc的PreciseShardingAlgorithm(精确分片算法)或RangeShardingAlgorithm(范围分片算法),
 * 只能使用HintShardingAlgorithm(基于暗示的数据分片算法),该算法的使用场景如下:
 * 1. 分片字段不存在SQL中、数据库表结构中,而存在于外部业务逻辑;
 * <p>
 * 2. 强制在主库进行某些数据操作。
 * <p>
 * <p>
 * Created by on 2019/4/25.
 */
@Component("hintTableShardingAlgorithm")
public class HintTableShardingAlgorithm implements HintShardingAlgorithm {

    /**
     * 分片算法
     *
     * @param availableTargetNames 逻辑数据库名称或逻辑数据表名称
     * @param shardingValue        用来确定分表的参数
     * @return 实际数据表名称列表,SQL实际操作的数据表
     */
    @Override
    public Collection<String> doSharding(Collection<String> availableTargetNames, ShardingValue shardingValue) {

        String realTableName = StringUtils.EMPTY;
        for (String each : availableTargetNames) {
            if (StringUtils.isNotBlank(each)) {
                // 基于hint的逻辑表名:settlement_hint
                realTableName = each.replace("_hint", StringUtils.EMPTY);
                break;
            }
        }

        List<String> tables = new ArrayList<>();

        ListShardingValue<String> listShardingValue = (ListShardingValue<String>) shardingValue;
        List<String> list = Lists.newArrayList(listShardingValue.getValues());

        // 缺少确定分表的参数,无法确定具体分表,直接返回真实表名称
        if (CollectionUtils.isEmpty(list)) {
            tables.add(realTableName);
            return tables;
        }

        // 拆分分表参数,此参数值来自:com.fcbox.manage.core.repo.FcBoxPostRepository.queryFcBoxPosts()
        String[] queryTime = list.get(0).split(CommonConst.UNDERLINE);
        Date startTime, endTime;
        try {
            startTime = DateUtils.parseDate(queryTime[0], DateConst.DATE_FORMAT_NORMAL);
            endTime   = DateUtils.parseDate(queryTime[1], DateConst.DATE_FORMAT_NORMAL);
        } catch (ParseException e) {
            // 分表参数解析错误,无法确定具体分表,直接返回真实表名称
            tables.add(realTableName);
            return tables;
        }

        Calendar calendar = Calendar.getInstance();
        // 组织startTime和endTime时段范围内的分表
        while (startTime.getTime() <= endTime.getTime()) {
            tables.add(realTableName + CommonConst.UNDERLINE + DateFormatUtils.format(startTime, DateConst.DATE_FORMAT_YYYY_MM_DD));
            calendar.setTime(startTime);
            calendar.add(Calendar.DATE, 1);
            startTime = calendar.getTime();
        }

        return tables;
    }
}

与Hint分片算法对应的Java查询方法 settlementMapper.selectByExample(example):

    public List<Settlement> querySettlements(SettlementExample example, String startTime, String endTime) {
        // 组织查询时间,传入org.cellphone.finance.repo.sharding.HintTableShardingAlgorithm分片算法中以确认具体分表
        String queryTime = startTime + CommonConst.UNDERLINE + endTime;

        // 获取HintManager
        HintManager hintManager = HintManager.getInstance();
        /*
         * 添加数据源分片键值,使用Sharding Jdbc 3.x版本一定要添加数据源分片键值,否则无法使用HintTableShardingAlgorithm分片算法
         * 若无分库,addDatabaseShardingValue方法的value字段随意填充
         * 若有分库,addDatabaseShardingValue方法的value字段填充实际参数值
         */
        hintManager.addDatabaseShardingValue("settlement_hint", StringUtils.EMPTY);
        // 添加数据表分片键值
        hintManager.addTableShardingValue("settlement_hint", queryTime);
        List<Settlement> settlements = settlementMapper.selectByExample(example);
        // 清除分片键值
        hintManager.close();
        return settlements;
    }

以及该查询方法对应的SQL语句:

select * from settlement_hint t where t.pay_serial_number = ?

单元测试代码:

@Test
public void test003QuerySettlements() throws ParseException {
    String startTime = "2018-04-03 00:00:00", endTime = "2018-04-05 00:00:00";

    SettlementExample example = new SettlementExample();
    SettlementExample.Criteria criteria = example.createCriteria();
    criteria.andPaySerialNumberEqualTo(paySerialNumber);

    List<Settlement> settlements = repository.querySettlements(example, startTime, endTime);

    Assert.assertEquals("136********", settlements.get(0).getUserMobile());
}

三、源码分析

和2、0.3版本相比,3.1.0版本的路由入口变成了 io.shardingsphere.core.routing.type.standard.StandardRoutingEngine#route() ,但基本上区别不大,仅仅是多了一步需要确定路由的真实数据源,尽管数据源只有一个,也需要显式配置数据源路由算法。代码中标注了注释的部分都是路由代码比较核心的部分。

/*
 * Copyright 2016-2018 shardingsphere.io.
 * <p>
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 * </p>
 */

package io.shardingsphere.core.routing.type.standard;

import com.google.common.base.Optional;
import com.google.common.base.Preconditions;
import io.shardingsphere.api.algorithm.sharding.ShardingValue;
import io.shardingsphere.core.hint.HintManagerHolder;
import io.shardingsphere.core.optimizer.condition.ShardingCondition;
import io.shardingsphere.core.optimizer.condition.ShardingConditions;
import io.shardingsphere.core.optimizer.insert.InsertShardingCondition;
import io.shardingsphere.core.routing.strategy.ShardingStrategy;
import io.shardingsphere.core.routing.strategy.hint.HintShardingStrategy;
import io.shardingsphere.core.routing.type.RoutingEngine;
import io.shardingsphere.core.routing.type.RoutingResult;
import io.shardingsphere.core.routing.type.RoutingTable;
import io.shardingsphere.core.routing.type.TableUnit;
import io.shardingsphere.core.rule.BindingTableRule;
import io.shardingsphere.core.rule.DataNode;
import io.shardingsphere.core.rule.ShardingRule;
import io.shardingsphere.core.rule.TableRule;
import lombok.RequiredArgsConstructor;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.LinkedHashSet;
import java.util.LinkedList;
import java.util.List;

/**
 * Standard routing engine.
 *
 * @author zhangliang
 * @author maxiaoguang
 * @author panjuan
 */
@RequiredArgsConstructor
public final class StandardRoutingEngine implements RoutingEngine {

    private final ShardingRule shardingRule;

    private final String logicTableName;

    private final ShardingConditions shardingConditions;

    /**
     * 相比2.0.3版本,精简成了1行代码
     *
     * @return 路由结果
     */
    @Override
    public RoutingResult route() {
        return generateRoutingResult(getDataNodes(shardingRule.getTableRuleByLogicTableName(logicTableName)));
    }

    private RoutingResult generateRoutingResult(final Collection<DataNode> routedDataNodes) {
        RoutingResult result = new RoutingResult();
        for (DataNode each : routedDataNodes) {
            TableUnit tableUnit = new TableUnit(each.getDataSourceName());
            tableUnit.getRoutingTables().add(new RoutingTable(logicTableName, each.getTableName()));
            result.getTableUnits().getTableUnits().add(tableUnit);
        }
        return result;
    }

    /**
     * 获取数据节点,即真实表
     *
     * @param tableRule 从XML读取的table rule
     * @return 数据节点列表
     */
    private Collection<DataNode> getDataNodes(final TableRule tableRule) {
        // 判断是否是通过Hint分片策略进行路由
        if (isRoutingByHint(tableRule)) {
            return routeByHint(tableRule);
        }
        if (isRoutingByShardingConditions(tableRule)) {
            return routeByShardingConditions(tableRule);
        }
        return routeByMixedConditions(tableRule);
    }

    /**
     * 判断是否是通过Hint分片策略进行路由
     * <p>
     * 数据源分片策略和数据表分片策略都必须是HintShardingStrategy,这意味者必须显式配置数据源Hint分片策略和数据表Hint分片策略
     *
     * @param tableRule 从XML读取的table rule
     * @return
     */
    private boolean isRoutingByHint(final TableRule tableRule) {
        return shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy && shardingRule.getTableShardingStrategy(tableRule) instanceof HintShardingStrategy;
    }

    /**
     * 通过Hint分片策略进行路由
     *
     * @param tableRule
     * @return
     */
    private Collection<DataNode> routeByHint(final TableRule tableRule) {
        return route(tableRule, getDatabaseShardingValuesFromHint(), getTableShardingValuesFromHint());
    }

    private boolean isRoutingByShardingConditions(final TableRule tableRule) {
        return !(shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy || shardingRule.getTableShardingStrategy(tableRule) instanceof HintShardingStrategy);
    }

    private Collection<DataNode> routeByShardingConditions(final TableRule tableRule) {
        return shardingConditions.getShardingConditions().isEmpty() ? route(tableRule, Collections.<ShardingValue>emptyList(), Collections.<ShardingValue>emptyList())
                : routeByShardingConditionsWithCondition(tableRule);
    }

    private Collection<DataNode> routeByShardingConditionsWithCondition(final TableRule tableRule) {
        Collection<DataNode> result = new LinkedList<>();
        for (ShardingCondition each : shardingConditions.getShardingConditions()) {
            Collection<DataNode> dataNodes = route(tableRule, getShardingValuesFromShardingConditions(shardingRule.getDatabaseShardingStrategy(tableRule).getShardingColumns(), each),
                    getShardingValuesFromShardingConditions(shardingRule.getTableShardingStrategy(tableRule).getShardingColumns(), each));
            reviseShardingConditions(each, dataNodes);
            result.addAll(dataNodes);
        }
        return result;
    }

    private Collection<DataNode> routeByMixedConditions(final TableRule tableRule) {
        return shardingConditions.getShardingConditions().isEmpty() ? routeByMixedConditionsWithHint(tableRule) : routeByMixedConditionsWithCondition(tableRule);
    }

    private Collection<DataNode> routeByMixedConditionsWithCondition(final TableRule tableRule) {
        Collection<DataNode> result = new LinkedList<>();
        for (ShardingCondition each : shardingConditions.getShardingConditions()) {
            Collection<DataNode> dataNodes = route(tableRule, getDatabaseShardingValues(tableRule, each), getTableShardingValues(tableRule, each));
            reviseShardingConditions(each, dataNodes);
            result.addAll(dataNodes);
        }
        return result;
    }

    private Collection<DataNode> routeByMixedConditionsWithHint(final TableRule tableRule) {
        if (shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy) {
            return route(tableRule, getDatabaseShardingValuesFromHint(), Collections.<ShardingValue>emptyList());
        }
        return route(tableRule, Collections.<ShardingValue>emptyList(), getTableShardingValuesFromHint());
    }

    private List<ShardingValue> getDatabaseShardingValues(final TableRule tableRule, final ShardingCondition shardingCondition) {
        ShardingStrategy dataBaseShardingStrategy = shardingRule.getDatabaseShardingStrategy(tableRule);
        return isGettingShardingValuesFromHint(dataBaseShardingStrategy)
                ? getDatabaseShardingValuesFromHint() : getShardingValuesFromShardingConditions(dataBaseShardingStrategy.getShardingColumns(), shardingCondition);
    }

    private List<ShardingValue> getTableShardingValues(final TableRule tableRule, final ShardingCondition shardingCondition) {
        ShardingStrategy tableShardingStrategy = shardingRule.getTableShardingStrategy(tableRule);
        return isGettingShardingValuesFromHint(tableShardingStrategy)
                ? getTableShardingValuesFromHint() : getShardingValuesFromShardingConditions(tableShardingStrategy.getShardingColumns(), shardingCondition);
    }

    private boolean isGettingShardingValuesFromHint(final ShardingStrategy shardingStrategy) {
        return shardingStrategy instanceof HintShardingStrategy;
    }

    /**
     * 从HintManagerHolder中获取数据源分片值
     *
     * @return 数据源分片值列表
     */
    private List<ShardingValue> getDatabaseShardingValuesFromHint() {
        // getDatabaseShardingValue方法实现有点恶心,不兼容大小写...
        Optional<ShardingValue> shardingValueOptional = HintManagerHolder.getDatabaseShardingValue(logicTableName);
        return shardingValueOptional.isPresent() ? Collections.singletonList(shardingValueOptional.get()) : Collections.<ShardingValue>emptyList();
    }

    private List<ShardingValue> getTableShardingValuesFromHint() {
        // getTableShardingValue方法实现有点恶心,不兼容大小写...
        Optional<ShardingValue> shardingValueOptional = HintManagerHolder.getTableShardingValue(logicTableName);
        return shardingValueOptional.isPresent() ? Collections.singletonList(shardingValueOptional.get()) : Collections.<ShardingValue>emptyList();
    }

    private List<ShardingValue> getShardingValuesFromShardingConditions(final Collection<String> shardingColumns, final ShardingCondition shardingCondition) {
        List<ShardingValue> result = new ArrayList<>(shardingColumns.size());
        for (ShardingValue each : shardingCondition.getShardingValues()) {
            Optional<BindingTableRule> bindingTableRule = shardingRule.findBindingTableRule(logicTableName);
            if ((logicTableName.equals(each.getLogicTableName()) || bindingTableRule.isPresent() && bindingTableRule.get().hasLogicTable(logicTableName))
                    && shardingColumns.contains(each.getColumnName())) {
                result.add(each);
            }
        }
        return result;
    }

    /**
     * 路由,获取真实表列表
     *
     * @param tableRule              从XML读取的table rule
     * @param databaseShardingValues 数据源分片值
     * @param tableShardingValues    数据表分片值
     * @return 真实表列表
     */
    private Collection<DataNode> route(final TableRule tableRule, final List<ShardingValue> databaseShardingValues, final List<ShardingValue> tableShardingValues) {
        Collection<String> routedDataSources = routeDataSources(tableRule, databaseShardingValues);
        Collection<DataNode> result = new LinkedList<>();
        for (String each : routedDataSources) {
            result.addAll(routeTables(tableRule, each, tableShardingValues));
        }
        return result;
    }

    /**
     * 路由到真实数据源
     *
     * @param tableRule              从XML读取的table rule
     * @param databaseShardingValues 数据源分片值
     * @return 真实数据源列表
     */
    private Collection<String> routeDataSources(final TableRule tableRule, final List<ShardingValue> databaseShardingValues) {
        Collection<String> availableTargetDatabases = tableRule.getActualDatasourceNames();
        if (databaseShardingValues.isEmpty()) {
            return availableTargetDatabases;
        }
        Collection<String> result = new LinkedHashSet<>(shardingRule.getDatabaseShardingStrategy(tableRule).doSharding(availableTargetDatabases, databaseShardingValues));
        Preconditions.checkState(!result.isEmpty(), "no database route info");
        return result;
    }

    /**
     * 路由到真实数据表,和2.0.3版本没啥区别
     *
     * @param tableRule           从XML读取的table rule
     * @param routedDataSource    已确认好的数据源
     * @param tableShardingValues 数据表分片值
     * @return 真实表列表
     */
    private Collection<DataNode> routeTables(final TableRule tableRule, final String routedDataSource, final List<ShardingValue> tableShardingValues) {
        Collection<String> availableTargetTables = tableRule.getActualTableNames(routedDataSource);
        Collection<String> routedTables = new LinkedHashSet<>(tableShardingValues.isEmpty() ? availableTargetTables
                : shardingRule.getTableShardingStrategy(tableRule).doSharding(availableTargetTables, tableShardingValues));
        Preconditions.checkState(!routedTables.isEmpty(), "no table route info");
        Collection<DataNode> result = new LinkedList<>();
        for (String each : routedTables) {
            result.add(new DataNode(routedDataSource, each));
        }
        return result;
    }

    private void reviseShardingConditions(final ShardingCondition each, final Collection<DataNode> dataNodes) {
        if (each instanceof InsertShardingCondition) {
            ((InsertShardingCondition) each).getDataNodes().addAll(dataNodes);
        }
    }
}

分析到此,Sharding-JDBC 3.1.0版本可以支持分片键不存在于SQL中和数据表结构中的使用场景。但3.1.0版本还有一个比较恶心的地方,Sharding-JDBC在初始化时,会连接数据库获取数据表的元数据,包括需要水平切分的表和不需水平切分的表。代码如下:

/*
 * Copyright 2016-2018 shardingsphere.io.
 * <p>
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 * </p>
 */

package io.shardingsphere.core.metadata.table.executor;

import com.google.common.base.Optional;
import io.shardingsphere.core.exception.ShardingException;
import io.shardingsphere.core.executor.ShardingExecuteEngine;
import io.shardingsphere.core.metadata.datasource.DataSourceMetaData;
import io.shardingsphere.core.metadata.datasource.ShardingDataSourceMetaData;
import io.shardingsphere.core.metadata.table.TableMetaData;
import io.shardingsphere.core.rule.ShardingRule;
import io.shardingsphere.core.rule.TableRule;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.Collection;
import java.util.HashMap;
import java.util.LinkedHashSet;
import java.util.Map;

/**
 * Table meta data initializer.
 *
 * @author zhangliang
 */
public final class TableMetaDataInitializer {

    private final ShardingDataSourceMetaData shardingDataSourceMetaData;

    private final TableMetaDataConnectionManager connectionManager;

    private final TableMetaDataLoader tableMetaDataLoader;

    public TableMetaDataInitializer(final ShardingDataSourceMetaData shardingDataSourceMetaData, final ShardingExecuteEngine executeEngine,
                                    final TableMetaDataConnectionManager connectionManager, final int maxConnectionsSizePerQuery, final boolean isCheckingMetaData) {
        this.shardingDataSourceMetaData = shardingDataSourceMetaData;
        this.connectionManager = connectionManager;
        tableMetaDataLoader = new TableMetaDataLoader(shardingDataSourceMetaData, executeEngine, connectionManager, maxConnectionsSizePerQuery, isCheckingMetaData);
    }

    /**
     * Load all table meta data.
     *
     * @param shardingRule sharding rule
     * @return all table meta data
     */
    public Map<String, TableMetaData> load(final ShardingRule shardingRule) {
        Map<String, TableMetaData> result = new HashMap<>();
        try {
            // 加载需要水平切分的表元数据
            result.putAll(loadShardingTables(shardingRule));
            // 加载不需水平切分的表元数据,如果数据库中表数量很大,这里耗时很久...
            result.putAll(loadDefaultTables(shardingRule));
        } catch (final SQLException ex) {
            throw new ShardingException(ex);
        }
        return result;
    }

    private Map<String, TableMetaData> loadShardingTables(final ShardingRule shardingRule) throws SQLException {
        Map<String, TableMetaData> result = new HashMap<>(shardingRule.getTableRules().size(), 1);
        for (TableRule each : shardingRule.getTableRules()) {
            result.put(each.getLogicTable(), tableMetaDataLoader.load(each.getLogicTable(), shardingRule));
        }
        return result;
    }

    private Map<String, TableMetaData> loadDefaultTables(final ShardingRule shardingRule) throws SQLException {
        Map<String, TableMetaData> result = new HashMap<>(shardingRule.getTableRules().size(), 1);
        Optional<String> actualDefaultDataSourceName = shardingRule.findActualDefaultDataSourceName();
        if (actualDefaultDataSourceName.isPresent()) {
            for (String each : getAllTableNames(actualDefaultDataSourceName.get())) {
                result.put(each, tableMetaDataLoader.load(each, shardingRule));
            }
        }
        return result;
    }

    /**
     * 连接数据库,获取所有表元数据
     *
     * @param dataSourceName 数据源名称
     * @return 所有数据表元数据列表
     * @throws SQLException
     */
    private Collection<String> getAllTableNames(final String dataSourceName) throws SQLException {
        Collection<String> result = new LinkedHashSet<>();
        DataSourceMetaData dataSourceMetaData = shardingDataSourceMetaData.getActualDataSourceMetaData(dataSourceName);
        String catalog = null == dataSourceMetaData ? null : dataSourceMetaData.getSchemeName();
        try (Connection connection = connectionManager.getConnection(dataSourceName);
             ResultSet resultSet = connection.getMetaData().getTables(catalog, null, null, new String[]{"TABLE"})) {
            while (resultSet.next()) {
                String tableName = resultSet.getString("TABLE_NAME");
                if (!tableName.contains("$")) {
                    result.add(tableName);
                }
            }
        }
        return result;
    }
}

文章永久链接:https://tech.souyunku.com/?p=15484

赞(62) 打赏



版权归原创作者所有,任何形式转载请联系作者;搜云库 » Sharding-JDBC(三)3.1.0版本实践

本站:免责声明!

评论 抢沙发

一个专注于Java技术系列文章的技术分享网站

觉得文章有用就打赏一下文章作者

微信扫一扫打赏

微信扫一扫打赏